From 8ee9cbaf607f7077d7ccc6b7d288f75cf55cf9b8 Mon Sep 17 00:00:00 2001
From: Lokathor <zefria@gmail.com>
Date: Mon, 21 Sep 2020 20:03:54 -0600
Subject: [PATCH 001/249] Create README.md

---
 README.md | 5 +++++
 1 file changed, 5 insertions(+)
 create mode 100644 README.md

diff --git a/README.md b/README.md
new file mode 100644
index 000000000000..a4efc29293d6
--- /dev/null
+++ b/README.md
@@ -0,0 +1,5 @@
+# stdsimd
+
+Code repository for the [Portable SIMD Project Group](https://github.com/rust-lang/project-portable-simd).
+
+More coming soon!

From e288fb8cba2b28ec9db6c891d719acf567f9cd72 Mon Sep 17 00:00:00 2001
From: Lokathor <zefria@gmail.com>
Date: Mon, 21 Sep 2020 21:56:09 -0600
Subject: [PATCH 002/249] start a guide

---
 beginners-guide.md | 28 ++++++++++++++++++++++++++++
 1 file changed, 28 insertions(+)
 create mode 100644 beginners-guide.md

diff --git a/beginners-guide.md b/beginners-guide.md
new file mode 100644
index 000000000000..3641624ffb02
--- /dev/null
+++ b/beginners-guide.md
@@ -0,0 +1,28 @@
+
+# Beginner's Guide To SIMD
+
+Hello and welcome to our SIMD basics guide!
+
+Because SIMD is a subject that many programmers haven't worked with before, we thought that it's best to outline some terms and other basics for you to get started with.
+
+## Quick Background
+
+**SIMD** stands for *Single Instruction, Multiple Data*. In other words, SIMD is when the CPU performs a single action on more that one logical pieces of data at the same time. Instead of adding two registers that each contain one `f32` value and getting an `f32` as the result, you might add two registers that each contain `f32x4` (128 bits of data) and then you get an `f32x4` as the output.
+
+This might seem a tiny bit weird at first, but there's a good reason for it. Back in the day, as CPUs got faster and faster, eventually they got so fast that the CPU would just melt itself. The heat management (heat sinks, fans, etc) simply couldn't keep up with how much electricity was going through the metal. Two main strategies were developed to help get around the limits of physics.
+* One of them you're probably familiar with: Multi-core processors. By giving a processor more than one core, each core can do its own work, and because they're physically distant (at least on the CPU's scale) the heat can still be managed. Unfortunately, not all tasks can just be split up across cores in an efficient way.
+* The second strategy is SIMD. If you can't make the register go any faster, you can still make the register *wider*. This lets you process more data at a time, which is *almost* as good as just having a faster CPU. As with multi-core programming, SIMD doesn't fit every kind of task, so you have to know when it will improve your program.
+
+## Terms
+
+SIMD has a few special vocabulary terms you should know:
+
+* **Vector:** A SIMD value is called a vector. This shouldn't be confused with the `Vec<T>` type. A SIMD vector has a fixed size, known at compile time. All of the elements within the vector are of the same type. This makes vectors *similar to* arrays. One difference is that a vector is generally aligned to its *entire* size (eg: 16 bytes, 32 bytes, etc), not just the size of an individual element. Sometimes vector data is called "packed" data.
+
+* **Lane:** A single element position within a vector is called a lane. If you have `N` lanes available then they're numbered from `0` to `N-1` when referring to them, again like an array. The biggest difference between an array element and a vector lane is that it is *relatively costly* to access an individual lane value. Generally, the vector has to be pushed out of register onto the stack, then an individual lane is accessed while it's on the stack. For this reason, when working with SIMD you should avoid reading or writing the value of an individual lane during hot loops.
+
+* **Bit Widths:** When talking about SIMD, the bit widths used are the bit size of the vectors involved, *not* the individual elements. So "128-bit SIMD" has 128-bit vectors, and that might be `f32x4`, `i32x4`, `i16x8`, or other variations. While 128-bit SIMD is the most common, there's also 64-bit, 256-bit, and even 512-bit on the newest CPUs.
+
+* **Vertical:** When an operation is "vertical", each lane processes individually without regard to the other lane in the same vector. For example, a "vertical add" between two vectors would add lane 0 in `a` with lane 0 in `b`, with the total in lane 0 of `out`, and then the same thing for lanes 1, 2, etc. Most SIMD operations are vertical operations, so if your problem is a vertical problem then you can probably solve it with SIMD.
+
+* **Horizontal:** When an operation is "horizontal", the lanes within a single vector interact in some way. A "horizontal add" might add up lane 0 of `a` with lane 1 of `a`, producing the output into lane 0 of `out`.

From 8204872d0147ef10c6ad4d5544eaaf6dcd7b6444 Mon Sep 17 00:00:00 2001
From: Lokathor <zefria@gmail.com>
Date: Tue, 22 Sep 2020 00:39:38 -0600
Subject: [PATCH 003/249] typo

---
 beginners-guide.md | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/beginners-guide.md b/beginners-guide.md
index 3641624ffb02..a2fda9b9b94d 100644
--- a/beginners-guide.md
+++ b/beginners-guide.md
@@ -23,6 +23,6 @@ SIMD has a few special vocabulary terms you should know:
 
 * **Bit Widths:** When talking about SIMD, the bit widths used are the bit size of the vectors involved, *not* the individual elements. So "128-bit SIMD" has 128-bit vectors, and that might be `f32x4`, `i32x4`, `i16x8`, or other variations. While 128-bit SIMD is the most common, there's also 64-bit, 256-bit, and even 512-bit on the newest CPUs.
 
-* **Vertical:** When an operation is "vertical", each lane processes individually without regard to the other lane in the same vector. For example, a "vertical add" between two vectors would add lane 0 in `a` with lane 0 in `b`, with the total in lane 0 of `out`, and then the same thing for lanes 1, 2, etc. Most SIMD operations are vertical operations, so if your problem is a vertical problem then you can probably solve it with SIMD.
+* **Vertical:** When an operation is "vertical", each lane processes individually without regard to the other lanes in the same vector. For example, a "vertical add" between two vectors would add lane 0 in `a` with lane 0 in `b`, with the total in lane 0 of `out`, and then the same thing for lanes 1, 2, etc. Most SIMD operations are vertical operations, so if your problem is a vertical problem then you can probably solve it with SIMD.
 
 * **Horizontal:** When an operation is "horizontal", the lanes within a single vector interact in some way. A "horizontal add" might add up lane 0 of `a` with lane 1 of `a`, producing the output into lane 0 of `out`.

From f6e6387fccd1f7c37c353ab4d674d641e21d4315 Mon Sep 17 00:00:00 2001
From: Lokathor <zefria@gmail.com>
Date: Tue, 22 Sep 2020 00:40:35 -0600
Subject: [PATCH 004/249] consistent wording.

---
 beginners-guide.md | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/beginners-guide.md b/beginners-guide.md
index a2fda9b9b94d..d5010e9847ef 100644
--- a/beginners-guide.md
+++ b/beginners-guide.md
@@ -25,4 +25,4 @@ SIMD has a few special vocabulary terms you should know:
 
 * **Vertical:** When an operation is "vertical", each lane processes individually without regard to the other lanes in the same vector. For example, a "vertical add" between two vectors would add lane 0 in `a` with lane 0 in `b`, with the total in lane 0 of `out`, and then the same thing for lanes 1, 2, etc. Most SIMD operations are vertical operations, so if your problem is a vertical problem then you can probably solve it with SIMD.
 
-* **Horizontal:** When an operation is "horizontal", the lanes within a single vector interact in some way. A "horizontal add" might add up lane 0 of `a` with lane 1 of `a`, producing the output into lane 0 of `out`.
+* **Horizontal:** When an operation is "horizontal", the lanes within a single vector interact in some way. A "horizontal add" might add up lane 0 of `a` with lane 1 of `a`, with the total in lane 0 of `out`.

From 011aafea16428443076a01be98e53018706a7964 Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Tue, 22 Sep 2020 19:41:42 -0400
Subject: [PATCH 005/249] Add initial type implementations

---
 .gitignore                           |   2 +
 Cargo.toml                           |   5 +
 crates/core_simd/Cargo.toml          |   5 +
 crates/core_simd/src/lib.rs          |  36 +++++++
 crates/core_simd/src/macros.rs       | 140 +++++++++++++++++++++++++++
 crates/core_simd/src/type_f32x16.rs  |   9 ++
 crates/core_simd/src/type_f32x2.rs   |   1 +
 crates/core_simd/src/type_f32x4.rs   |   7 ++
 crates/core_simd/src/type_f32x8.rs   |   7 ++
 crates/core_simd/src/type_f64x2.rs   |   7 ++
 crates/core_simd/src/type_f64x4.rs   |   7 ++
 crates/core_simd/src/type_f64x8.rs   |   9 ++
 crates/core_simd/src/type_i128x2.rs  |   7 ++
 crates/core_simd/src/type_i128x4.rs  |   9 ++
 crates/core_simd/src/type_i16x16.rs  |   7 ++
 crates/core_simd/src/type_i16x2.rs   |   1 +
 crates/core_simd/src/type_i16x32.rs  |   9 ++
 crates/core_simd/src/type_i16x4.rs   |   1 +
 crates/core_simd/src/type_i16x8.rs   |   7 ++
 crates/core_simd/src/type_i32x16.rs  |   9 ++
 crates/core_simd/src/type_i32x2.rs   |   1 +
 crates/core_simd/src/type_i32x4.rs   |   7 ++
 crates/core_simd/src/type_i32x8.rs   |   7 ++
 crates/core_simd/src/type_i64x2.rs   |   7 ++
 crates/core_simd/src/type_i64x4.rs   |   7 ++
 crates/core_simd/src/type_i64x8.rs   |   9 ++
 crates/core_simd/src/type_i8x16.rs   |   7 ++
 crates/core_simd/src/type_i8x2.rs    |   1 +
 crates/core_simd/src/type_i8x32.rs   |   7 ++
 crates/core_simd/src/type_i8x4.rs    |   1 +
 crates/core_simd/src/type_i8x64.rs   |   9 ++
 crates/core_simd/src/type_i8x8.rs    |   1 +
 crates/core_simd/src/type_isizex2.rs |   7 ++
 crates/core_simd/src/type_isizex4.rs |  13 +++
 crates/core_simd/src/type_isizex8.rs |  15 +++
 crates/core_simd/src/type_u128x2.rs  |   7 ++
 crates/core_simd/src/type_u128x4.rs  |   9 ++
 crates/core_simd/src/type_u16x16.rs  |   7 ++
 crates/core_simd/src/type_u16x2.rs   |   1 +
 crates/core_simd/src/type_u16x32.rs  |   9 ++
 crates/core_simd/src/type_u16x4.rs   |   1 +
 crates/core_simd/src/type_u16x8.rs   |   7 ++
 crates/core_simd/src/type_u32x16.rs  |   9 ++
 crates/core_simd/src/type_u32x2.rs   |   1 +
 crates/core_simd/src/type_u32x4.rs   |   7 ++
 crates/core_simd/src/type_u32x8.rs   |   7 ++
 crates/core_simd/src/type_u64x2.rs   |   7 ++
 crates/core_simd/src/type_u64x4.rs   |   7 ++
 crates/core_simd/src/type_u64x8.rs   |   9 ++
 crates/core_simd/src/type_u8x16.rs   |   7 ++
 crates/core_simd/src/type_u8x2.rs    |   1 +
 crates/core_simd/src/type_u8x32.rs   |   7 ++
 crates/core_simd/src/type_u8x4.rs    |   1 +
 crates/core_simd/src/type_u8x64.rs   |   9 ++
 crates/core_simd/src/type_u8x8.rs    |   1 +
 crates/core_simd/src/type_usizex2.rs |   7 ++
 crates/core_simd/src/type_usizex4.rs |  13 +++
 crates/core_simd/src/type_usizex8.rs |  15 +++
 58 files changed, 533 insertions(+)
 create mode 100644 .gitignore
 create mode 100644 Cargo.toml
 create mode 100644 crates/core_simd/Cargo.toml
 create mode 100644 crates/core_simd/src/lib.rs
 create mode 100644 crates/core_simd/src/macros.rs
 create mode 100644 crates/core_simd/src/type_f32x16.rs
 create mode 100644 crates/core_simd/src/type_f32x2.rs
 create mode 100644 crates/core_simd/src/type_f32x4.rs
 create mode 100644 crates/core_simd/src/type_f32x8.rs
 create mode 100644 crates/core_simd/src/type_f64x2.rs
 create mode 100644 crates/core_simd/src/type_f64x4.rs
 create mode 100644 crates/core_simd/src/type_f64x8.rs
 create mode 100644 crates/core_simd/src/type_i128x2.rs
 create mode 100644 crates/core_simd/src/type_i128x4.rs
 create mode 100644 crates/core_simd/src/type_i16x16.rs
 create mode 100644 crates/core_simd/src/type_i16x2.rs
 create mode 100644 crates/core_simd/src/type_i16x32.rs
 create mode 100644 crates/core_simd/src/type_i16x4.rs
 create mode 100644 crates/core_simd/src/type_i16x8.rs
 create mode 100644 crates/core_simd/src/type_i32x16.rs
 create mode 100644 crates/core_simd/src/type_i32x2.rs
 create mode 100644 crates/core_simd/src/type_i32x4.rs
 create mode 100644 crates/core_simd/src/type_i32x8.rs
 create mode 100644 crates/core_simd/src/type_i64x2.rs
 create mode 100644 crates/core_simd/src/type_i64x4.rs
 create mode 100644 crates/core_simd/src/type_i64x8.rs
 create mode 100644 crates/core_simd/src/type_i8x16.rs
 create mode 100644 crates/core_simd/src/type_i8x2.rs
 create mode 100644 crates/core_simd/src/type_i8x32.rs
 create mode 100644 crates/core_simd/src/type_i8x4.rs
 create mode 100644 crates/core_simd/src/type_i8x64.rs
 create mode 100644 crates/core_simd/src/type_i8x8.rs
 create mode 100644 crates/core_simd/src/type_isizex2.rs
 create mode 100644 crates/core_simd/src/type_isizex4.rs
 create mode 100644 crates/core_simd/src/type_isizex8.rs
 create mode 100644 crates/core_simd/src/type_u128x2.rs
 create mode 100644 crates/core_simd/src/type_u128x4.rs
 create mode 100644 crates/core_simd/src/type_u16x16.rs
 create mode 100644 crates/core_simd/src/type_u16x2.rs
 create mode 100644 crates/core_simd/src/type_u16x32.rs
 create mode 100644 crates/core_simd/src/type_u16x4.rs
 create mode 100644 crates/core_simd/src/type_u16x8.rs
 create mode 100644 crates/core_simd/src/type_u32x16.rs
 create mode 100644 crates/core_simd/src/type_u32x2.rs
 create mode 100644 crates/core_simd/src/type_u32x4.rs
 create mode 100644 crates/core_simd/src/type_u32x8.rs
 create mode 100644 crates/core_simd/src/type_u64x2.rs
 create mode 100644 crates/core_simd/src/type_u64x4.rs
 create mode 100644 crates/core_simd/src/type_u64x8.rs
 create mode 100644 crates/core_simd/src/type_u8x16.rs
 create mode 100644 crates/core_simd/src/type_u8x2.rs
 create mode 100644 crates/core_simd/src/type_u8x32.rs
 create mode 100644 crates/core_simd/src/type_u8x4.rs
 create mode 100644 crates/core_simd/src/type_u8x64.rs
 create mode 100644 crates/core_simd/src/type_u8x8.rs
 create mode 100644 crates/core_simd/src/type_usizex2.rs
 create mode 100644 crates/core_simd/src/type_usizex4.rs
 create mode 100644 crates/core_simd/src/type_usizex8.rs

diff --git a/.gitignore b/.gitignore
new file mode 100644
index 000000000000..96ef6c0b944e
--- /dev/null
+++ b/.gitignore
@@ -0,0 +1,2 @@
+/target
+Cargo.lock
diff --git a/Cargo.toml b/Cargo.toml
new file mode 100644
index 000000000000..f3538db75590
--- /dev/null
+++ b/Cargo.toml
@@ -0,0 +1,5 @@
+[workspace]
+
+members = [
+    "crates/core_simd",
+]
diff --git a/crates/core_simd/Cargo.toml b/crates/core_simd/Cargo.toml
new file mode 100644
index 000000000000..d4aef6f059c5
--- /dev/null
+++ b/crates/core_simd/Cargo.toml
@@ -0,0 +1,5 @@
+[package]
+name = "core_simd"
+version = "0.1.0"
+authors = ["Caleb Zulawski <caleb.zulawski@gmail.com>"]
+edition = "2018"
diff --git a/crates/core_simd/src/lib.rs b/crates/core_simd/src/lib.rs
new file mode 100644
index 000000000000..8db640157eed
--- /dev/null
+++ b/crates/core_simd/src/lib.rs
@@ -0,0 +1,36 @@
+#![feature(repr_simd)]
+
+#[macro_use]
+mod macros;
+
+macro_rules! import_types {
+    { $($mod:ident,)* } => {
+        $(
+        mod $mod;
+        pub use $mod::*;
+        )*
+    }
+}
+
+import_types! {
+    type_u8x2,   type_u8x4,   type_u8x8,   type_u8x16,   type_u8x32,   type_u8x64,
+    type_i8x2,   type_i8x4,   type_i8x8,   type_i8x16,   type_i8x32,   type_i8x64,
+    type_u16x2,  type_u16x4,  type_u16x8,  type_u16x16,  type_u16x32,
+    type_i16x2,  type_i16x4,  type_i16x8,  type_i16x16,  type_i16x32,
+    type_u32x2,  type_u32x4,  type_u32x8,  type_u32x16,
+    type_i32x2,  type_i32x4,  type_i32x8,  type_i32x16,
+    type_u64x2,  type_u64x4,  type_u64x8,
+    type_i64x2,  type_i64x4,  type_i64x8,
+    type_u128x2, type_u128x4,
+    type_i128x2, type_i128x4,
+}
+
+import_types! {
+    type_usizex2, type_usizex4, type_usizex8,
+    type_isizex2, type_isizex4, type_isizex8,
+}
+
+import_types! {
+    type_f32x2, type_f32x4, type_f32x8, type_f32x16,
+    type_f64x2, type_f64x4, type_f64x8,
+}
diff --git a/crates/core_simd/src/macros.rs b/crates/core_simd/src/macros.rs
new file mode 100644
index 000000000000..70b19e5f4dbc
--- /dev/null
+++ b/crates/core_simd/src/macros.rs
@@ -0,0 +1,140 @@
+macro_rules! from_aligned {
+    { unsafe $from:ty => $to:ty } => {
+        impl core::convert::From<$from> for $to {
+            #[inline]
+            fn from(value: $from) -> $to {
+                assert_eq!(core::mem::size_of::<$from>(), core::mem::size_of::<$to>());
+                assert!(core::mem::align_of::<$from>() >= core::mem::align_of::<$to>());
+                unsafe { core::mem::transmute(value) }
+            }
+        }
+    };
+    { unsafe $a:ty |bidirectional| $b:ty } => {
+        from_aligned!{ unsafe $a => $b }
+        from_aligned!{ unsafe $b => $a }
+    }
+}
+
+macro_rules! from_unaligned {
+    { unsafe $from:ty => $to:ty } => {
+        impl core::convert::From<$from> for $to {
+            #[inline]
+            fn from(value: $from) -> $to {
+                assert_eq!(core::mem::size_of::<$from>(), core::mem::size_of::<$to>());
+                unsafe { (&value as *const $from as *const $to).read_unaligned() }
+            }
+        }
+    }
+}
+
+macro_rules! define_type {
+    { struct $name:ident([$type:ty; $lanes:tt]); } => {
+        define_type! { @impl $name [$type; $lanes] }
+
+        // array references
+        impl AsRef<[$type; $lanes]> for $name {
+            #[inline]
+            fn as_ref(&self) -> &[$type; $lanes] {
+                unsafe { &*(self as *const _ as *const _) }
+            }
+        }
+
+        impl AsMut<[$type; $lanes]> for $name {
+            #[inline]
+            fn as_mut(&mut self) -> &mut [$type; $lanes] {
+                unsafe { &mut *(self as *mut _ as *mut _) }
+            }
+        }
+
+        // slice references
+        impl AsRef<[$type]> for $name {
+            #[inline]
+            fn as_ref(&self) -> &[$type] {
+                AsRef::<[$type; $lanes]>::as_ref(self)
+            }
+        }
+
+        impl AsMut<[$type]> for $name {
+            #[inline]
+            fn as_mut(&mut self) -> &mut [$type] {
+                AsMut::<[$type; $lanes]>::as_mut(self)
+            }
+        }
+
+        // vector to array
+        from_aligned! { unsafe $name => [$type; $lanes] }
+
+        // array to vector
+        from_unaligned! { unsafe [$type; $lanes] => $name }
+
+        // splat
+        impl From<$type> for $name {
+            fn from(value: $type) -> Self {
+                Self::splat(value)
+            }
+        }
+    };
+    { @impl $name:ident [$type:ty; 1] } => {
+        define_type! { @impl $name | $type | $type, | v0, }
+    };
+    { @impl $name:ident [$type:ty; 2] } => {
+        define_type! { @impl $name | $type | $type, $type, | v0, v1, }
+    };
+    { @impl $name:ident [$type:ty; 4] } => {
+        define_type! { @impl $name | $type |
+            $type, $type, $type, $type, |
+            v0, v1, v2, v3,
+        }
+    };
+    { @impl $name:ident [$type:ty; 8] } => {
+        define_type! { @impl $name | $type |
+            $type, $type, $type, $type, $type, $type, $type, $type, |
+            v0, v1, v2, v3, v4, v5, v6, v7,
+        }
+    };
+    { @impl $name:ident [$type:ty; 16] } => {
+        define_type! { @impl $name | $type |
+            $type, $type, $type, $type, $type, $type, $type, $type, $type, $type, $type, $type, $type, $type, $type, $type, |
+            v0, v1, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13, v14, v15,
+        }
+    };
+    { @impl $name:ident [$type:ty; 32] } => {
+        define_type! { @impl $name | $type |
+            $type, $type, $type, $type, $type, $type, $type, $type, $type, $type, $type, $type, $type, $type, $type, $type,
+            $type, $type, $type, $type, $type, $type, $type, $type, $type, $type, $type, $type, $type, $type, $type, $type, |
+            v0,  v1,  v2,  v3,  v4,  v5,  v6,  v7,  v8,  v9,  v10, v11, v12, v13, v14, v15,
+            v16, v17, v18, v19, v20, v21, v22, v23, v24, v25, v26, v27, v28, v29, v30, v31,
+        }
+    };
+    { @impl $name:ident [$type:ty; 64] } => {
+        define_type! { @impl $name | $type |
+            $type, $type, $type, $type, $type, $type, $type, $type, $type, $type, $type, $type, $type, $type, $type, $type,
+            $type, $type, $type, $type, $type, $type, $type, $type, $type, $type, $type, $type, $type, $type, $type, $type,
+            $type, $type, $type, $type, $type, $type, $type, $type, $type, $type, $type, $type, $type, $type, $type, $type,
+            $type, $type, $type, $type, $type, $type, $type, $type, $type, $type, $type, $type, $type, $type, $type, $type, |
+            v0,  v1,  v2,  v3,  v4,  v5,  v6,  v7,  v8,  v9,  v10, v11, v12, v13, v14, v15,
+            v16, v17, v18, v19, v20, v21, v22, v23, v24, v25, v26, v27, v28, v29, v30, v31,
+            v32, v33, v34, v35, v36, v37, v38, v39, v40, v41, v42, v43, v44, v45, v46, v47,
+            v48, v49, v50, v51, v52, v53, v54, v55, v56, v57, v58, v59, v60, v61, v62, v63,
+        }
+    };
+    { @impl $name:ident | $type:ty | $($itype:ty,)* | $($ivar:ident,)* } => {
+        #[allow(non_camel_case_types)]
+        #[derive(Copy, Clone, Debug, Default, PartialEq, PartialOrd)]
+        #[repr(simd)]
+        pub struct $name($($itype),*);
+
+        impl $name {
+            #[inline]
+            pub fn splat(value: $type) -> Self {
+                Self($(value as $itype),*)
+            }
+
+            #[allow(clippy::too_many_arguments)]
+            #[inline]
+            pub fn new($($ivar: $itype),*) -> Self {
+                Self($($ivar),*)
+            }
+        }
+    }
+}
diff --git a/crates/core_simd/src/type_f32x16.rs b/crates/core_simd/src/type_f32x16.rs
new file mode 100644
index 000000000000..f001b9c42df4
--- /dev/null
+++ b/crates/core_simd/src/type_f32x16.rs
@@ -0,0 +1,9 @@
+define_type! { struct f32x16([f32; 16]); }
+
+/*
+#[cfg(target_arch = "x86")]
+from_aligned! { unsafe f32x16 |bidirectional| core::arch::x86::__m512 }
+
+#[cfg(target_arch = "x86_64")]
+from_aligned! { unsafe f32x16 |bidirectional| core::arch::x86_64::__m512 }
+*/
diff --git a/crates/core_simd/src/type_f32x2.rs b/crates/core_simd/src/type_f32x2.rs
new file mode 100644
index 000000000000..e6691523ea9f
--- /dev/null
+++ b/crates/core_simd/src/type_f32x2.rs
@@ -0,0 +1 @@
+define_type! { struct f32x2([f32; 2]); }
diff --git a/crates/core_simd/src/type_f32x4.rs b/crates/core_simd/src/type_f32x4.rs
new file mode 100644
index 000000000000..4fd937425c50
--- /dev/null
+++ b/crates/core_simd/src/type_f32x4.rs
@@ -0,0 +1,7 @@
+define_type! { struct f32x4([f32; 4]); }
+
+#[cfg(target_arch = "x86")]
+from_aligned! { unsafe f32x4 |bidirectional| core::arch::x86::__m128 }
+
+#[cfg(target_arch = "x86_64")]
+from_aligned! { unsafe f32x4 |bidirectional| core::arch::x86_64::__m128 }
diff --git a/crates/core_simd/src/type_f32x8.rs b/crates/core_simd/src/type_f32x8.rs
new file mode 100644
index 000000000000..a6754d490c1a
--- /dev/null
+++ b/crates/core_simd/src/type_f32x8.rs
@@ -0,0 +1,7 @@
+define_type! { struct f32x8([f32; 8]); }
+
+#[cfg(target_arch = "x86")]
+from_aligned! { unsafe f32x8 |bidirectional| core::arch::x86::__m256 }
+
+#[cfg(target_arch = "x86_64")]
+from_aligned! { unsafe f32x8 |bidirectional| core::arch::x86_64::__m256 }
diff --git a/crates/core_simd/src/type_f64x2.rs b/crates/core_simd/src/type_f64x2.rs
new file mode 100644
index 000000000000..227ef951434d
--- /dev/null
+++ b/crates/core_simd/src/type_f64x2.rs
@@ -0,0 +1,7 @@
+define_type! { struct f64x2([f64; 2]); }
+
+#[cfg(target_arch = "x86")]
+from_aligned! { unsafe f64x2 |bidirectional| core::arch::x86::__m128d }
+
+#[cfg(target_arch = "x86_64")]
+from_aligned! { unsafe f64x2 |bidirectional| core::arch::x86_64::__m128d }
diff --git a/crates/core_simd/src/type_f64x4.rs b/crates/core_simd/src/type_f64x4.rs
new file mode 100644
index 000000000000..8e9debd8bfde
--- /dev/null
+++ b/crates/core_simd/src/type_f64x4.rs
@@ -0,0 +1,7 @@
+define_type! { struct f64x4([f64; 4]); }
+
+#[cfg(target_arch = "x86")]
+from_aligned! { unsafe f64x4 |bidirectional| core::arch::x86::__m256d }
+
+#[cfg(target_arch = "x86_64")]
+from_aligned! { unsafe f64x4 |bidirectional| core::arch::x86_64::__m256d }
diff --git a/crates/core_simd/src/type_f64x8.rs b/crates/core_simd/src/type_f64x8.rs
new file mode 100644
index 000000000000..8aa3812b4f43
--- /dev/null
+++ b/crates/core_simd/src/type_f64x8.rs
@@ -0,0 +1,9 @@
+define_type! { struct f64x8([f64; 8]); }
+
+/*
+#[cfg(target_arch = "x86")]
+from_aligned! { unsafe f64x8 |bidirectional| core::arch::x86::__m512d }
+
+#[cfg(target_arch = "x86_64")]
+from_aligned! { unsafe f64x8 |bidirectional| core::arch::x86_64::__m512d }
+*/
diff --git a/crates/core_simd/src/type_i128x2.rs b/crates/core_simd/src/type_i128x2.rs
new file mode 100644
index 000000000000..cb1be5a9866c
--- /dev/null
+++ b/crates/core_simd/src/type_i128x2.rs
@@ -0,0 +1,7 @@
+define_type! { struct i128x2([i128; 2]); }
+
+#[cfg(target_arch = "x86")]
+from_aligned! { unsafe i128x2 |bidirectional| core::arch::x86::__m256i }
+
+#[cfg(target_arch = "x86_64")]
+from_aligned! { unsafe i128x2 |bidirectional| core::arch::x86_64::__m256i }
diff --git a/crates/core_simd/src/type_i128x4.rs b/crates/core_simd/src/type_i128x4.rs
new file mode 100644
index 000000000000..8888d4205f86
--- /dev/null
+++ b/crates/core_simd/src/type_i128x4.rs
@@ -0,0 +1,9 @@
+define_type! { struct i128x4([i128; 4]); }
+
+/*
+#[cfg(target_arch = "x86")]
+from_aligned! { unsafe i128x4 |bidirectional| core::arch::x86::__m512i }
+
+#[cfg(target_arch = "x86_64")]
+from_aligned! { unsafe i128x4 |bidirectional| core::arch::x86_64::__m512i }
+*/
diff --git a/crates/core_simd/src/type_i16x16.rs b/crates/core_simd/src/type_i16x16.rs
new file mode 100644
index 000000000000..ec52c4857763
--- /dev/null
+++ b/crates/core_simd/src/type_i16x16.rs
@@ -0,0 +1,7 @@
+define_type! { struct i16x16([i16; 16]); }
+
+#[cfg(target_arch = "x86")]
+from_aligned! { unsafe i16x16 |bidirectional| core::arch::x86::__m256i }
+
+#[cfg(target_arch = "x86_64")]
+from_aligned! { unsafe i16x16 |bidirectional| core::arch::x86_64::__m256i }
diff --git a/crates/core_simd/src/type_i16x2.rs b/crates/core_simd/src/type_i16x2.rs
new file mode 100644
index 000000000000..313ba334da19
--- /dev/null
+++ b/crates/core_simd/src/type_i16x2.rs
@@ -0,0 +1 @@
+define_type! { struct i16x2([i16; 2]); }
diff --git a/crates/core_simd/src/type_i16x32.rs b/crates/core_simd/src/type_i16x32.rs
new file mode 100644
index 000000000000..53679e2ac5c4
--- /dev/null
+++ b/crates/core_simd/src/type_i16x32.rs
@@ -0,0 +1,9 @@
+define_type! { struct i16x32([i16; 32]); }
+
+/*
+#[cfg(target_arch = "x86")]
+from_aligned! { unsafe u8x32 |bidirectional| core::arch::x86::__m512i }
+
+#[cfg(target_arch = "x86_64")]
+from_aligned! { unsafe u8x32 |bidirectional| core::arch::x86_64::__m512i }
+*/
diff --git a/crates/core_simd/src/type_i16x4.rs b/crates/core_simd/src/type_i16x4.rs
new file mode 100644
index 000000000000..f69f5dc2acda
--- /dev/null
+++ b/crates/core_simd/src/type_i16x4.rs
@@ -0,0 +1 @@
+define_type! { struct i16x4([i16; 4]); }
diff --git a/crates/core_simd/src/type_i16x8.rs b/crates/core_simd/src/type_i16x8.rs
new file mode 100644
index 000000000000..b1a3100dd6e2
--- /dev/null
+++ b/crates/core_simd/src/type_i16x8.rs
@@ -0,0 +1,7 @@
+define_type! { struct i16x8([i16; 8]); }
+
+#[cfg(target_arch = "x86")]
+from_aligned! { unsafe i16x8 |bidirectional| core::arch::x86::__m128i }
+
+#[cfg(target_arch = "x86_64")]
+from_aligned! { unsafe i16x8 |bidirectional| core::arch::x86_64::__m128i }
diff --git a/crates/core_simd/src/type_i32x16.rs b/crates/core_simd/src/type_i32x16.rs
new file mode 100644
index 000000000000..5b7edb0f51bb
--- /dev/null
+++ b/crates/core_simd/src/type_i32x16.rs
@@ -0,0 +1,9 @@
+define_type! { struct i32x16([i32; 16]); }
+
+/*
+#[cfg(target_arch = "x86")]
+from_aligned! { unsafe u32x16 |bidirectional| core::arch::x86::__m512i }
+
+#[cfg(target_arch = "x86_64")]
+from_aligned! { unsafe u32x16 |bidirectional| core::arch::x86_64::__m512i }
+*/
diff --git a/crates/core_simd/src/type_i32x2.rs b/crates/core_simd/src/type_i32x2.rs
new file mode 100644
index 000000000000..425b6fd3bca8
--- /dev/null
+++ b/crates/core_simd/src/type_i32x2.rs
@@ -0,0 +1 @@
+define_type! { struct i32x2([i32; 2]); }
diff --git a/crates/core_simd/src/type_i32x4.rs b/crates/core_simd/src/type_i32x4.rs
new file mode 100644
index 000000000000..d62fc3856340
--- /dev/null
+++ b/crates/core_simd/src/type_i32x4.rs
@@ -0,0 +1,7 @@
+define_type! { struct i32x4([i32; 4]); }
+
+#[cfg(target_arch = "x86")]
+from_aligned! { unsafe i32x4 |bidirectional| core::arch::x86::__m128i }
+
+#[cfg(target_arch = "x86_64")]
+from_aligned! { unsafe i32x4 |bidirectional| core::arch::x86_64::__m128i }
diff --git a/crates/core_simd/src/type_i32x8.rs b/crates/core_simd/src/type_i32x8.rs
new file mode 100644
index 000000000000..ee83854cee4e
--- /dev/null
+++ b/crates/core_simd/src/type_i32x8.rs
@@ -0,0 +1,7 @@
+define_type! { struct i32x8([i32; 8]); }
+
+#[cfg(target_arch = "x86")]
+from_aligned! { unsafe i32x8 |bidirectional| core::arch::x86::__m256i }
+
+#[cfg(target_arch = "x86_64")]
+from_aligned! { unsafe i32x8 |bidirectional| core::arch::x86_64::__m256i }
diff --git a/crates/core_simd/src/type_i64x2.rs b/crates/core_simd/src/type_i64x2.rs
new file mode 100644
index 000000000000..35af990ec649
--- /dev/null
+++ b/crates/core_simd/src/type_i64x2.rs
@@ -0,0 +1,7 @@
+define_type! { struct i64x2([i64; 2]); }
+
+#[cfg(target_arch = "x86")]
+from_aligned! { unsafe i64x2 |bidirectional| core::arch::x86::__m128i }
+
+#[cfg(target_arch = "x86_64")]
+from_aligned! { unsafe i64x2 |bidirectional| core::arch::x86_64::__m128i }
diff --git a/crates/core_simd/src/type_i64x4.rs b/crates/core_simd/src/type_i64x4.rs
new file mode 100644
index 000000000000..6f69a492ede9
--- /dev/null
+++ b/crates/core_simd/src/type_i64x4.rs
@@ -0,0 +1,7 @@
+define_type! { struct i64x4([i64; 4]); }
+
+#[cfg(target_arch = "x86")]
+from_aligned! { unsafe i64x4 |bidirectional| core::arch::x86::__m256i }
+
+#[cfg(target_arch = "x86_64")]
+from_aligned! { unsafe i64x4 |bidirectional| core::arch::x86_64::__m256i }
diff --git a/crates/core_simd/src/type_i64x8.rs b/crates/core_simd/src/type_i64x8.rs
new file mode 100644
index 000000000000..888c9b6612c0
--- /dev/null
+++ b/crates/core_simd/src/type_i64x8.rs
@@ -0,0 +1,9 @@
+define_type! { struct i64x8([i64; 8]); }
+
+/*
+#[cfg(target_arch = "x86")]
+from_aligned! { unsafe i64x8 |bidirectional| core::arch::x86::__m512i }
+
+#[cfg(target_arch = "x86_64")]
+from_aligned! { unsafe i64x8 |bidirectional| core::arch::x86_64::__m512i }
+*/
diff --git a/crates/core_simd/src/type_i8x16.rs b/crates/core_simd/src/type_i8x16.rs
new file mode 100644
index 000000000000..32ea4fa82934
--- /dev/null
+++ b/crates/core_simd/src/type_i8x16.rs
@@ -0,0 +1,7 @@
+define_type! { struct i8x16([i8; 16]); }
+
+#[cfg(target_arch = "x86")]
+from_aligned! { unsafe i8x16 |bidirectional| core::arch::x86::__m128i }
+
+#[cfg(target_arch = "x86_64")]
+from_aligned! { unsafe i8x16 |bidirectional| core::arch::x86_64::__m128i }
diff --git a/crates/core_simd/src/type_i8x2.rs b/crates/core_simd/src/type_i8x2.rs
new file mode 100644
index 000000000000..a36de2c5421d
--- /dev/null
+++ b/crates/core_simd/src/type_i8x2.rs
@@ -0,0 +1 @@
+define_type! { struct i8x2([i8; 2]); }
diff --git a/crates/core_simd/src/type_i8x32.rs b/crates/core_simd/src/type_i8x32.rs
new file mode 100644
index 000000000000..874fcdd1e96a
--- /dev/null
+++ b/crates/core_simd/src/type_i8x32.rs
@@ -0,0 +1,7 @@
+define_type! { struct i8x32([i8; 32]); }
+
+#[cfg(target_arch = "x86")]
+from_aligned! { unsafe i8x32 |bidirectional| core::arch::x86::__m256i }
+
+#[cfg(target_arch = "x86_64")]
+from_aligned! { unsafe i8x32 |bidirectional| core::arch::x86_64::__m256i }
diff --git a/crates/core_simd/src/type_i8x4.rs b/crates/core_simd/src/type_i8x4.rs
new file mode 100644
index 000000000000..5be83d5e0377
--- /dev/null
+++ b/crates/core_simd/src/type_i8x4.rs
@@ -0,0 +1 @@
+define_type! { struct i8x4([i8; 4]); }
diff --git a/crates/core_simd/src/type_i8x64.rs b/crates/core_simd/src/type_i8x64.rs
new file mode 100644
index 000000000000..d21baf12640a
--- /dev/null
+++ b/crates/core_simd/src/type_i8x64.rs
@@ -0,0 +1,9 @@
+define_type! { struct i8x64([i8; 64]); }
+
+/*
+#[cfg(target_arch = "x86")]
+from_aligned! { unsafe u8x64 |bidirectional| core::arch::x86::__m512i }
+
+#[cfg(target_arch = "x86_64")]
+from_aligned! { unsafe u8x64 |bidirectional| core::arch::x86_64::__m512i }
+*/
diff --git a/crates/core_simd/src/type_i8x8.rs b/crates/core_simd/src/type_i8x8.rs
new file mode 100644
index 000000000000..f09a70f6cb9d
--- /dev/null
+++ b/crates/core_simd/src/type_i8x8.rs
@@ -0,0 +1 @@
+define_type! { struct i8x8([i8; 8]); }
diff --git a/crates/core_simd/src/type_isizex2.rs b/crates/core_simd/src/type_isizex2.rs
new file mode 100644
index 000000000000..aa3abaa8c4e7
--- /dev/null
+++ b/crates/core_simd/src/type_isizex2.rs
@@ -0,0 +1,7 @@
+define_type! { struct isizex2([isize; 2]); }
+
+#[cfg(all(target_arch = "x86", target_pointer_width = "64"))]
+from_aligned! { unsafe isizex2 |bidirectional| core::arch::x86::__m128i }
+
+#[cfg(all(target_arch = "x86_64", target_pointer_width = "64"))]
+from_aligned! { unsafe isizex2 |bidirectional| core::arch::x86_64::__m128i }
diff --git a/crates/core_simd/src/type_isizex4.rs b/crates/core_simd/src/type_isizex4.rs
new file mode 100644
index 000000000000..2c0d7d33ba53
--- /dev/null
+++ b/crates/core_simd/src/type_isizex4.rs
@@ -0,0 +1,13 @@
+define_type! { struct isizex4([isize; 4]); }
+
+#[cfg(all(target_arch = "x86", target_pointer_width = "32"))]
+from_aligned! { unsafe isizex4 |bidirectional| core::arch::x86::__m128i }
+
+#[cfg(all(target_arch = "x86_64", target_pointer_width = "32"))]
+from_aligned! { unsafe isizex4 |bidirectional| core::arch::x86_64::__m128i }
+
+#[cfg(all(target_arch = "x86", target_pointer_width = "64"))]
+from_aligned! { unsafe isizex4 |bidirectional| core::arch::x86::__m256i }
+
+#[cfg(all(target_arch = "x86_64", target_pointer_width = "64"))]
+from_aligned! { unsafe isizex4 |bidirectional| core::arch::x86_64::__m256i }
diff --git a/crates/core_simd/src/type_isizex8.rs b/crates/core_simd/src/type_isizex8.rs
new file mode 100644
index 000000000000..cc2f9ad550f1
--- /dev/null
+++ b/crates/core_simd/src/type_isizex8.rs
@@ -0,0 +1,15 @@
+define_type! { struct isizex8([isize; 8]); }
+
+#[cfg(all(target_arch = "x86", target_pointer_width = "32"))]
+from_aligned! { unsafe isizex8 |bidirectional| core::arch::x86::__m256i }
+
+#[cfg(all(target_arch = "x86_64", target_pointer_width = "32"))]
+from_aligned! { unsafe isizex8 |bidirectional| core::arch::x86_64::__m256i }
+
+/*
+#[cfg(all(target_arch = "x86", target_pointer_width = "64"))]
+from_aligned! { unsafe isizex8 |bidirectional| core::arch::x86::__m512i }
+
+#[cfg(all(target_arch = "x86_64", target_pointer_width = "64"))]
+from_aligned! { unsafe isizex8 |bidirectional| core::arch::x86_64::__m512i }
+*/
diff --git a/crates/core_simd/src/type_u128x2.rs b/crates/core_simd/src/type_u128x2.rs
new file mode 100644
index 000000000000..cbed75896160
--- /dev/null
+++ b/crates/core_simd/src/type_u128x2.rs
@@ -0,0 +1,7 @@
+define_type! { struct u128x2([u128; 2]); }
+
+#[cfg(target_arch = "x86")]
+from_aligned! { unsafe u128x2 |bidirectional| core::arch::x86::__m256i }
+
+#[cfg(target_arch = "x86_64")]
+from_aligned! { unsafe u128x2 |bidirectional| core::arch::x86_64::__m256i }
diff --git a/crates/core_simd/src/type_u128x4.rs b/crates/core_simd/src/type_u128x4.rs
new file mode 100644
index 000000000000..563689621eec
--- /dev/null
+++ b/crates/core_simd/src/type_u128x4.rs
@@ -0,0 +1,9 @@
+define_type! { struct u128x4([u128; 4]); }
+
+/*
+#[cfg(target_arch = "x86")]
+from_aligned! { unsafe u128x4 |bidirectional| core::arch::x86::__m512i }
+
+#[cfg(target_arch = "x86_64")]
+from_aligned! { unsafe u128x4 |bidirectional| core::arch::x86_64::__m512i }
+*/
diff --git a/crates/core_simd/src/type_u16x16.rs b/crates/core_simd/src/type_u16x16.rs
new file mode 100644
index 000000000000..50f39983b385
--- /dev/null
+++ b/crates/core_simd/src/type_u16x16.rs
@@ -0,0 +1,7 @@
+define_type! { struct u16x16([u16; 16]); }
+
+#[cfg(target_arch = "x86")]
+from_aligned! { unsafe u16x16 |bidirectional| core::arch::x86::__m256i }
+
+#[cfg(target_arch = "x86_64")]
+from_aligned! { unsafe u16x16 |bidirectional| core::arch::x86_64::__m256i }
diff --git a/crates/core_simd/src/type_u16x2.rs b/crates/core_simd/src/type_u16x2.rs
new file mode 100644
index 000000000000..d0384e40f94a
--- /dev/null
+++ b/crates/core_simd/src/type_u16x2.rs
@@ -0,0 +1 @@
+define_type! { struct u16x2([u16; 2]); }
diff --git a/crates/core_simd/src/type_u16x32.rs b/crates/core_simd/src/type_u16x32.rs
new file mode 100644
index 000000000000..29cf58581a92
--- /dev/null
+++ b/crates/core_simd/src/type_u16x32.rs
@@ -0,0 +1,9 @@
+define_type! { struct u16x32([u16; 32]); }
+
+/*
+#[cfg(target_arch = "x86")]
+from_aligned! { unsafe u16x32 |bidirectional| core::arch::x86::__m512i }
+
+#[cfg(target_arch = "x86_64")]
+from_aligned! { unsafe u16x32 |bidirectional| core::arch::x86_64::__m512i }
+*/
diff --git a/crates/core_simd/src/type_u16x4.rs b/crates/core_simd/src/type_u16x4.rs
new file mode 100644
index 000000000000..0c4e3ed9b7de
--- /dev/null
+++ b/crates/core_simd/src/type_u16x4.rs
@@ -0,0 +1 @@
+define_type! { struct u16x4([u16; 4]); }
diff --git a/crates/core_simd/src/type_u16x8.rs b/crates/core_simd/src/type_u16x8.rs
new file mode 100644
index 000000000000..69127996f7f9
--- /dev/null
+++ b/crates/core_simd/src/type_u16x8.rs
@@ -0,0 +1,7 @@
+define_type! { struct u16x8([u16; 8]); }
+
+#[cfg(target_arch = "x86")]
+from_aligned! { unsafe u16x8 |bidirectional| core::arch::x86::__m128i }
+
+#[cfg(target_arch = "x86_64")]
+from_aligned! { unsafe u16x8 |bidirectional| core::arch::x86_64::__m128i }
diff --git a/crates/core_simd/src/type_u32x16.rs b/crates/core_simd/src/type_u32x16.rs
new file mode 100644
index 000000000000..1b8965685b8d
--- /dev/null
+++ b/crates/core_simd/src/type_u32x16.rs
@@ -0,0 +1,9 @@
+define_type! { struct u32x16([u32; 16]); }
+
+/*
+#[cfg(target_arch = "x86")]
+from_aligned! { unsafe u32x16 |bidirectional| core::arch::x86::__m512i }
+
+#[cfg(target_arch = "x86_64")]
+from_aligned! { unsafe u32x16 |bidirectional| core::arch::x86_64::__m512i }
+*/
diff --git a/crates/core_simd/src/type_u32x2.rs b/crates/core_simd/src/type_u32x2.rs
new file mode 100644
index 000000000000..8d9d53595253
--- /dev/null
+++ b/crates/core_simd/src/type_u32x2.rs
@@ -0,0 +1 @@
+define_type! { struct u32x2([u32; 2]); }
diff --git a/crates/core_simd/src/type_u32x4.rs b/crates/core_simd/src/type_u32x4.rs
new file mode 100644
index 000000000000..fc79dff0e5b7
--- /dev/null
+++ b/crates/core_simd/src/type_u32x4.rs
@@ -0,0 +1,7 @@
+define_type! { struct u32x4([u32; 4]); }
+
+#[cfg(target_arch = "x86")]
+from_aligned! { unsafe u32x4 |bidirectional| core::arch::x86::__m128i }
+
+#[cfg(target_arch = "x86_64")]
+from_aligned! { unsafe u32x4 |bidirectional| core::arch::x86_64::__m128i }
diff --git a/crates/core_simd/src/type_u32x8.rs b/crates/core_simd/src/type_u32x8.rs
new file mode 100644
index 000000000000..727b87860c96
--- /dev/null
+++ b/crates/core_simd/src/type_u32x8.rs
@@ -0,0 +1,7 @@
+define_type! { struct u32x8([u32; 8]); }
+
+#[cfg(target_arch = "x86")]
+from_aligned! { unsafe u32x8 |bidirectional| core::arch::x86::__m256i }
+
+#[cfg(target_arch = "x86_64")]
+from_aligned! { unsafe u32x8 |bidirectional| core::arch::x86_64::__m256i }
diff --git a/crates/core_simd/src/type_u64x2.rs b/crates/core_simd/src/type_u64x2.rs
new file mode 100644
index 000000000000..729982f77356
--- /dev/null
+++ b/crates/core_simd/src/type_u64x2.rs
@@ -0,0 +1,7 @@
+define_type! { struct u64x2([u64; 2]); }
+
+#[cfg(target_arch = "x86")]
+from_aligned! { unsafe u64x2 |bidirectional| core::arch::x86::__m128i }
+
+#[cfg(target_arch = "x86_64")]
+from_aligned! { unsafe u64x2 |bidirectional| core::arch::x86_64::__m128i }
diff --git a/crates/core_simd/src/type_u64x4.rs b/crates/core_simd/src/type_u64x4.rs
new file mode 100644
index 000000000000..7e1aa9733819
--- /dev/null
+++ b/crates/core_simd/src/type_u64x4.rs
@@ -0,0 +1,7 @@
+define_type! { struct u64x4([u64; 4]); }
+
+#[cfg(target_arch = "x86")]
+from_aligned! { unsafe u64x4 |bidirectional| core::arch::x86::__m256i }
+
+#[cfg(target_arch = "x86_64")]
+from_aligned! { unsafe u64x4 |bidirectional| core::arch::x86_64::__m256i }
diff --git a/crates/core_simd/src/type_u64x8.rs b/crates/core_simd/src/type_u64x8.rs
new file mode 100644
index 000000000000..ed0769d13117
--- /dev/null
+++ b/crates/core_simd/src/type_u64x8.rs
@@ -0,0 +1,9 @@
+define_type! { struct u64x8([u64; 8]); }
+
+/*
+#[cfg(target_arch = "x86")]
+from_aligned! { unsafe u64x8 |bidirectional| core::arch::x86::__m512i }
+
+#[cfg(target_arch = "x86_64")]
+from_aligned! { unsafe u64x8 |bidirectional| core::arch::x86_64::__m512i }
+*/
diff --git a/crates/core_simd/src/type_u8x16.rs b/crates/core_simd/src/type_u8x16.rs
new file mode 100644
index 000000000000..69d788ef2b64
--- /dev/null
+++ b/crates/core_simd/src/type_u8x16.rs
@@ -0,0 +1,7 @@
+define_type! { struct u8x16([u8; 16]); }
+
+#[cfg(target_arch = "x86")]
+from_aligned! { unsafe u8x16 |bidirectional| core::arch::x86::__m128i }
+
+#[cfg(target_arch = "x86_64")]
+from_aligned! { unsafe u8x16 |bidirectional| core::arch::x86_64::__m128i }
diff --git a/crates/core_simd/src/type_u8x2.rs b/crates/core_simd/src/type_u8x2.rs
new file mode 100644
index 000000000000..5ab66878b82e
--- /dev/null
+++ b/crates/core_simd/src/type_u8x2.rs
@@ -0,0 +1 @@
+define_type! { struct u8x2([u8; 2]); }
diff --git a/crates/core_simd/src/type_u8x32.rs b/crates/core_simd/src/type_u8x32.rs
new file mode 100644
index 000000000000..102724684cb8
--- /dev/null
+++ b/crates/core_simd/src/type_u8x32.rs
@@ -0,0 +1,7 @@
+define_type! { struct u8x32([u8; 32]); }
+
+#[cfg(target_arch = "x86")]
+from_aligned! { unsafe u8x32 |bidirectional| core::arch::x86::__m256i }
+
+#[cfg(target_arch = "x86_64")]
+from_aligned! { unsafe u8x32 |bidirectional| core::arch::x86_64::__m256i }
diff --git a/crates/core_simd/src/type_u8x4.rs b/crates/core_simd/src/type_u8x4.rs
new file mode 100644
index 000000000000..648d12b9e2d0
--- /dev/null
+++ b/crates/core_simd/src/type_u8x4.rs
@@ -0,0 +1 @@
+define_type! { struct u8x4([u8; 4]); }
diff --git a/crates/core_simd/src/type_u8x64.rs b/crates/core_simd/src/type_u8x64.rs
new file mode 100644
index 000000000000..1e2967d485ae
--- /dev/null
+++ b/crates/core_simd/src/type_u8x64.rs
@@ -0,0 +1,9 @@
+define_type! { struct u8x64([u8; 64]); }
+
+/*
+#[cfg(target_arch = "x86")]
+from_aligned! { unsafe u8x64 |bidirectional| core::arch::x86::__m512i }
+
+#[cfg(target_arch = "x86_64")]
+from_aligned! { unsafe u8x64 |bidirectional| core::arch::x86_64::__m512i }
+*/
diff --git a/crates/core_simd/src/type_u8x8.rs b/crates/core_simd/src/type_u8x8.rs
new file mode 100644
index 000000000000..aa62e04ddf23
--- /dev/null
+++ b/crates/core_simd/src/type_u8x8.rs
@@ -0,0 +1 @@
+define_type! { struct u8x8([u8; 8]); }
diff --git a/crates/core_simd/src/type_usizex2.rs b/crates/core_simd/src/type_usizex2.rs
new file mode 100644
index 000000000000..083cd499a074
--- /dev/null
+++ b/crates/core_simd/src/type_usizex2.rs
@@ -0,0 +1,7 @@
+define_type! { struct usizex2([usize; 2]); }
+
+#[cfg(all(target_arch = "x86", target_pointer_width = "64"))]
+from_aligned! { unsafe usizex2 |bidirectional| core::arch::x86::__m128i }
+
+#[cfg(all(target_arch = "x86_64", target_pointer_width = "64"))]
+from_aligned! { unsafe usizex2 |bidirectional| core::arch::x86_64::__m128i }
diff --git a/crates/core_simd/src/type_usizex4.rs b/crates/core_simd/src/type_usizex4.rs
new file mode 100644
index 000000000000..3301039a05a1
--- /dev/null
+++ b/crates/core_simd/src/type_usizex4.rs
@@ -0,0 +1,13 @@
+define_type! { struct usizex4([usize; 4]); }
+
+#[cfg(all(target_arch = "x86", target_pointer_width = "32"))]
+from_aligned! { unsafe usizex4 |bidirectional| core::arch::x86::__m128i }
+
+#[cfg(all(target_arch = "x86_64", target_pointer_width = "32"))]
+from_aligned! { unsafe usizex4 |bidirectional| core::arch::x86_64::__m128i }
+
+#[cfg(all(target_arch = "x86", target_pointer_width = "64"))]
+from_aligned! { unsafe usizex4 |bidirectional| core::arch::x86::__m256i }
+
+#[cfg(all(target_arch = "x86_64", target_pointer_width = "64"))]
+from_aligned! { unsafe usizex4 |bidirectional| core::arch::x86_64::__m256i }
diff --git a/crates/core_simd/src/type_usizex8.rs b/crates/core_simd/src/type_usizex8.rs
new file mode 100644
index 000000000000..e21b1ea6b7a0
--- /dev/null
+++ b/crates/core_simd/src/type_usizex8.rs
@@ -0,0 +1,15 @@
+define_type! { struct usizex8([usize; 8]); }
+
+#[cfg(all(target_arch = "x86", target_pointer_width = "32"))]
+from_aligned! { unsafe usizex8 |bidirectional| core::arch::x86::__m256i }
+
+#[cfg(all(target_arch = "x86_64", target_pointer_width = "32"))]
+from_aligned! { unsafe usizex8 |bidirectional| core::arch::x86_64::__m256i }
+
+/*
+#[cfg(all(target_arch = "x86", target_pointer_width = "64"))]
+from_aligned! { unsafe usizex8 |bidirectional| core::arch::x86::__m512i }
+
+#[cfg(all(target_arch = "x86_64", target_pointer_width = "64"))]
+from_aligned! { unsafe usizex8 |bidirectional| core::arch::x86_64::__m512i }
+*/

From 0f837a9147a1ba22cfb12a4480ce1c18d3afc6b8 Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Tue, 22 Sep 2020 20:26:25 -0400
Subject: [PATCH 006/249] Add docs

---
 crates/core_simd/src/macros.rs       | 37 +++++++++++++++-------------
 crates/core_simd/src/type_f32x16.rs  |  5 +++-
 crates/core_simd/src/type_f32x2.rs   |  5 +++-
 crates/core_simd/src/type_f32x4.rs   |  5 +++-
 crates/core_simd/src/type_f32x8.rs   |  5 +++-
 crates/core_simd/src/type_f64x2.rs   |  5 +++-
 crates/core_simd/src/type_f64x4.rs   |  5 +++-
 crates/core_simd/src/type_f64x8.rs   |  5 +++-
 crates/core_simd/src/type_i128x2.rs  |  5 +++-
 crates/core_simd/src/type_i128x4.rs  |  5 +++-
 crates/core_simd/src/type_i16x16.rs  |  5 +++-
 crates/core_simd/src/type_i16x2.rs   |  5 +++-
 crates/core_simd/src/type_i16x32.rs  |  5 +++-
 crates/core_simd/src/type_i16x4.rs   |  5 +++-
 crates/core_simd/src/type_i16x8.rs   |  5 +++-
 crates/core_simd/src/type_i32x16.rs  |  5 +++-
 crates/core_simd/src/type_i32x2.rs   |  5 +++-
 crates/core_simd/src/type_i32x4.rs   |  5 +++-
 crates/core_simd/src/type_i32x8.rs   |  5 +++-
 crates/core_simd/src/type_i64x2.rs   |  5 +++-
 crates/core_simd/src/type_i64x4.rs   |  5 +++-
 crates/core_simd/src/type_i64x8.rs   |  5 +++-
 crates/core_simd/src/type_i8x16.rs   |  5 +++-
 crates/core_simd/src/type_i8x2.rs    |  5 +++-
 crates/core_simd/src/type_i8x32.rs   |  5 +++-
 crates/core_simd/src/type_i8x4.rs    |  5 +++-
 crates/core_simd/src/type_i8x64.rs   |  5 +++-
 crates/core_simd/src/type_i8x8.rs    |  5 +++-
 crates/core_simd/src/type_isizex2.rs |  5 +++-
 crates/core_simd/src/type_isizex4.rs |  5 +++-
 crates/core_simd/src/type_isizex8.rs |  5 +++-
 crates/core_simd/src/type_u128x2.rs  |  5 +++-
 crates/core_simd/src/type_u128x4.rs  |  5 +++-
 crates/core_simd/src/type_u16x16.rs  |  5 +++-
 crates/core_simd/src/type_u16x2.rs   |  5 +++-
 crates/core_simd/src/type_u16x32.rs  |  5 +++-
 crates/core_simd/src/type_u16x4.rs   |  5 +++-
 crates/core_simd/src/type_u16x8.rs   |  5 +++-
 crates/core_simd/src/type_u32x16.rs  |  5 +++-
 crates/core_simd/src/type_u32x2.rs   |  5 +++-
 crates/core_simd/src/type_u32x4.rs   |  5 +++-
 crates/core_simd/src/type_u32x8.rs   |  5 +++-
 crates/core_simd/src/type_u64x2.rs   |  5 +++-
 crates/core_simd/src/type_u64x4.rs   |  5 +++-
 crates/core_simd/src/type_u64x8.rs   |  5 +++-
 crates/core_simd/src/type_u8x16.rs   |  5 +++-
 crates/core_simd/src/type_u8x2.rs    |  5 +++-
 crates/core_simd/src/type_u8x32.rs   |  5 +++-
 crates/core_simd/src/type_u8x4.rs    |  5 +++-
 crates/core_simd/src/type_u8x64.rs   |  5 +++-
 crates/core_simd/src/type_u8x8.rs    |  5 +++-
 crates/core_simd/src/type_usizex2.rs |  5 +++-
 crates/core_simd/src/type_usizex4.rs |  5 +++-
 crates/core_simd/src/type_usizex8.rs |  5 +++-
 54 files changed, 232 insertions(+), 70 deletions(-)

diff --git a/crates/core_simd/src/macros.rs b/crates/core_simd/src/macros.rs
index 70b19e5f4dbc..64d9c4b217f7 100644
--- a/crates/core_simd/src/macros.rs
+++ b/crates/core_simd/src/macros.rs
@@ -28,8 +28,8 @@ macro_rules! from_unaligned {
 }
 
 macro_rules! define_type {
-    { struct $name:ident([$type:ty; $lanes:tt]); } => {
-        define_type! { @impl $name [$type; $lanes] }
+    { $(#[$attr:meta])* struct $name:ident([$type:ty; $lanes:tt]); } => {
+        define_type! { @impl $(#[$attr])* | $name [$type; $lanes] }
 
         // array references
         impl AsRef<[$type; $lanes]> for $name {
@@ -74,40 +74,40 @@ macro_rules! define_type {
             }
         }
     };
-    { @impl $name:ident [$type:ty; 1] } => {
-        define_type! { @impl $name | $type | $type, | v0, }
+    { @impl $(#[$attr:meta])* | $name:ident [$type:ty; 1] } => {
+        define_type! { @def $(#[$attr])* | $name | $type | $type, | v0, }
     };
-    { @impl $name:ident [$type:ty; 2] } => {
-        define_type! { @impl $name | $type | $type, $type, | v0, v1, }
+    { @impl $(#[$attr:meta])* | $name:ident [$type:ty; 2] } => {
+        define_type! { @def $(#[$attr])* | $name | $type | $type, $type, | v0, v1, }
     };
-    { @impl $name:ident [$type:ty; 4] } => {
-        define_type! { @impl $name | $type |
+    { @impl $(#[$attr:meta])* | $name:ident [$type:ty; 4] } => {
+        define_type! { @def $(#[$attr])* | $name | $type |
             $type, $type, $type, $type, |
             v0, v1, v2, v3,
         }
     };
-    { @impl $name:ident [$type:ty; 8] } => {
-        define_type! { @impl $name | $type |
+    { @impl $(#[$attr:meta])* | $name:ident [$type:ty; 8] } => {
+        define_type! { @def $(#[$attr])* | $name | $type |
             $type, $type, $type, $type, $type, $type, $type, $type, |
             v0, v1, v2, v3, v4, v5, v6, v7,
         }
     };
-    { @impl $name:ident [$type:ty; 16] } => {
-        define_type! { @impl $name | $type |
+    { @impl $(#[$attr:meta])* | $name:ident [$type:ty; 16] } => {
+        define_type! { @def $(#[$attr])* | $name | $type |
             $type, $type, $type, $type, $type, $type, $type, $type, $type, $type, $type, $type, $type, $type, $type, $type, |
             v0, v1, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13, v14, v15,
         }
     };
-    { @impl $name:ident [$type:ty; 32] } => {
-        define_type! { @impl $name | $type |
+    { @impl $(#[$attr:meta])* | $name:ident [$type:ty; 32] } => {
+        define_type! { @def $(#[$attr])* | $name | $type |
             $type, $type, $type, $type, $type, $type, $type, $type, $type, $type, $type, $type, $type, $type, $type, $type,
             $type, $type, $type, $type, $type, $type, $type, $type, $type, $type, $type, $type, $type, $type, $type, $type, |
             v0,  v1,  v2,  v3,  v4,  v5,  v6,  v7,  v8,  v9,  v10, v11, v12, v13, v14, v15,
             v16, v17, v18, v19, v20, v21, v22, v23, v24, v25, v26, v27, v28, v29, v30, v31,
         }
     };
-    { @impl $name:ident [$type:ty; 64] } => {
-        define_type! { @impl $name | $type |
+    { @impl $(#[$attr:meta])* | $name:ident [$type:ty; 64] } => {
+        define_type! { @def $(#[$attr])* | $name | $type |
             $type, $type, $type, $type, $type, $type, $type, $type, $type, $type, $type, $type, $type, $type, $type, $type,
             $type, $type, $type, $type, $type, $type, $type, $type, $type, $type, $type, $type, $type, $type, $type, $type,
             $type, $type, $type, $type, $type, $type, $type, $type, $type, $type, $type, $type, $type, $type, $type, $type,
@@ -118,18 +118,21 @@ macro_rules! define_type {
             v48, v49, v50, v51, v52, v53, v54, v55, v56, v57, v58, v59, v60, v61, v62, v63,
         }
     };
-    { @impl $name:ident | $type:ty | $($itype:ty,)* | $($ivar:ident,)* } => {
+    { @def $(#[$attr:meta])* | $name:ident | $type:ty | $($itype:ty,)* | $($ivar:ident,)* } => {
+        $(#[$attr])*
         #[allow(non_camel_case_types)]
         #[derive(Copy, Clone, Debug, Default, PartialEq, PartialOrd)]
         #[repr(simd)]
         pub struct $name($($itype),*);
 
         impl $name {
+            /// Construct a vector by setting each lane to a single value.
             #[inline]
             pub fn splat(value: $type) -> Self {
                 Self($(value as $itype),*)
             }
 
+            /// Construct a vector by setting each lane.
             #[allow(clippy::too_many_arguments)]
             #[inline]
             pub fn new($($ivar: $itype),*) -> Self {
diff --git a/crates/core_simd/src/type_f32x16.rs b/crates/core_simd/src/type_f32x16.rs
index f001b9c42df4..45fb4a3175bb 100644
--- a/crates/core_simd/src/type_f32x16.rs
+++ b/crates/core_simd/src/type_f32x16.rs
@@ -1,4 +1,7 @@
-define_type! { struct f32x16([f32; 16]); }
+define_type! {
+    #[doc = "Vector of 16 `f32` types"]
+    struct f32x16([f32; 16]);
+}
 
 /*
 #[cfg(target_arch = "x86")]
diff --git a/crates/core_simd/src/type_f32x2.rs b/crates/core_simd/src/type_f32x2.rs
index e6691523ea9f..8b60a2ee76f4 100644
--- a/crates/core_simd/src/type_f32x2.rs
+++ b/crates/core_simd/src/type_f32x2.rs
@@ -1 +1,4 @@
-define_type! { struct f32x2([f32; 2]); }
+define_type! {
+    #[doc = "Vector of two `f32` types"]
+    struct f32x2([f32; 2]);
+}
diff --git a/crates/core_simd/src/type_f32x4.rs b/crates/core_simd/src/type_f32x4.rs
index 4fd937425c50..452e607732cf 100644
--- a/crates/core_simd/src/type_f32x4.rs
+++ b/crates/core_simd/src/type_f32x4.rs
@@ -1,4 +1,7 @@
-define_type! { struct f32x4([f32; 4]); }
+define_type! {
+    #[doc = "Vector of four `f32` types"]
+    struct f32x4([f32; 4]);
+}
 
 #[cfg(target_arch = "x86")]
 from_aligned! { unsafe f32x4 |bidirectional| core::arch::x86::__m128 }
diff --git a/crates/core_simd/src/type_f32x8.rs b/crates/core_simd/src/type_f32x8.rs
index a6754d490c1a..790a546e4e73 100644
--- a/crates/core_simd/src/type_f32x8.rs
+++ b/crates/core_simd/src/type_f32x8.rs
@@ -1,4 +1,7 @@
-define_type! { struct f32x8([f32; 8]); }
+define_type! {
+    #[doc = "Vector of eight `f32` types"]
+    struct f32x8([f32; 8]);
+}
 
 #[cfg(target_arch = "x86")]
 from_aligned! { unsafe f32x8 |bidirectional| core::arch::x86::__m256 }
diff --git a/crates/core_simd/src/type_f64x2.rs b/crates/core_simd/src/type_f64x2.rs
index 227ef951434d..0c349f38c862 100644
--- a/crates/core_simd/src/type_f64x2.rs
+++ b/crates/core_simd/src/type_f64x2.rs
@@ -1,4 +1,7 @@
-define_type! { struct f64x2([f64; 2]); }
+define_type! {
+    #[doc = "Vector of two `f64` types"]
+    struct f64x2([f64; 2]);
+}
 
 #[cfg(target_arch = "x86")]
 from_aligned! { unsafe f64x2 |bidirectional| core::arch::x86::__m128d }
diff --git a/crates/core_simd/src/type_f64x4.rs b/crates/core_simd/src/type_f64x4.rs
index 8e9debd8bfde..ec6b46bc8c84 100644
--- a/crates/core_simd/src/type_f64x4.rs
+++ b/crates/core_simd/src/type_f64x4.rs
@@ -1,4 +1,7 @@
-define_type! { struct f64x4([f64; 4]); }
+define_type! {
+    #[doc = "Vector of four `f64` types"]
+    struct f64x4([f64; 4]);
+}
 
 #[cfg(target_arch = "x86")]
 from_aligned! { unsafe f64x4 |bidirectional| core::arch::x86::__m256d }
diff --git a/crates/core_simd/src/type_f64x8.rs b/crates/core_simd/src/type_f64x8.rs
index 8aa3812b4f43..dd65dc6b39ab 100644
--- a/crates/core_simd/src/type_f64x8.rs
+++ b/crates/core_simd/src/type_f64x8.rs
@@ -1,4 +1,7 @@
-define_type! { struct f64x8([f64; 8]); }
+define_type! {
+    #[doc = "Vector of eight `f64` types"]
+    struct f64x8([f64; 8]);
+}
 
 /*
 #[cfg(target_arch = "x86")]
diff --git a/crates/core_simd/src/type_i128x2.rs b/crates/core_simd/src/type_i128x2.rs
index cb1be5a9866c..1ac736ddf3a2 100644
--- a/crates/core_simd/src/type_i128x2.rs
+++ b/crates/core_simd/src/type_i128x2.rs
@@ -1,4 +1,7 @@
-define_type! { struct i128x2([i128; 2]); }
+define_type! {
+    #[doc = "Vector of two `i128` types"]
+    struct i128x2([i128; 2]);
+}
 
 #[cfg(target_arch = "x86")]
 from_aligned! { unsafe i128x2 |bidirectional| core::arch::x86::__m256i }
diff --git a/crates/core_simd/src/type_i128x4.rs b/crates/core_simd/src/type_i128x4.rs
index 8888d4205f86..6cef039d947a 100644
--- a/crates/core_simd/src/type_i128x4.rs
+++ b/crates/core_simd/src/type_i128x4.rs
@@ -1,4 +1,7 @@
-define_type! { struct i128x4([i128; 4]); }
+define_type! {
+    #[doc = "Vector of four `i128` types"]
+    struct i128x4([i128; 4]);
+}
 
 /*
 #[cfg(target_arch = "x86")]
diff --git a/crates/core_simd/src/type_i16x16.rs b/crates/core_simd/src/type_i16x16.rs
index ec52c4857763..1721286128fe 100644
--- a/crates/core_simd/src/type_i16x16.rs
+++ b/crates/core_simd/src/type_i16x16.rs
@@ -1,4 +1,7 @@
-define_type! { struct i16x16([i16; 16]); }
+define_type! {
+    #[doc = "Vector of 16 `i16` types"]
+    struct i16x16([i16; 16]);
+}
 
 #[cfg(target_arch = "x86")]
 from_aligned! { unsafe i16x16 |bidirectional| core::arch::x86::__m256i }
diff --git a/crates/core_simd/src/type_i16x2.rs b/crates/core_simd/src/type_i16x2.rs
index 313ba334da19..7ccbbe7f2d0a 100644
--- a/crates/core_simd/src/type_i16x2.rs
+++ b/crates/core_simd/src/type_i16x2.rs
@@ -1 +1,4 @@
-define_type! { struct i16x2([i16; 2]); }
+define_type! {
+    #[doc = "Vector of two `i16` types"]
+    struct i16x2([i16; 2]);
+}
diff --git a/crates/core_simd/src/type_i16x32.rs b/crates/core_simd/src/type_i16x32.rs
index 53679e2ac5c4..349d094a1bc9 100644
--- a/crates/core_simd/src/type_i16x32.rs
+++ b/crates/core_simd/src/type_i16x32.rs
@@ -1,4 +1,7 @@
-define_type! { struct i16x32([i16; 32]); }
+define_type! {
+    #[doc = "Vector of 32 `i16` types"]
+    struct i16x32([i16; 32]);
+}
 
 /*
 #[cfg(target_arch = "x86")]
diff --git a/crates/core_simd/src/type_i16x4.rs b/crates/core_simd/src/type_i16x4.rs
index f69f5dc2acda..348bc4c53a94 100644
--- a/crates/core_simd/src/type_i16x4.rs
+++ b/crates/core_simd/src/type_i16x4.rs
@@ -1 +1,4 @@
-define_type! { struct i16x4([i16; 4]); }
+define_type! {
+    #[doc = "Vector of four `i16` types"]
+    struct i16x4([i16; 4]);
+}
diff --git a/crates/core_simd/src/type_i16x8.rs b/crates/core_simd/src/type_i16x8.rs
index b1a3100dd6e2..87ded0f3a49c 100644
--- a/crates/core_simd/src/type_i16x8.rs
+++ b/crates/core_simd/src/type_i16x8.rs
@@ -1,4 +1,7 @@
-define_type! { struct i16x8([i16; 8]); }
+define_type! {
+    #[doc = "Vector of eight `i16` types"]
+    struct i16x8([i16; 8]);
+}
 
 #[cfg(target_arch = "x86")]
 from_aligned! { unsafe i16x8 |bidirectional| core::arch::x86::__m128i }
diff --git a/crates/core_simd/src/type_i32x16.rs b/crates/core_simd/src/type_i32x16.rs
index 5b7edb0f51bb..7c52e9cb9e55 100644
--- a/crates/core_simd/src/type_i32x16.rs
+++ b/crates/core_simd/src/type_i32x16.rs
@@ -1,4 +1,7 @@
-define_type! { struct i32x16([i32; 16]); }
+define_type! {
+    #[doc = "Vector of 16 `i32` types"]
+    struct i32x16([i32; 16]);
+}
 
 /*
 #[cfg(target_arch = "x86")]
diff --git a/crates/core_simd/src/type_i32x2.rs b/crates/core_simd/src/type_i32x2.rs
index 425b6fd3bca8..e9845ae75c73 100644
--- a/crates/core_simd/src/type_i32x2.rs
+++ b/crates/core_simd/src/type_i32x2.rs
@@ -1 +1,4 @@
-define_type! { struct i32x2([i32; 2]); }
+define_type! {
+    #[doc = "Vector of two `i32` types"]
+    struct i32x2([i32; 2]);
+}
diff --git a/crates/core_simd/src/type_i32x4.rs b/crates/core_simd/src/type_i32x4.rs
index d62fc3856340..47374f7ce439 100644
--- a/crates/core_simd/src/type_i32x4.rs
+++ b/crates/core_simd/src/type_i32x4.rs
@@ -1,4 +1,7 @@
-define_type! { struct i32x4([i32; 4]); }
+define_type! {
+    #[doc = "Vector of four `i32` types"]
+    struct i32x4([i32; 4]);
+}
 
 #[cfg(target_arch = "x86")]
 from_aligned! { unsafe i32x4 |bidirectional| core::arch::x86::__m128i }
diff --git a/crates/core_simd/src/type_i32x8.rs b/crates/core_simd/src/type_i32x8.rs
index ee83854cee4e..79b4ea180c31 100644
--- a/crates/core_simd/src/type_i32x8.rs
+++ b/crates/core_simd/src/type_i32x8.rs
@@ -1,4 +1,7 @@
-define_type! { struct i32x8([i32; 8]); }
+define_type! {
+    #[doc = "Vector of eight `i32` types"]
+    struct i32x8([i32; 8]);
+}
 
 #[cfg(target_arch = "x86")]
 from_aligned! { unsafe i32x8 |bidirectional| core::arch::x86::__m256i }
diff --git a/crates/core_simd/src/type_i64x2.rs b/crates/core_simd/src/type_i64x2.rs
index 35af990ec649..7268ad5dfbe8 100644
--- a/crates/core_simd/src/type_i64x2.rs
+++ b/crates/core_simd/src/type_i64x2.rs
@@ -1,4 +1,7 @@
-define_type! { struct i64x2([i64; 2]); }
+define_type! {
+    #[doc = "Vector of two `i64` types"]
+    struct i64x2([i64; 2]);
+}
 
 #[cfg(target_arch = "x86")]
 from_aligned! { unsafe i64x2 |bidirectional| core::arch::x86::__m128i }
diff --git a/crates/core_simd/src/type_i64x4.rs b/crates/core_simd/src/type_i64x4.rs
index 6f69a492ede9..3535783c3000 100644
--- a/crates/core_simd/src/type_i64x4.rs
+++ b/crates/core_simd/src/type_i64x4.rs
@@ -1,4 +1,7 @@
-define_type! { struct i64x4([i64; 4]); }
+define_type! {
+    #[doc = "Vector of four `i64` types"]
+    struct i64x4([i64; 4]);
+}
 
 #[cfg(target_arch = "x86")]
 from_aligned! { unsafe i64x4 |bidirectional| core::arch::x86::__m256i }
diff --git a/crates/core_simd/src/type_i64x8.rs b/crates/core_simd/src/type_i64x8.rs
index 888c9b6612c0..42d28db8c551 100644
--- a/crates/core_simd/src/type_i64x8.rs
+++ b/crates/core_simd/src/type_i64x8.rs
@@ -1,4 +1,7 @@
-define_type! { struct i64x8([i64; 8]); }
+define_type! {
+    #[doc = "Vector of eight `i64` types"]
+    struct i64x8([i64; 8]);
+}
 
 /*
 #[cfg(target_arch = "x86")]
diff --git a/crates/core_simd/src/type_i8x16.rs b/crates/core_simd/src/type_i8x16.rs
index 32ea4fa82934..d7aadd4d7ab2 100644
--- a/crates/core_simd/src/type_i8x16.rs
+++ b/crates/core_simd/src/type_i8x16.rs
@@ -1,4 +1,7 @@
-define_type! { struct i8x16([i8; 16]); }
+define_type! {
+    #[doc = "Vector of 16 `i8` types"]
+    struct i8x16([i8; 16]);
+}
 
 #[cfg(target_arch = "x86")]
 from_aligned! { unsafe i8x16 |bidirectional| core::arch::x86::__m128i }
diff --git a/crates/core_simd/src/type_i8x2.rs b/crates/core_simd/src/type_i8x2.rs
index a36de2c5421d..aca22993c2e3 100644
--- a/crates/core_simd/src/type_i8x2.rs
+++ b/crates/core_simd/src/type_i8x2.rs
@@ -1 +1,4 @@
-define_type! { struct i8x2([i8; 2]); }
+define_type! {
+    #[doc = "Vector of two `i8` types"]
+    struct i8x2([i8; 2]);
+}
diff --git a/crates/core_simd/src/type_i8x32.rs b/crates/core_simd/src/type_i8x32.rs
index 874fcdd1e96a..a323565c85cf 100644
--- a/crates/core_simd/src/type_i8x32.rs
+++ b/crates/core_simd/src/type_i8x32.rs
@@ -1,4 +1,7 @@
-define_type! { struct i8x32([i8; 32]); }
+define_type! {
+    #[doc = "Vector of 32 `i8` types"]
+    struct i8x32([i8; 32]);
+}
 
 #[cfg(target_arch = "x86")]
 from_aligned! { unsafe i8x32 |bidirectional| core::arch::x86::__m256i }
diff --git a/crates/core_simd/src/type_i8x4.rs b/crates/core_simd/src/type_i8x4.rs
index 5be83d5e0377..246703ebc50d 100644
--- a/crates/core_simd/src/type_i8x4.rs
+++ b/crates/core_simd/src/type_i8x4.rs
@@ -1 +1,4 @@
-define_type! { struct i8x4([i8; 4]); }
+define_type! {
+    #[doc = "Vector of four `i8` types"]
+    struct i8x4([i8; 4]);
+}
diff --git a/crates/core_simd/src/type_i8x64.rs b/crates/core_simd/src/type_i8x64.rs
index d21baf12640a..26934df2a484 100644
--- a/crates/core_simd/src/type_i8x64.rs
+++ b/crates/core_simd/src/type_i8x64.rs
@@ -1,4 +1,7 @@
-define_type! { struct i8x64([i8; 64]); }
+define_type! {
+    #[doc = "Vector of 64 `i8` types"]
+    struct i8x64([i8; 64]);
+}
 
 /*
 #[cfg(target_arch = "x86")]
diff --git a/crates/core_simd/src/type_i8x8.rs b/crates/core_simd/src/type_i8x8.rs
index f09a70f6cb9d..bc30e2daf840 100644
--- a/crates/core_simd/src/type_i8x8.rs
+++ b/crates/core_simd/src/type_i8x8.rs
@@ -1 +1,4 @@
-define_type! { struct i8x8([i8; 8]); }
+define_type! {
+    #[doc = "Vector of eight `i8` types"]
+    struct i8x8([i8; 8]);
+}
diff --git a/crates/core_simd/src/type_isizex2.rs b/crates/core_simd/src/type_isizex2.rs
index aa3abaa8c4e7..464f64955522 100644
--- a/crates/core_simd/src/type_isizex2.rs
+++ b/crates/core_simd/src/type_isizex2.rs
@@ -1,4 +1,7 @@
-define_type! { struct isizex2([isize; 2]); }
+define_type! {
+    #[doc = "Vector of two `isize` types"]
+    struct isizex2([isize; 2]);
+}
 
 #[cfg(all(target_arch = "x86", target_pointer_width = "64"))]
 from_aligned! { unsafe isizex2 |bidirectional| core::arch::x86::__m128i }
diff --git a/crates/core_simd/src/type_isizex4.rs b/crates/core_simd/src/type_isizex4.rs
index 2c0d7d33ba53..3be457393ec2 100644
--- a/crates/core_simd/src/type_isizex4.rs
+++ b/crates/core_simd/src/type_isizex4.rs
@@ -1,4 +1,7 @@
-define_type! { struct isizex4([isize; 4]); }
+define_type! {
+    #[doc = "Vector of four `isize` types"]
+    struct isizex4([isize; 4]);
+}
 
 #[cfg(all(target_arch = "x86", target_pointer_width = "32"))]
 from_aligned! { unsafe isizex4 |bidirectional| core::arch::x86::__m128i }
diff --git a/crates/core_simd/src/type_isizex8.rs b/crates/core_simd/src/type_isizex8.rs
index cc2f9ad550f1..e21c2cf624b0 100644
--- a/crates/core_simd/src/type_isizex8.rs
+++ b/crates/core_simd/src/type_isizex8.rs
@@ -1,4 +1,7 @@
-define_type! { struct isizex8([isize; 8]); }
+define_type! {
+    #[doc = "Vector of eight `isize` types"]
+    struct isizex8([isize; 8]);
+}
 
 #[cfg(all(target_arch = "x86", target_pointer_width = "32"))]
 from_aligned! { unsafe isizex8 |bidirectional| core::arch::x86::__m256i }
diff --git a/crates/core_simd/src/type_u128x2.rs b/crates/core_simd/src/type_u128x2.rs
index cbed75896160..f448e4f0e626 100644
--- a/crates/core_simd/src/type_u128x2.rs
+++ b/crates/core_simd/src/type_u128x2.rs
@@ -1,4 +1,7 @@
-define_type! { struct u128x2([u128; 2]); }
+define_type! {
+    #[doc = "Vector of two `u128` types"]
+    struct u128x2([u128; 2]);
+}
 
 #[cfg(target_arch = "x86")]
 from_aligned! { unsafe u128x2 |bidirectional| core::arch::x86::__m256i }
diff --git a/crates/core_simd/src/type_u128x4.rs b/crates/core_simd/src/type_u128x4.rs
index 563689621eec..07466c720853 100644
--- a/crates/core_simd/src/type_u128x4.rs
+++ b/crates/core_simd/src/type_u128x4.rs
@@ -1,4 +1,7 @@
-define_type! { struct u128x4([u128; 4]); }
+define_type! {
+    #[doc = "Vector of four `u128` types"]
+    struct u128x4([u128; 4]);
+}
 
 /*
 #[cfg(target_arch = "x86")]
diff --git a/crates/core_simd/src/type_u16x16.rs b/crates/core_simd/src/type_u16x16.rs
index 50f39983b385..5460c3740adf 100644
--- a/crates/core_simd/src/type_u16x16.rs
+++ b/crates/core_simd/src/type_u16x16.rs
@@ -1,4 +1,7 @@
-define_type! { struct u16x16([u16; 16]); }
+define_type! {
+    #[doc = "Vector of 16 `u16` types"]
+    struct u16x16([u16; 16]);
+}
 
 #[cfg(target_arch = "x86")]
 from_aligned! { unsafe u16x16 |bidirectional| core::arch::x86::__m256i }
diff --git a/crates/core_simd/src/type_u16x2.rs b/crates/core_simd/src/type_u16x2.rs
index d0384e40f94a..480be8b6eaef 100644
--- a/crates/core_simd/src/type_u16x2.rs
+++ b/crates/core_simd/src/type_u16x2.rs
@@ -1 +1,4 @@
-define_type! { struct u16x2([u16; 2]); }
+define_type! {
+    #[doc = "Vector of two `u16` types"]
+    struct u16x2([u16; 2]);
+}
diff --git a/crates/core_simd/src/type_u16x32.rs b/crates/core_simd/src/type_u16x32.rs
index 29cf58581a92..67d60535f776 100644
--- a/crates/core_simd/src/type_u16x32.rs
+++ b/crates/core_simd/src/type_u16x32.rs
@@ -1,4 +1,7 @@
-define_type! { struct u16x32([u16; 32]); }
+define_type! {
+    #[doc = "Vector of 32 `u16` types"]
+    struct u16x32([u16; 32]);
+}
 
 /*
 #[cfg(target_arch = "x86")]
diff --git a/crates/core_simd/src/type_u16x4.rs b/crates/core_simd/src/type_u16x4.rs
index 0c4e3ed9b7de..874891363da8 100644
--- a/crates/core_simd/src/type_u16x4.rs
+++ b/crates/core_simd/src/type_u16x4.rs
@@ -1 +1,4 @@
-define_type! { struct u16x4([u16; 4]); }
+define_type! {
+    #[doc = "Vector of four `u16` types"]
+    struct u16x4([u16; 4]);
+}
diff --git a/crates/core_simd/src/type_u16x8.rs b/crates/core_simd/src/type_u16x8.rs
index 69127996f7f9..5ba0af151e3f 100644
--- a/crates/core_simd/src/type_u16x8.rs
+++ b/crates/core_simd/src/type_u16x8.rs
@@ -1,4 +1,7 @@
-define_type! { struct u16x8([u16; 8]); }
+define_type! {
+    #[doc = "Vector of eight `u16` types"]
+    struct u16x8([u16; 8]);
+}
 
 #[cfg(target_arch = "x86")]
 from_aligned! { unsafe u16x8 |bidirectional| core::arch::x86::__m128i }
diff --git a/crates/core_simd/src/type_u32x16.rs b/crates/core_simd/src/type_u32x16.rs
index 1b8965685b8d..40e557b2d6c9 100644
--- a/crates/core_simd/src/type_u32x16.rs
+++ b/crates/core_simd/src/type_u32x16.rs
@@ -1,4 +1,7 @@
-define_type! { struct u32x16([u32; 16]); }
+define_type! {
+    #[doc = "Vector of 16 `u32` types"]
+    struct u32x16([u32; 16]);
+}
 
 /*
 #[cfg(target_arch = "x86")]
diff --git a/crates/core_simd/src/type_u32x2.rs b/crates/core_simd/src/type_u32x2.rs
index 8d9d53595253..001fd31db39c 100644
--- a/crates/core_simd/src/type_u32x2.rs
+++ b/crates/core_simd/src/type_u32x2.rs
@@ -1 +1,4 @@
-define_type! { struct u32x2([u32; 2]); }
+define_type! {
+    #[doc = "Vector of two `u32` types"]
+    struct u32x2([u32; 2]);
+}
diff --git a/crates/core_simd/src/type_u32x4.rs b/crates/core_simd/src/type_u32x4.rs
index fc79dff0e5b7..0582b51ead14 100644
--- a/crates/core_simd/src/type_u32x4.rs
+++ b/crates/core_simd/src/type_u32x4.rs
@@ -1,4 +1,7 @@
-define_type! { struct u32x4([u32; 4]); }
+define_type! {
+    #[doc = "Vector of four `u32` types"]
+    struct u32x4([u32; 4]);
+}
 
 #[cfg(target_arch = "x86")]
 from_aligned! { unsafe u32x4 |bidirectional| core::arch::x86::__m128i }
diff --git a/crates/core_simd/src/type_u32x8.rs b/crates/core_simd/src/type_u32x8.rs
index 727b87860c96..686833c250c4 100644
--- a/crates/core_simd/src/type_u32x8.rs
+++ b/crates/core_simd/src/type_u32x8.rs
@@ -1,4 +1,7 @@
-define_type! { struct u32x8([u32; 8]); }
+define_type! {
+    #[doc = "Vector of eight `u32` types"]
+    struct u32x8([u32; 8]);
+}
 
 #[cfg(target_arch = "x86")]
 from_aligned! { unsafe u32x8 |bidirectional| core::arch::x86::__m256i }
diff --git a/crates/core_simd/src/type_u64x2.rs b/crates/core_simd/src/type_u64x2.rs
index 729982f77356..e3ba3dc03b51 100644
--- a/crates/core_simd/src/type_u64x2.rs
+++ b/crates/core_simd/src/type_u64x2.rs
@@ -1,4 +1,7 @@
-define_type! { struct u64x2([u64; 2]); }
+define_type! {
+    #[doc = "Vector of two `u64` types"]
+    struct u64x2([u64; 2]);
+}
 
 #[cfg(target_arch = "x86")]
 from_aligned! { unsafe u64x2 |bidirectional| core::arch::x86::__m128i }
diff --git a/crates/core_simd/src/type_u64x4.rs b/crates/core_simd/src/type_u64x4.rs
index 7e1aa9733819..005f26012ad5 100644
--- a/crates/core_simd/src/type_u64x4.rs
+++ b/crates/core_simd/src/type_u64x4.rs
@@ -1,4 +1,7 @@
-define_type! { struct u64x4([u64; 4]); }
+define_type! {
+    #[doc = "Vector of four `u64` types"]
+    struct u64x4([u64; 4]);
+}
 
 #[cfg(target_arch = "x86")]
 from_aligned! { unsafe u64x4 |bidirectional| core::arch::x86::__m256i }
diff --git a/crates/core_simd/src/type_u64x8.rs b/crates/core_simd/src/type_u64x8.rs
index ed0769d13117..fbb41f36e5da 100644
--- a/crates/core_simd/src/type_u64x8.rs
+++ b/crates/core_simd/src/type_u64x8.rs
@@ -1,4 +1,7 @@
-define_type! { struct u64x8([u64; 8]); }
+define_type! {
+    #[doc = "Vector of eight `u64` types"]
+    struct u64x8([u64; 8]);
+}
 
 /*
 #[cfg(target_arch = "x86")]
diff --git a/crates/core_simd/src/type_u8x16.rs b/crates/core_simd/src/type_u8x16.rs
index 69d788ef2b64..a44b74c122b2 100644
--- a/crates/core_simd/src/type_u8x16.rs
+++ b/crates/core_simd/src/type_u8x16.rs
@@ -1,4 +1,7 @@
-define_type! { struct u8x16([u8; 16]); }
+define_type! {
+    #[doc = "Vector of 16 `u8` types"]
+    struct u8x16([u8; 16]);
+}
 
 #[cfg(target_arch = "x86")]
 from_aligned! { unsafe u8x16 |bidirectional| core::arch::x86::__m128i }
diff --git a/crates/core_simd/src/type_u8x2.rs b/crates/core_simd/src/type_u8x2.rs
index 5ab66878b82e..c6162cc73a25 100644
--- a/crates/core_simd/src/type_u8x2.rs
+++ b/crates/core_simd/src/type_u8x2.rs
@@ -1 +1,4 @@
-define_type! { struct u8x2([u8; 2]); }
+define_type! {
+    #[doc = "Vector of two `u8` types"]
+    struct u8x2([u8; 2]);
+}
diff --git a/crates/core_simd/src/type_u8x32.rs b/crates/core_simd/src/type_u8x32.rs
index 102724684cb8..012286846db9 100644
--- a/crates/core_simd/src/type_u8x32.rs
+++ b/crates/core_simd/src/type_u8x32.rs
@@ -1,4 +1,7 @@
-define_type! { struct u8x32([u8; 32]); }
+define_type! {
+    #[doc = "Vector of 32 `u8` types"]
+    struct u8x32([u8; 32]);
+}
 
 #[cfg(target_arch = "x86")]
 from_aligned! { unsafe u8x32 |bidirectional| core::arch::x86::__m256i }
diff --git a/crates/core_simd/src/type_u8x4.rs b/crates/core_simd/src/type_u8x4.rs
index 648d12b9e2d0..ecdffa119931 100644
--- a/crates/core_simd/src/type_u8x4.rs
+++ b/crates/core_simd/src/type_u8x4.rs
@@ -1 +1,4 @@
-define_type! { struct u8x4([u8; 4]); }
+define_type! {
+    #[doc = "Vector of four `u8` types"]
+    struct u8x4([u8; 4]);
+}
diff --git a/crates/core_simd/src/type_u8x64.rs b/crates/core_simd/src/type_u8x64.rs
index 1e2967d485ae..6f05f086a64a 100644
--- a/crates/core_simd/src/type_u8x64.rs
+++ b/crates/core_simd/src/type_u8x64.rs
@@ -1,4 +1,7 @@
-define_type! { struct u8x64([u8; 64]); }
+define_type! {
+    #[doc = "Vector of 64 `u8` types"]
+    struct u8x64([u8; 64]);
+}
 
 /*
 #[cfg(target_arch = "x86")]
diff --git a/crates/core_simd/src/type_u8x8.rs b/crates/core_simd/src/type_u8x8.rs
index aa62e04ddf23..7810ac743f8d 100644
--- a/crates/core_simd/src/type_u8x8.rs
+++ b/crates/core_simd/src/type_u8x8.rs
@@ -1 +1,4 @@
-define_type! { struct u8x8([u8; 8]); }
+define_type! {
+    #[doc = "Vector of eight `u8` types"]
+    struct u8x8([u8; 8]);
+}
diff --git a/crates/core_simd/src/type_usizex2.rs b/crates/core_simd/src/type_usizex2.rs
index 083cd499a074..218d5a1056eb 100644
--- a/crates/core_simd/src/type_usizex2.rs
+++ b/crates/core_simd/src/type_usizex2.rs
@@ -1,4 +1,7 @@
-define_type! { struct usizex2([usize; 2]); }
+define_type! {
+    #[doc = "Vector of two `usize` types"]
+    struct usizex2([usize; 2]);
+}
 
 #[cfg(all(target_arch = "x86", target_pointer_width = "64"))]
 from_aligned! { unsafe usizex2 |bidirectional| core::arch::x86::__m128i }
diff --git a/crates/core_simd/src/type_usizex4.rs b/crates/core_simd/src/type_usizex4.rs
index 3301039a05a1..60160ecc217b 100644
--- a/crates/core_simd/src/type_usizex4.rs
+++ b/crates/core_simd/src/type_usizex4.rs
@@ -1,4 +1,7 @@
-define_type! { struct usizex4([usize; 4]); }
+define_type! {
+    #[doc = "Vector of four `usize` types"]
+    struct usizex4([usize; 4]);
+}
 
 #[cfg(all(target_arch = "x86", target_pointer_width = "32"))]
 from_aligned! { unsafe usizex4 |bidirectional| core::arch::x86::__m128i }
diff --git a/crates/core_simd/src/type_usizex8.rs b/crates/core_simd/src/type_usizex8.rs
index e21b1ea6b7a0..5f5a445f4308 100644
--- a/crates/core_simd/src/type_usizex8.rs
+++ b/crates/core_simd/src/type_usizex8.rs
@@ -1,4 +1,7 @@
-define_type! { struct usizex8([usize; 8]); }
+define_type! {
+    #[doc = "Vector of eight `usize` types"]
+    struct usizex8([usize; 8]);
+}
 
 #[cfg(all(target_arch = "x86", target_pointer_width = "32"))]
 from_aligned! { unsafe usizex8 |bidirectional| core::arch::x86::__m256i }

From c74eec7e25d0ab505dbb8f31d2e63839e5384c15 Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Wed, 23 Sep 2020 08:11:43 -0400
Subject: [PATCH 007/249] Apply suggestions from code review

Co-authored-by: Lokathor <zefria@gmail.com>
---
 crates/core_simd/src/lib.rs    | 2 +-
 crates/core_simd/src/macros.rs | 9 +++++----
 2 files changed, 6 insertions(+), 5 deletions(-)

diff --git a/crates/core_simd/src/lib.rs b/crates/core_simd/src/lib.rs
index 8db640157eed..b4fc58476f8f 100644
--- a/crates/core_simd/src/lib.rs
+++ b/crates/core_simd/src/lib.rs
@@ -1,5 +1,5 @@
 #![feature(repr_simd)]
-
+#![warn(missing_docs)]
 #[macro_use]
 mod macros;
 
diff --git a/crates/core_simd/src/macros.rs b/crates/core_simd/src/macros.rs
index 64d9c4b217f7..91c1071a2ee5 100644
--- a/crates/core_simd/src/macros.rs
+++ b/crates/core_simd/src/macros.rs
@@ -69,6 +69,7 @@ macro_rules! define_type {
 
         // splat
         impl From<$type> for $name {
+            #[inline]
             fn from(value: $type) -> Self {
                 Self::splat(value)
             }
@@ -126,16 +127,16 @@ macro_rules! define_type {
         pub struct $name($($itype),*);
 
         impl $name {
-            /// Construct a vector by setting each lane to a single value.
+            /// Construct a vector by setting all lanes to the given value.
             #[inline]
-            pub fn splat(value: $type) -> Self {
+            pub const fn splat(value: $type) -> Self {
                 Self($(value as $itype),*)
             }
 
-            /// Construct a vector by setting each lane.
+            /// Construct a vector by setting each lane to the given values.
             #[allow(clippy::too_many_arguments)]
             #[inline]
-            pub fn new($($ivar: $itype),*) -> Self {
+            pub const fn new($($ivar: $itype),*) -> Self {
                 Self($($ivar),*)
             }
         }

From 543bcd37e91afc1271befde3a33d6a73abc204d4 Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Wed, 23 Sep 2020 10:21:19 -0400
Subject: [PATCH 008/249] Combine vector definition modules by scalar type

---
 crates/core_simd/src/lib.rs           | 61 +++++++++++++--------------
 crates/core_simd/src/type_f32x16.rs   | 12 ------
 crates/core_simd/src/type_f32x2.rs    |  4 --
 crates/core_simd/src/type_f32x4.rs    | 10 -----
 crates/core_simd/src/type_f32x8.rs    | 10 -----
 crates/core_simd/src/type_f64x2.rs    | 10 -----
 crates/core_simd/src/type_f64x4.rs    | 10 -----
 crates/core_simd/src/type_f64x8.rs    | 12 ------
 crates/core_simd/src/type_i128x2.rs   | 10 -----
 crates/core_simd/src/type_i128x4.rs   | 12 ------
 crates/core_simd/src/type_i16x16.rs   | 10 -----
 crates/core_simd/src/type_i16x2.rs    |  4 --
 crates/core_simd/src/type_i16x32.rs   | 12 ------
 crates/core_simd/src/type_i16x4.rs    |  4 --
 crates/core_simd/src/type_i16x8.rs    | 10 -----
 crates/core_simd/src/type_i32x16.rs   | 12 ------
 crates/core_simd/src/type_i32x2.rs    |  4 --
 crates/core_simd/src/type_i32x4.rs    | 10 -----
 crates/core_simd/src/type_i32x8.rs    | 10 -----
 crates/core_simd/src/type_i64x2.rs    | 10 -----
 crates/core_simd/src/type_i64x4.rs    | 10 -----
 crates/core_simd/src/type_i64x8.rs    | 12 ------
 crates/core_simd/src/type_i8x16.rs    | 10 -----
 crates/core_simd/src/type_i8x2.rs     |  4 --
 crates/core_simd/src/type_i8x32.rs    | 10 -----
 crates/core_simd/src/type_i8x4.rs     |  4 --
 crates/core_simd/src/type_i8x64.rs    | 12 ------
 crates/core_simd/src/type_i8x8.rs     |  4 --
 crates/core_simd/src/type_isizex2.rs  | 10 -----
 crates/core_simd/src/type_isizex4.rs  | 16 -------
 crates/core_simd/src/type_isizex8.rs  | 18 --------
 crates/core_simd/src/type_u128x2.rs   | 10 -----
 crates/core_simd/src/type_u128x4.rs   | 12 ------
 crates/core_simd/src/type_u16x16.rs   | 10 -----
 crates/core_simd/src/type_u16x2.rs    |  4 --
 crates/core_simd/src/type_u16x32.rs   | 12 ------
 crates/core_simd/src/type_u16x4.rs    |  4 --
 crates/core_simd/src/type_u16x8.rs    | 10 -----
 crates/core_simd/src/type_u32x16.rs   | 12 ------
 crates/core_simd/src/type_u32x2.rs    |  4 --
 crates/core_simd/src/type_u32x4.rs    | 10 -----
 crates/core_simd/src/type_u32x8.rs    | 10 -----
 crates/core_simd/src/type_u64x2.rs    | 10 -----
 crates/core_simd/src/type_u64x4.rs    | 10 -----
 crates/core_simd/src/type_u64x8.rs    | 12 ------
 crates/core_simd/src/type_u8x16.rs    | 10 -----
 crates/core_simd/src/type_u8x2.rs     |  4 --
 crates/core_simd/src/type_u8x32.rs    | 10 -----
 crates/core_simd/src/type_u8x4.rs     |  4 --
 crates/core_simd/src/type_u8x64.rs    | 12 ------
 crates/core_simd/src/type_u8x8.rs     |  4 --
 crates/core_simd/src/type_usizex2.rs  | 10 -----
 crates/core_simd/src/type_usizex4.rs  | 16 -------
 crates/core_simd/src/type_usizex8.rs  | 18 --------
 crates/core_simd/src/vectors_f32.rs   | 39 +++++++++++++++++
 crates/core_simd/src/vectors_f64.rs   | 34 +++++++++++++++
 crates/core_simd/src/vectors_i128.rs  | 23 ++++++++++
 crates/core_simd/src/vectors_i16.rs   | 44 +++++++++++++++++++
 crates/core_simd/src/vectors_i32.rs   | 39 +++++++++++++++++
 crates/core_simd/src/vectors_i64.rs   | 34 +++++++++++++++
 crates/core_simd/src/vectors_i8.rs    | 49 +++++++++++++++++++++
 crates/core_simd/src/vectors_isize.rs | 46 ++++++++++++++++++++
 crates/core_simd/src/vectors_u128.rs  | 23 ++++++++++
 crates/core_simd/src/vectors_u16.rs   | 44 +++++++++++++++++++
 crates/core_simd/src/vectors_u32.rs   | 39 +++++++++++++++++
 crates/core_simd/src/vectors_u64.rs   | 34 +++++++++++++++
 crates/core_simd/src/vectors_u8.rs    | 49 +++++++++++++++++++++
 crates/core_simd/src/vectors_usize.rs | 46 ++++++++++++++++++++
 68 files changed, 573 insertions(+), 535 deletions(-)
 delete mode 100644 crates/core_simd/src/type_f32x16.rs
 delete mode 100644 crates/core_simd/src/type_f32x2.rs
 delete mode 100644 crates/core_simd/src/type_f32x4.rs
 delete mode 100644 crates/core_simd/src/type_f32x8.rs
 delete mode 100644 crates/core_simd/src/type_f64x2.rs
 delete mode 100644 crates/core_simd/src/type_f64x4.rs
 delete mode 100644 crates/core_simd/src/type_f64x8.rs
 delete mode 100644 crates/core_simd/src/type_i128x2.rs
 delete mode 100644 crates/core_simd/src/type_i128x4.rs
 delete mode 100644 crates/core_simd/src/type_i16x16.rs
 delete mode 100644 crates/core_simd/src/type_i16x2.rs
 delete mode 100644 crates/core_simd/src/type_i16x32.rs
 delete mode 100644 crates/core_simd/src/type_i16x4.rs
 delete mode 100644 crates/core_simd/src/type_i16x8.rs
 delete mode 100644 crates/core_simd/src/type_i32x16.rs
 delete mode 100644 crates/core_simd/src/type_i32x2.rs
 delete mode 100644 crates/core_simd/src/type_i32x4.rs
 delete mode 100644 crates/core_simd/src/type_i32x8.rs
 delete mode 100644 crates/core_simd/src/type_i64x2.rs
 delete mode 100644 crates/core_simd/src/type_i64x4.rs
 delete mode 100644 crates/core_simd/src/type_i64x8.rs
 delete mode 100644 crates/core_simd/src/type_i8x16.rs
 delete mode 100644 crates/core_simd/src/type_i8x2.rs
 delete mode 100644 crates/core_simd/src/type_i8x32.rs
 delete mode 100644 crates/core_simd/src/type_i8x4.rs
 delete mode 100644 crates/core_simd/src/type_i8x64.rs
 delete mode 100644 crates/core_simd/src/type_i8x8.rs
 delete mode 100644 crates/core_simd/src/type_isizex2.rs
 delete mode 100644 crates/core_simd/src/type_isizex4.rs
 delete mode 100644 crates/core_simd/src/type_isizex8.rs
 delete mode 100644 crates/core_simd/src/type_u128x2.rs
 delete mode 100644 crates/core_simd/src/type_u128x4.rs
 delete mode 100644 crates/core_simd/src/type_u16x16.rs
 delete mode 100644 crates/core_simd/src/type_u16x2.rs
 delete mode 100644 crates/core_simd/src/type_u16x32.rs
 delete mode 100644 crates/core_simd/src/type_u16x4.rs
 delete mode 100644 crates/core_simd/src/type_u16x8.rs
 delete mode 100644 crates/core_simd/src/type_u32x16.rs
 delete mode 100644 crates/core_simd/src/type_u32x2.rs
 delete mode 100644 crates/core_simd/src/type_u32x4.rs
 delete mode 100644 crates/core_simd/src/type_u32x8.rs
 delete mode 100644 crates/core_simd/src/type_u64x2.rs
 delete mode 100644 crates/core_simd/src/type_u64x4.rs
 delete mode 100644 crates/core_simd/src/type_u64x8.rs
 delete mode 100644 crates/core_simd/src/type_u8x16.rs
 delete mode 100644 crates/core_simd/src/type_u8x2.rs
 delete mode 100644 crates/core_simd/src/type_u8x32.rs
 delete mode 100644 crates/core_simd/src/type_u8x4.rs
 delete mode 100644 crates/core_simd/src/type_u8x64.rs
 delete mode 100644 crates/core_simd/src/type_u8x8.rs
 delete mode 100644 crates/core_simd/src/type_usizex2.rs
 delete mode 100644 crates/core_simd/src/type_usizex4.rs
 delete mode 100644 crates/core_simd/src/type_usizex8.rs
 create mode 100644 crates/core_simd/src/vectors_f32.rs
 create mode 100644 crates/core_simd/src/vectors_f64.rs
 create mode 100644 crates/core_simd/src/vectors_i128.rs
 create mode 100644 crates/core_simd/src/vectors_i16.rs
 create mode 100644 crates/core_simd/src/vectors_i32.rs
 create mode 100644 crates/core_simd/src/vectors_i64.rs
 create mode 100644 crates/core_simd/src/vectors_i8.rs
 create mode 100644 crates/core_simd/src/vectors_isize.rs
 create mode 100644 crates/core_simd/src/vectors_u128.rs
 create mode 100644 crates/core_simd/src/vectors_u16.rs
 create mode 100644 crates/core_simd/src/vectors_u32.rs
 create mode 100644 crates/core_simd/src/vectors_u64.rs
 create mode 100644 crates/core_simd/src/vectors_u8.rs
 create mode 100644 crates/core_simd/src/vectors_usize.rs

diff --git a/crates/core_simd/src/lib.rs b/crates/core_simd/src/lib.rs
index b4fc58476f8f..a5dae4b02792 100644
--- a/crates/core_simd/src/lib.rs
+++ b/crates/core_simd/src/lib.rs
@@ -1,36 +1,35 @@
+#![no_std]
 #![feature(repr_simd)]
 #![warn(missing_docs)]
+
 #[macro_use]
 mod macros;
 
-macro_rules! import_types {
-    { $($mod:ident,)* } => {
-        $(
-        mod $mod;
-        pub use $mod::*;
-        )*
-    }
-}
-
-import_types! {
-    type_u8x2,   type_u8x4,   type_u8x8,   type_u8x16,   type_u8x32,   type_u8x64,
-    type_i8x2,   type_i8x4,   type_i8x8,   type_i8x16,   type_i8x32,   type_i8x64,
-    type_u16x2,  type_u16x4,  type_u16x8,  type_u16x16,  type_u16x32,
-    type_i16x2,  type_i16x4,  type_i16x8,  type_i16x16,  type_i16x32,
-    type_u32x2,  type_u32x4,  type_u32x8,  type_u32x16,
-    type_i32x2,  type_i32x4,  type_i32x8,  type_i32x16,
-    type_u64x2,  type_u64x4,  type_u64x8,
-    type_i64x2,  type_i64x4,  type_i64x8,
-    type_u128x2, type_u128x4,
-    type_i128x2, type_i128x4,
-}
-
-import_types! {
-    type_usizex2, type_usizex4, type_usizex8,
-    type_isizex2, type_isizex4, type_isizex8,
-}
-
-import_types! {
-    type_f32x2, type_f32x4, type_f32x8, type_f32x16,
-    type_f64x2, type_f64x4, type_f64x8,
-}
+mod vectors_u8;
+pub use vectors_u8::*;
+mod vectors_u16;
+pub use vectors_u16::*;
+mod vectors_u32;
+pub use vectors_u32::*;
+mod vectors_u64;
+pub use vectors_u64::*;
+mod vectors_u128;
+pub use vectors_u128::*;
+mod vectors_usize;
+pub use vectors_usize::*;
+mod vectors_i8;
+pub use vectors_i8::*;
+mod vectors_i16;
+pub use vectors_i16::*;
+mod vectors_i32;
+pub use vectors_i32::*;
+mod vectors_i64;
+pub use vectors_i64::*;
+mod vectors_i128;
+pub use vectors_i128::*;
+mod vectors_isize;
+pub use vectors_isize::*;
+mod vectors_f32;
+pub use vectors_f32::*;
+mod vectors_f64;
+pub use vectors_f64::*;
diff --git a/crates/core_simd/src/type_f32x16.rs b/crates/core_simd/src/type_f32x16.rs
deleted file mode 100644
index 45fb4a3175bb..000000000000
--- a/crates/core_simd/src/type_f32x16.rs
+++ /dev/null
@@ -1,12 +0,0 @@
-define_type! {
-    #[doc = "Vector of 16 `f32` types"]
-    struct f32x16([f32; 16]);
-}
-
-/*
-#[cfg(target_arch = "x86")]
-from_aligned! { unsafe f32x16 |bidirectional| core::arch::x86::__m512 }
-
-#[cfg(target_arch = "x86_64")]
-from_aligned! { unsafe f32x16 |bidirectional| core::arch::x86_64::__m512 }
-*/
diff --git a/crates/core_simd/src/type_f32x2.rs b/crates/core_simd/src/type_f32x2.rs
deleted file mode 100644
index 8b60a2ee76f4..000000000000
--- a/crates/core_simd/src/type_f32x2.rs
+++ /dev/null
@@ -1,4 +0,0 @@
-define_type! {
-    #[doc = "Vector of two `f32` types"]
-    struct f32x2([f32; 2]);
-}
diff --git a/crates/core_simd/src/type_f32x4.rs b/crates/core_simd/src/type_f32x4.rs
deleted file mode 100644
index 452e607732cf..000000000000
--- a/crates/core_simd/src/type_f32x4.rs
+++ /dev/null
@@ -1,10 +0,0 @@
-define_type! {
-    #[doc = "Vector of four `f32` types"]
-    struct f32x4([f32; 4]);
-}
-
-#[cfg(target_arch = "x86")]
-from_aligned! { unsafe f32x4 |bidirectional| core::arch::x86::__m128 }
-
-#[cfg(target_arch = "x86_64")]
-from_aligned! { unsafe f32x4 |bidirectional| core::arch::x86_64::__m128 }
diff --git a/crates/core_simd/src/type_f32x8.rs b/crates/core_simd/src/type_f32x8.rs
deleted file mode 100644
index 790a546e4e73..000000000000
--- a/crates/core_simd/src/type_f32x8.rs
+++ /dev/null
@@ -1,10 +0,0 @@
-define_type! {
-    #[doc = "Vector of eight `f32` types"]
-    struct f32x8([f32; 8]);
-}
-
-#[cfg(target_arch = "x86")]
-from_aligned! { unsafe f32x8 |bidirectional| core::arch::x86::__m256 }
-
-#[cfg(target_arch = "x86_64")]
-from_aligned! { unsafe f32x8 |bidirectional| core::arch::x86_64::__m256 }
diff --git a/crates/core_simd/src/type_f64x2.rs b/crates/core_simd/src/type_f64x2.rs
deleted file mode 100644
index 0c349f38c862..000000000000
--- a/crates/core_simd/src/type_f64x2.rs
+++ /dev/null
@@ -1,10 +0,0 @@
-define_type! {
-    #[doc = "Vector of two `f64` types"]
-    struct f64x2([f64; 2]);
-}
-
-#[cfg(target_arch = "x86")]
-from_aligned! { unsafe f64x2 |bidirectional| core::arch::x86::__m128d }
-
-#[cfg(target_arch = "x86_64")]
-from_aligned! { unsafe f64x2 |bidirectional| core::arch::x86_64::__m128d }
diff --git a/crates/core_simd/src/type_f64x4.rs b/crates/core_simd/src/type_f64x4.rs
deleted file mode 100644
index ec6b46bc8c84..000000000000
--- a/crates/core_simd/src/type_f64x4.rs
+++ /dev/null
@@ -1,10 +0,0 @@
-define_type! {
-    #[doc = "Vector of four `f64` types"]
-    struct f64x4([f64; 4]);
-}
-
-#[cfg(target_arch = "x86")]
-from_aligned! { unsafe f64x4 |bidirectional| core::arch::x86::__m256d }
-
-#[cfg(target_arch = "x86_64")]
-from_aligned! { unsafe f64x4 |bidirectional| core::arch::x86_64::__m256d }
diff --git a/crates/core_simd/src/type_f64x8.rs b/crates/core_simd/src/type_f64x8.rs
deleted file mode 100644
index dd65dc6b39ab..000000000000
--- a/crates/core_simd/src/type_f64x8.rs
+++ /dev/null
@@ -1,12 +0,0 @@
-define_type! {
-    #[doc = "Vector of eight `f64` types"]
-    struct f64x8([f64; 8]);
-}
-
-/*
-#[cfg(target_arch = "x86")]
-from_aligned! { unsafe f64x8 |bidirectional| core::arch::x86::__m512d }
-
-#[cfg(target_arch = "x86_64")]
-from_aligned! { unsafe f64x8 |bidirectional| core::arch::x86_64::__m512d }
-*/
diff --git a/crates/core_simd/src/type_i128x2.rs b/crates/core_simd/src/type_i128x2.rs
deleted file mode 100644
index 1ac736ddf3a2..000000000000
--- a/crates/core_simd/src/type_i128x2.rs
+++ /dev/null
@@ -1,10 +0,0 @@
-define_type! {
-    #[doc = "Vector of two `i128` types"]
-    struct i128x2([i128; 2]);
-}
-
-#[cfg(target_arch = "x86")]
-from_aligned! { unsafe i128x2 |bidirectional| core::arch::x86::__m256i }
-
-#[cfg(target_arch = "x86_64")]
-from_aligned! { unsafe i128x2 |bidirectional| core::arch::x86_64::__m256i }
diff --git a/crates/core_simd/src/type_i128x4.rs b/crates/core_simd/src/type_i128x4.rs
deleted file mode 100644
index 6cef039d947a..000000000000
--- a/crates/core_simd/src/type_i128x4.rs
+++ /dev/null
@@ -1,12 +0,0 @@
-define_type! {
-    #[doc = "Vector of four `i128` types"]
-    struct i128x4([i128; 4]);
-}
-
-/*
-#[cfg(target_arch = "x86")]
-from_aligned! { unsafe i128x4 |bidirectional| core::arch::x86::__m512i }
-
-#[cfg(target_arch = "x86_64")]
-from_aligned! { unsafe i128x4 |bidirectional| core::arch::x86_64::__m512i }
-*/
diff --git a/crates/core_simd/src/type_i16x16.rs b/crates/core_simd/src/type_i16x16.rs
deleted file mode 100644
index 1721286128fe..000000000000
--- a/crates/core_simd/src/type_i16x16.rs
+++ /dev/null
@@ -1,10 +0,0 @@
-define_type! {
-    #[doc = "Vector of 16 `i16` types"]
-    struct i16x16([i16; 16]);
-}
-
-#[cfg(target_arch = "x86")]
-from_aligned! { unsafe i16x16 |bidirectional| core::arch::x86::__m256i }
-
-#[cfg(target_arch = "x86_64")]
-from_aligned! { unsafe i16x16 |bidirectional| core::arch::x86_64::__m256i }
diff --git a/crates/core_simd/src/type_i16x2.rs b/crates/core_simd/src/type_i16x2.rs
deleted file mode 100644
index 7ccbbe7f2d0a..000000000000
--- a/crates/core_simd/src/type_i16x2.rs
+++ /dev/null
@@ -1,4 +0,0 @@
-define_type! {
-    #[doc = "Vector of two `i16` types"]
-    struct i16x2([i16; 2]);
-}
diff --git a/crates/core_simd/src/type_i16x32.rs b/crates/core_simd/src/type_i16x32.rs
deleted file mode 100644
index 349d094a1bc9..000000000000
--- a/crates/core_simd/src/type_i16x32.rs
+++ /dev/null
@@ -1,12 +0,0 @@
-define_type! {
-    #[doc = "Vector of 32 `i16` types"]
-    struct i16x32([i16; 32]);
-}
-
-/*
-#[cfg(target_arch = "x86")]
-from_aligned! { unsafe u8x32 |bidirectional| core::arch::x86::__m512i }
-
-#[cfg(target_arch = "x86_64")]
-from_aligned! { unsafe u8x32 |bidirectional| core::arch::x86_64::__m512i }
-*/
diff --git a/crates/core_simd/src/type_i16x4.rs b/crates/core_simd/src/type_i16x4.rs
deleted file mode 100644
index 348bc4c53a94..000000000000
--- a/crates/core_simd/src/type_i16x4.rs
+++ /dev/null
@@ -1,4 +0,0 @@
-define_type! {
-    #[doc = "Vector of four `i16` types"]
-    struct i16x4([i16; 4]);
-}
diff --git a/crates/core_simd/src/type_i16x8.rs b/crates/core_simd/src/type_i16x8.rs
deleted file mode 100644
index 87ded0f3a49c..000000000000
--- a/crates/core_simd/src/type_i16x8.rs
+++ /dev/null
@@ -1,10 +0,0 @@
-define_type! {
-    #[doc = "Vector of eight `i16` types"]
-    struct i16x8([i16; 8]);
-}
-
-#[cfg(target_arch = "x86")]
-from_aligned! { unsafe i16x8 |bidirectional| core::arch::x86::__m128i }
-
-#[cfg(target_arch = "x86_64")]
-from_aligned! { unsafe i16x8 |bidirectional| core::arch::x86_64::__m128i }
diff --git a/crates/core_simd/src/type_i32x16.rs b/crates/core_simd/src/type_i32x16.rs
deleted file mode 100644
index 7c52e9cb9e55..000000000000
--- a/crates/core_simd/src/type_i32x16.rs
+++ /dev/null
@@ -1,12 +0,0 @@
-define_type! {
-    #[doc = "Vector of 16 `i32` types"]
-    struct i32x16([i32; 16]);
-}
-
-/*
-#[cfg(target_arch = "x86")]
-from_aligned! { unsafe u32x16 |bidirectional| core::arch::x86::__m512i }
-
-#[cfg(target_arch = "x86_64")]
-from_aligned! { unsafe u32x16 |bidirectional| core::arch::x86_64::__m512i }
-*/
diff --git a/crates/core_simd/src/type_i32x2.rs b/crates/core_simd/src/type_i32x2.rs
deleted file mode 100644
index e9845ae75c73..000000000000
--- a/crates/core_simd/src/type_i32x2.rs
+++ /dev/null
@@ -1,4 +0,0 @@
-define_type! {
-    #[doc = "Vector of two `i32` types"]
-    struct i32x2([i32; 2]);
-}
diff --git a/crates/core_simd/src/type_i32x4.rs b/crates/core_simd/src/type_i32x4.rs
deleted file mode 100644
index 47374f7ce439..000000000000
--- a/crates/core_simd/src/type_i32x4.rs
+++ /dev/null
@@ -1,10 +0,0 @@
-define_type! {
-    #[doc = "Vector of four `i32` types"]
-    struct i32x4([i32; 4]);
-}
-
-#[cfg(target_arch = "x86")]
-from_aligned! { unsafe i32x4 |bidirectional| core::arch::x86::__m128i }
-
-#[cfg(target_arch = "x86_64")]
-from_aligned! { unsafe i32x4 |bidirectional| core::arch::x86_64::__m128i }
diff --git a/crates/core_simd/src/type_i32x8.rs b/crates/core_simd/src/type_i32x8.rs
deleted file mode 100644
index 79b4ea180c31..000000000000
--- a/crates/core_simd/src/type_i32x8.rs
+++ /dev/null
@@ -1,10 +0,0 @@
-define_type! {
-    #[doc = "Vector of eight `i32` types"]
-    struct i32x8([i32; 8]);
-}
-
-#[cfg(target_arch = "x86")]
-from_aligned! { unsafe i32x8 |bidirectional| core::arch::x86::__m256i }
-
-#[cfg(target_arch = "x86_64")]
-from_aligned! { unsafe i32x8 |bidirectional| core::arch::x86_64::__m256i }
diff --git a/crates/core_simd/src/type_i64x2.rs b/crates/core_simd/src/type_i64x2.rs
deleted file mode 100644
index 7268ad5dfbe8..000000000000
--- a/crates/core_simd/src/type_i64x2.rs
+++ /dev/null
@@ -1,10 +0,0 @@
-define_type! {
-    #[doc = "Vector of two `i64` types"]
-    struct i64x2([i64; 2]);
-}
-
-#[cfg(target_arch = "x86")]
-from_aligned! { unsafe i64x2 |bidirectional| core::arch::x86::__m128i }
-
-#[cfg(target_arch = "x86_64")]
-from_aligned! { unsafe i64x2 |bidirectional| core::arch::x86_64::__m128i }
diff --git a/crates/core_simd/src/type_i64x4.rs b/crates/core_simd/src/type_i64x4.rs
deleted file mode 100644
index 3535783c3000..000000000000
--- a/crates/core_simd/src/type_i64x4.rs
+++ /dev/null
@@ -1,10 +0,0 @@
-define_type! {
-    #[doc = "Vector of four `i64` types"]
-    struct i64x4([i64; 4]);
-}
-
-#[cfg(target_arch = "x86")]
-from_aligned! { unsafe i64x4 |bidirectional| core::arch::x86::__m256i }
-
-#[cfg(target_arch = "x86_64")]
-from_aligned! { unsafe i64x4 |bidirectional| core::arch::x86_64::__m256i }
diff --git a/crates/core_simd/src/type_i64x8.rs b/crates/core_simd/src/type_i64x8.rs
deleted file mode 100644
index 42d28db8c551..000000000000
--- a/crates/core_simd/src/type_i64x8.rs
+++ /dev/null
@@ -1,12 +0,0 @@
-define_type! {
-    #[doc = "Vector of eight `i64` types"]
-    struct i64x8([i64; 8]);
-}
-
-/*
-#[cfg(target_arch = "x86")]
-from_aligned! { unsafe i64x8 |bidirectional| core::arch::x86::__m512i }
-
-#[cfg(target_arch = "x86_64")]
-from_aligned! { unsafe i64x8 |bidirectional| core::arch::x86_64::__m512i }
-*/
diff --git a/crates/core_simd/src/type_i8x16.rs b/crates/core_simd/src/type_i8x16.rs
deleted file mode 100644
index d7aadd4d7ab2..000000000000
--- a/crates/core_simd/src/type_i8x16.rs
+++ /dev/null
@@ -1,10 +0,0 @@
-define_type! {
-    #[doc = "Vector of 16 `i8` types"]
-    struct i8x16([i8; 16]);
-}
-
-#[cfg(target_arch = "x86")]
-from_aligned! { unsafe i8x16 |bidirectional| core::arch::x86::__m128i }
-
-#[cfg(target_arch = "x86_64")]
-from_aligned! { unsafe i8x16 |bidirectional| core::arch::x86_64::__m128i }
diff --git a/crates/core_simd/src/type_i8x2.rs b/crates/core_simd/src/type_i8x2.rs
deleted file mode 100644
index aca22993c2e3..000000000000
--- a/crates/core_simd/src/type_i8x2.rs
+++ /dev/null
@@ -1,4 +0,0 @@
-define_type! {
-    #[doc = "Vector of two `i8` types"]
-    struct i8x2([i8; 2]);
-}
diff --git a/crates/core_simd/src/type_i8x32.rs b/crates/core_simd/src/type_i8x32.rs
deleted file mode 100644
index a323565c85cf..000000000000
--- a/crates/core_simd/src/type_i8x32.rs
+++ /dev/null
@@ -1,10 +0,0 @@
-define_type! {
-    #[doc = "Vector of 32 `i8` types"]
-    struct i8x32([i8; 32]);
-}
-
-#[cfg(target_arch = "x86")]
-from_aligned! { unsafe i8x32 |bidirectional| core::arch::x86::__m256i }
-
-#[cfg(target_arch = "x86_64")]
-from_aligned! { unsafe i8x32 |bidirectional| core::arch::x86_64::__m256i }
diff --git a/crates/core_simd/src/type_i8x4.rs b/crates/core_simd/src/type_i8x4.rs
deleted file mode 100644
index 246703ebc50d..000000000000
--- a/crates/core_simd/src/type_i8x4.rs
+++ /dev/null
@@ -1,4 +0,0 @@
-define_type! {
-    #[doc = "Vector of four `i8` types"]
-    struct i8x4([i8; 4]);
-}
diff --git a/crates/core_simd/src/type_i8x64.rs b/crates/core_simd/src/type_i8x64.rs
deleted file mode 100644
index 26934df2a484..000000000000
--- a/crates/core_simd/src/type_i8x64.rs
+++ /dev/null
@@ -1,12 +0,0 @@
-define_type! {
-    #[doc = "Vector of 64 `i8` types"]
-    struct i8x64([i8; 64]);
-}
-
-/*
-#[cfg(target_arch = "x86")]
-from_aligned! { unsafe u8x64 |bidirectional| core::arch::x86::__m512i }
-
-#[cfg(target_arch = "x86_64")]
-from_aligned! { unsafe u8x64 |bidirectional| core::arch::x86_64::__m512i }
-*/
diff --git a/crates/core_simd/src/type_i8x8.rs b/crates/core_simd/src/type_i8x8.rs
deleted file mode 100644
index bc30e2daf840..000000000000
--- a/crates/core_simd/src/type_i8x8.rs
+++ /dev/null
@@ -1,4 +0,0 @@
-define_type! {
-    #[doc = "Vector of eight `i8` types"]
-    struct i8x8([i8; 8]);
-}
diff --git a/crates/core_simd/src/type_isizex2.rs b/crates/core_simd/src/type_isizex2.rs
deleted file mode 100644
index 464f64955522..000000000000
--- a/crates/core_simd/src/type_isizex2.rs
+++ /dev/null
@@ -1,10 +0,0 @@
-define_type! {
-    #[doc = "Vector of two `isize` types"]
-    struct isizex2([isize; 2]);
-}
-
-#[cfg(all(target_arch = "x86", target_pointer_width = "64"))]
-from_aligned! { unsafe isizex2 |bidirectional| core::arch::x86::__m128i }
-
-#[cfg(all(target_arch = "x86_64", target_pointer_width = "64"))]
-from_aligned! { unsafe isizex2 |bidirectional| core::arch::x86_64::__m128i }
diff --git a/crates/core_simd/src/type_isizex4.rs b/crates/core_simd/src/type_isizex4.rs
deleted file mode 100644
index 3be457393ec2..000000000000
--- a/crates/core_simd/src/type_isizex4.rs
+++ /dev/null
@@ -1,16 +0,0 @@
-define_type! {
-    #[doc = "Vector of four `isize` types"]
-    struct isizex4([isize; 4]);
-}
-
-#[cfg(all(target_arch = "x86", target_pointer_width = "32"))]
-from_aligned! { unsafe isizex4 |bidirectional| core::arch::x86::__m128i }
-
-#[cfg(all(target_arch = "x86_64", target_pointer_width = "32"))]
-from_aligned! { unsafe isizex4 |bidirectional| core::arch::x86_64::__m128i }
-
-#[cfg(all(target_arch = "x86", target_pointer_width = "64"))]
-from_aligned! { unsafe isizex4 |bidirectional| core::arch::x86::__m256i }
-
-#[cfg(all(target_arch = "x86_64", target_pointer_width = "64"))]
-from_aligned! { unsafe isizex4 |bidirectional| core::arch::x86_64::__m256i }
diff --git a/crates/core_simd/src/type_isizex8.rs b/crates/core_simd/src/type_isizex8.rs
deleted file mode 100644
index e21c2cf624b0..000000000000
--- a/crates/core_simd/src/type_isizex8.rs
+++ /dev/null
@@ -1,18 +0,0 @@
-define_type! {
-    #[doc = "Vector of eight `isize` types"]
-    struct isizex8([isize; 8]);
-}
-
-#[cfg(all(target_arch = "x86", target_pointer_width = "32"))]
-from_aligned! { unsafe isizex8 |bidirectional| core::arch::x86::__m256i }
-
-#[cfg(all(target_arch = "x86_64", target_pointer_width = "32"))]
-from_aligned! { unsafe isizex8 |bidirectional| core::arch::x86_64::__m256i }
-
-/*
-#[cfg(all(target_arch = "x86", target_pointer_width = "64"))]
-from_aligned! { unsafe isizex8 |bidirectional| core::arch::x86::__m512i }
-
-#[cfg(all(target_arch = "x86_64", target_pointer_width = "64"))]
-from_aligned! { unsafe isizex8 |bidirectional| core::arch::x86_64::__m512i }
-*/
diff --git a/crates/core_simd/src/type_u128x2.rs b/crates/core_simd/src/type_u128x2.rs
deleted file mode 100644
index f448e4f0e626..000000000000
--- a/crates/core_simd/src/type_u128x2.rs
+++ /dev/null
@@ -1,10 +0,0 @@
-define_type! {
-    #[doc = "Vector of two `u128` types"]
-    struct u128x2([u128; 2]);
-}
-
-#[cfg(target_arch = "x86")]
-from_aligned! { unsafe u128x2 |bidirectional| core::arch::x86::__m256i }
-
-#[cfg(target_arch = "x86_64")]
-from_aligned! { unsafe u128x2 |bidirectional| core::arch::x86_64::__m256i }
diff --git a/crates/core_simd/src/type_u128x4.rs b/crates/core_simd/src/type_u128x4.rs
deleted file mode 100644
index 07466c720853..000000000000
--- a/crates/core_simd/src/type_u128x4.rs
+++ /dev/null
@@ -1,12 +0,0 @@
-define_type! {
-    #[doc = "Vector of four `u128` types"]
-    struct u128x4([u128; 4]);
-}
-
-/*
-#[cfg(target_arch = "x86")]
-from_aligned! { unsafe u128x4 |bidirectional| core::arch::x86::__m512i }
-
-#[cfg(target_arch = "x86_64")]
-from_aligned! { unsafe u128x4 |bidirectional| core::arch::x86_64::__m512i }
-*/
diff --git a/crates/core_simd/src/type_u16x16.rs b/crates/core_simd/src/type_u16x16.rs
deleted file mode 100644
index 5460c3740adf..000000000000
--- a/crates/core_simd/src/type_u16x16.rs
+++ /dev/null
@@ -1,10 +0,0 @@
-define_type! {
-    #[doc = "Vector of 16 `u16` types"]
-    struct u16x16([u16; 16]);
-}
-
-#[cfg(target_arch = "x86")]
-from_aligned! { unsafe u16x16 |bidirectional| core::arch::x86::__m256i }
-
-#[cfg(target_arch = "x86_64")]
-from_aligned! { unsafe u16x16 |bidirectional| core::arch::x86_64::__m256i }
diff --git a/crates/core_simd/src/type_u16x2.rs b/crates/core_simd/src/type_u16x2.rs
deleted file mode 100644
index 480be8b6eaef..000000000000
--- a/crates/core_simd/src/type_u16x2.rs
+++ /dev/null
@@ -1,4 +0,0 @@
-define_type! {
-    #[doc = "Vector of two `u16` types"]
-    struct u16x2([u16; 2]);
-}
diff --git a/crates/core_simd/src/type_u16x32.rs b/crates/core_simd/src/type_u16x32.rs
deleted file mode 100644
index 67d60535f776..000000000000
--- a/crates/core_simd/src/type_u16x32.rs
+++ /dev/null
@@ -1,12 +0,0 @@
-define_type! {
-    #[doc = "Vector of 32 `u16` types"]
-    struct u16x32([u16; 32]);
-}
-
-/*
-#[cfg(target_arch = "x86")]
-from_aligned! { unsafe u16x32 |bidirectional| core::arch::x86::__m512i }
-
-#[cfg(target_arch = "x86_64")]
-from_aligned! { unsafe u16x32 |bidirectional| core::arch::x86_64::__m512i }
-*/
diff --git a/crates/core_simd/src/type_u16x4.rs b/crates/core_simd/src/type_u16x4.rs
deleted file mode 100644
index 874891363da8..000000000000
--- a/crates/core_simd/src/type_u16x4.rs
+++ /dev/null
@@ -1,4 +0,0 @@
-define_type! {
-    #[doc = "Vector of four `u16` types"]
-    struct u16x4([u16; 4]);
-}
diff --git a/crates/core_simd/src/type_u16x8.rs b/crates/core_simd/src/type_u16x8.rs
deleted file mode 100644
index 5ba0af151e3f..000000000000
--- a/crates/core_simd/src/type_u16x8.rs
+++ /dev/null
@@ -1,10 +0,0 @@
-define_type! {
-    #[doc = "Vector of eight `u16` types"]
-    struct u16x8([u16; 8]);
-}
-
-#[cfg(target_arch = "x86")]
-from_aligned! { unsafe u16x8 |bidirectional| core::arch::x86::__m128i }
-
-#[cfg(target_arch = "x86_64")]
-from_aligned! { unsafe u16x8 |bidirectional| core::arch::x86_64::__m128i }
diff --git a/crates/core_simd/src/type_u32x16.rs b/crates/core_simd/src/type_u32x16.rs
deleted file mode 100644
index 40e557b2d6c9..000000000000
--- a/crates/core_simd/src/type_u32x16.rs
+++ /dev/null
@@ -1,12 +0,0 @@
-define_type! {
-    #[doc = "Vector of 16 `u32` types"]
-    struct u32x16([u32; 16]);
-}
-
-/*
-#[cfg(target_arch = "x86")]
-from_aligned! { unsafe u32x16 |bidirectional| core::arch::x86::__m512i }
-
-#[cfg(target_arch = "x86_64")]
-from_aligned! { unsafe u32x16 |bidirectional| core::arch::x86_64::__m512i }
-*/
diff --git a/crates/core_simd/src/type_u32x2.rs b/crates/core_simd/src/type_u32x2.rs
deleted file mode 100644
index 001fd31db39c..000000000000
--- a/crates/core_simd/src/type_u32x2.rs
+++ /dev/null
@@ -1,4 +0,0 @@
-define_type! {
-    #[doc = "Vector of two `u32` types"]
-    struct u32x2([u32; 2]);
-}
diff --git a/crates/core_simd/src/type_u32x4.rs b/crates/core_simd/src/type_u32x4.rs
deleted file mode 100644
index 0582b51ead14..000000000000
--- a/crates/core_simd/src/type_u32x4.rs
+++ /dev/null
@@ -1,10 +0,0 @@
-define_type! {
-    #[doc = "Vector of four `u32` types"]
-    struct u32x4([u32; 4]);
-}
-
-#[cfg(target_arch = "x86")]
-from_aligned! { unsafe u32x4 |bidirectional| core::arch::x86::__m128i }
-
-#[cfg(target_arch = "x86_64")]
-from_aligned! { unsafe u32x4 |bidirectional| core::arch::x86_64::__m128i }
diff --git a/crates/core_simd/src/type_u32x8.rs b/crates/core_simd/src/type_u32x8.rs
deleted file mode 100644
index 686833c250c4..000000000000
--- a/crates/core_simd/src/type_u32x8.rs
+++ /dev/null
@@ -1,10 +0,0 @@
-define_type! {
-    #[doc = "Vector of eight `u32` types"]
-    struct u32x8([u32; 8]);
-}
-
-#[cfg(target_arch = "x86")]
-from_aligned! { unsafe u32x8 |bidirectional| core::arch::x86::__m256i }
-
-#[cfg(target_arch = "x86_64")]
-from_aligned! { unsafe u32x8 |bidirectional| core::arch::x86_64::__m256i }
diff --git a/crates/core_simd/src/type_u64x2.rs b/crates/core_simd/src/type_u64x2.rs
deleted file mode 100644
index e3ba3dc03b51..000000000000
--- a/crates/core_simd/src/type_u64x2.rs
+++ /dev/null
@@ -1,10 +0,0 @@
-define_type! {
-    #[doc = "Vector of two `u64` types"]
-    struct u64x2([u64; 2]);
-}
-
-#[cfg(target_arch = "x86")]
-from_aligned! { unsafe u64x2 |bidirectional| core::arch::x86::__m128i }
-
-#[cfg(target_arch = "x86_64")]
-from_aligned! { unsafe u64x2 |bidirectional| core::arch::x86_64::__m128i }
diff --git a/crates/core_simd/src/type_u64x4.rs b/crates/core_simd/src/type_u64x4.rs
deleted file mode 100644
index 005f26012ad5..000000000000
--- a/crates/core_simd/src/type_u64x4.rs
+++ /dev/null
@@ -1,10 +0,0 @@
-define_type! {
-    #[doc = "Vector of four `u64` types"]
-    struct u64x4([u64; 4]);
-}
-
-#[cfg(target_arch = "x86")]
-from_aligned! { unsafe u64x4 |bidirectional| core::arch::x86::__m256i }
-
-#[cfg(target_arch = "x86_64")]
-from_aligned! { unsafe u64x4 |bidirectional| core::arch::x86_64::__m256i }
diff --git a/crates/core_simd/src/type_u64x8.rs b/crates/core_simd/src/type_u64x8.rs
deleted file mode 100644
index fbb41f36e5da..000000000000
--- a/crates/core_simd/src/type_u64x8.rs
+++ /dev/null
@@ -1,12 +0,0 @@
-define_type! {
-    #[doc = "Vector of eight `u64` types"]
-    struct u64x8([u64; 8]);
-}
-
-/*
-#[cfg(target_arch = "x86")]
-from_aligned! { unsafe u64x8 |bidirectional| core::arch::x86::__m512i }
-
-#[cfg(target_arch = "x86_64")]
-from_aligned! { unsafe u64x8 |bidirectional| core::arch::x86_64::__m512i }
-*/
diff --git a/crates/core_simd/src/type_u8x16.rs b/crates/core_simd/src/type_u8x16.rs
deleted file mode 100644
index a44b74c122b2..000000000000
--- a/crates/core_simd/src/type_u8x16.rs
+++ /dev/null
@@ -1,10 +0,0 @@
-define_type! {
-    #[doc = "Vector of 16 `u8` types"]
-    struct u8x16([u8; 16]);
-}
-
-#[cfg(target_arch = "x86")]
-from_aligned! { unsafe u8x16 |bidirectional| core::arch::x86::__m128i }
-
-#[cfg(target_arch = "x86_64")]
-from_aligned! { unsafe u8x16 |bidirectional| core::arch::x86_64::__m128i }
diff --git a/crates/core_simd/src/type_u8x2.rs b/crates/core_simd/src/type_u8x2.rs
deleted file mode 100644
index c6162cc73a25..000000000000
--- a/crates/core_simd/src/type_u8x2.rs
+++ /dev/null
@@ -1,4 +0,0 @@
-define_type! {
-    #[doc = "Vector of two `u8` types"]
-    struct u8x2([u8; 2]);
-}
diff --git a/crates/core_simd/src/type_u8x32.rs b/crates/core_simd/src/type_u8x32.rs
deleted file mode 100644
index 012286846db9..000000000000
--- a/crates/core_simd/src/type_u8x32.rs
+++ /dev/null
@@ -1,10 +0,0 @@
-define_type! {
-    #[doc = "Vector of 32 `u8` types"]
-    struct u8x32([u8; 32]);
-}
-
-#[cfg(target_arch = "x86")]
-from_aligned! { unsafe u8x32 |bidirectional| core::arch::x86::__m256i }
-
-#[cfg(target_arch = "x86_64")]
-from_aligned! { unsafe u8x32 |bidirectional| core::arch::x86_64::__m256i }
diff --git a/crates/core_simd/src/type_u8x4.rs b/crates/core_simd/src/type_u8x4.rs
deleted file mode 100644
index ecdffa119931..000000000000
--- a/crates/core_simd/src/type_u8x4.rs
+++ /dev/null
@@ -1,4 +0,0 @@
-define_type! {
-    #[doc = "Vector of four `u8` types"]
-    struct u8x4([u8; 4]);
-}
diff --git a/crates/core_simd/src/type_u8x64.rs b/crates/core_simd/src/type_u8x64.rs
deleted file mode 100644
index 6f05f086a64a..000000000000
--- a/crates/core_simd/src/type_u8x64.rs
+++ /dev/null
@@ -1,12 +0,0 @@
-define_type! {
-    #[doc = "Vector of 64 `u8` types"]
-    struct u8x64([u8; 64]);
-}
-
-/*
-#[cfg(target_arch = "x86")]
-from_aligned! { unsafe u8x64 |bidirectional| core::arch::x86::__m512i }
-
-#[cfg(target_arch = "x86_64")]
-from_aligned! { unsafe u8x64 |bidirectional| core::arch::x86_64::__m512i }
-*/
diff --git a/crates/core_simd/src/type_u8x8.rs b/crates/core_simd/src/type_u8x8.rs
deleted file mode 100644
index 7810ac743f8d..000000000000
--- a/crates/core_simd/src/type_u8x8.rs
+++ /dev/null
@@ -1,4 +0,0 @@
-define_type! {
-    #[doc = "Vector of eight `u8` types"]
-    struct u8x8([u8; 8]);
-}
diff --git a/crates/core_simd/src/type_usizex2.rs b/crates/core_simd/src/type_usizex2.rs
deleted file mode 100644
index 218d5a1056eb..000000000000
--- a/crates/core_simd/src/type_usizex2.rs
+++ /dev/null
@@ -1,10 +0,0 @@
-define_type! {
-    #[doc = "Vector of two `usize` types"]
-    struct usizex2([usize; 2]);
-}
-
-#[cfg(all(target_arch = "x86", target_pointer_width = "64"))]
-from_aligned! { unsafe usizex2 |bidirectional| core::arch::x86::__m128i }
-
-#[cfg(all(target_arch = "x86_64", target_pointer_width = "64"))]
-from_aligned! { unsafe usizex2 |bidirectional| core::arch::x86_64::__m128i }
diff --git a/crates/core_simd/src/type_usizex4.rs b/crates/core_simd/src/type_usizex4.rs
deleted file mode 100644
index 60160ecc217b..000000000000
--- a/crates/core_simd/src/type_usizex4.rs
+++ /dev/null
@@ -1,16 +0,0 @@
-define_type! {
-    #[doc = "Vector of four `usize` types"]
-    struct usizex4([usize; 4]);
-}
-
-#[cfg(all(target_arch = "x86", target_pointer_width = "32"))]
-from_aligned! { unsafe usizex4 |bidirectional| core::arch::x86::__m128i }
-
-#[cfg(all(target_arch = "x86_64", target_pointer_width = "32"))]
-from_aligned! { unsafe usizex4 |bidirectional| core::arch::x86_64::__m128i }
-
-#[cfg(all(target_arch = "x86", target_pointer_width = "64"))]
-from_aligned! { unsafe usizex4 |bidirectional| core::arch::x86::__m256i }
-
-#[cfg(all(target_arch = "x86_64", target_pointer_width = "64"))]
-from_aligned! { unsafe usizex4 |bidirectional| core::arch::x86_64::__m256i }
diff --git a/crates/core_simd/src/type_usizex8.rs b/crates/core_simd/src/type_usizex8.rs
deleted file mode 100644
index 5f5a445f4308..000000000000
--- a/crates/core_simd/src/type_usizex8.rs
+++ /dev/null
@@ -1,18 +0,0 @@
-define_type! {
-    #[doc = "Vector of eight `usize` types"]
-    struct usizex8([usize; 8]);
-}
-
-#[cfg(all(target_arch = "x86", target_pointer_width = "32"))]
-from_aligned! { unsafe usizex8 |bidirectional| core::arch::x86::__m256i }
-
-#[cfg(all(target_arch = "x86_64", target_pointer_width = "32"))]
-from_aligned! { unsafe usizex8 |bidirectional| core::arch::x86_64::__m256i }
-
-/*
-#[cfg(all(target_arch = "x86", target_pointer_width = "64"))]
-from_aligned! { unsafe usizex8 |bidirectional| core::arch::x86::__m512i }
-
-#[cfg(all(target_arch = "x86_64", target_pointer_width = "64"))]
-from_aligned! { unsafe usizex8 |bidirectional| core::arch::x86_64::__m512i }
-*/
diff --git a/crates/core_simd/src/vectors_f32.rs b/crates/core_simd/src/vectors_f32.rs
new file mode 100644
index 000000000000..fe24ec6b727e
--- /dev/null
+++ b/crates/core_simd/src/vectors_f32.rs
@@ -0,0 +1,39 @@
+define_type! {
+    #[doc = "Vector of two `f32` values"]
+    struct f32x2([f32; 2]);
+}
+
+define_type! {
+    #[doc = "Vector of four `f32` values"]
+    struct f32x4([f32; 4]);
+}
+
+define_type! {
+    #[doc = "Vector of eight `f32` values"]
+    struct f32x8([f32; 8]);
+}
+
+define_type! {
+    #[doc = "Vector of 16 `f32` values"]
+    struct f32x16([f32; 16]);
+}
+
+#[cfg(target_arch = "x86")]
+from_aligned! { unsafe f32x4 |bidirectional| core::arch::x86::__m128 }
+
+#[cfg(target_arch = "x86_64")]
+from_aligned! { unsafe f32x4 |bidirectional| core::arch::x86_64::__m128 }
+
+#[cfg(target_arch = "x86")]
+from_aligned! { unsafe f32x8 |bidirectional| core::arch::x86::__m256 }
+
+#[cfg(target_arch = "x86_64")]
+from_aligned! { unsafe f32x8 |bidirectional| core::arch::x86_64::__m256 }
+
+/*
+#[cfg(target_arch = "x86")]
+from_aligned! { unsafe f32x16 |bidirectional| core::arch::x86::__m512 }
+
+#[cfg(target_arch = "x86_64")]
+from_aligned! { unsafe f32x16 |bidirectional| core::arch::x86_64::__m512 }
+*/
diff --git a/crates/core_simd/src/vectors_f64.rs b/crates/core_simd/src/vectors_f64.rs
new file mode 100644
index 000000000000..8934cec0517d
--- /dev/null
+++ b/crates/core_simd/src/vectors_f64.rs
@@ -0,0 +1,34 @@
+define_type! {
+    #[doc = "Vector of two `f64` values"]
+    struct f64x2([f64; 2]);
+}
+
+define_type! {
+    #[doc = "Vector of four `f64` values"]
+    struct f64x4([f64; 4]);
+}
+
+define_type! {
+    #[doc = "Vector of eight `f64` values"]
+    struct f64x8([f64; 8]);
+}
+
+#[cfg(target_arch = "x86")]
+from_aligned! { unsafe f64x2 |bidirectional| core::arch::x86::__m128d }
+
+#[cfg(target_arch = "x86_64")]
+from_aligned! { unsafe f64x2 |bidirectional| core::arch::x86_64::__m128d }
+
+#[cfg(target_arch = "x86")]
+from_aligned! { unsafe f64x4 |bidirectional| core::arch::x86::__m256d }
+
+#[cfg(target_arch = "x86_64")]
+from_aligned! { unsafe f64x4 |bidirectional| core::arch::x86_64::__m256d }
+
+/*
+#[cfg(target_arch = "x86")]
+from_aligned! { unsafe f64x8 |bidirectional| core::arch::x86::__m512d }
+
+#[cfg(target_arch = "x86_64")]
+from_aligned! { unsafe f64x8 |bidirectional| core::arch::x86_64::__m512d }
+*/
diff --git a/crates/core_simd/src/vectors_i128.rs b/crates/core_simd/src/vectors_i128.rs
new file mode 100644
index 000000000000..1cdd3c91211e
--- /dev/null
+++ b/crates/core_simd/src/vectors_i128.rs
@@ -0,0 +1,23 @@
+define_type! {
+    #[doc = "Vector of two `i128` values"]
+    struct i128x2([i128; 2]);
+}
+
+define_type! {
+    #[doc = "Vector of four `i128` values"]
+    struct i128x4([i128; 4]);
+}
+
+#[cfg(target_arch = "x86")]
+from_aligned! { unsafe i128x2 |bidirectional| core::arch::x86::__m256i }
+
+#[cfg(target_arch = "x86_64")]
+from_aligned! { unsafe i128x2 |bidirectional| core::arch::x86_64::__m256i }
+
+/*
+#[cfg(target_arch = "x86")]
+from_aligned! { unsafe i128x4 |bidirectional| core::arch::x86::__m512i }
+
+#[cfg(target_arch = "x86_64")]
+from_aligned! { unsafe i128x4 |bidirectional| core::arch::x86_64::__m512i }
+*/
diff --git a/crates/core_simd/src/vectors_i16.rs b/crates/core_simd/src/vectors_i16.rs
new file mode 100644
index 000000000000..724fe7efd1a4
--- /dev/null
+++ b/crates/core_simd/src/vectors_i16.rs
@@ -0,0 +1,44 @@
+define_type! {
+    #[doc = "Vector of two `i16` values"]
+    struct i16x2([i16; 2]);
+}
+
+define_type! {
+    #[doc = "Vector of four `i16` values"]
+    struct i16x4([i16; 4]);
+}
+
+define_type! {
+    #[doc = "Vector of eight `i16` values"]
+    struct i16x8([i16; 8]);
+}
+
+define_type! {
+    #[doc = "Vector of 16 `i16` values"]
+    struct i16x16([i16; 16]);
+}
+
+define_type! {
+    #[doc = "Vector of 32 `i16` values"]
+    struct i16x32([i16; 32]);
+}
+
+#[cfg(target_arch = "x86")]
+from_aligned! { unsafe i16x8 |bidirectional| core::arch::x86::__m128i }
+
+#[cfg(target_arch = "x86_64")]
+from_aligned! { unsafe i16x8 |bidirectional| core::arch::x86_64::__m128i }
+
+#[cfg(target_arch = "x86")]
+from_aligned! { unsafe i16x16 |bidirectional| core::arch::x86::__m256i }
+
+#[cfg(target_arch = "x86_64")]
+from_aligned! { unsafe i16x16 |bidirectional| core::arch::x86_64::__m256i }
+
+/*
+#[cfg(target_arch = "x86")]
+from_aligned! { unsafe u8x32 |bidirectional| core::arch::x86::__m512i }
+
+#[cfg(target_arch = "x86_64")]
+from_aligned! { unsafe u8x32 |bidirectional| core::arch::x86_64::__m512i }
+*/
diff --git a/crates/core_simd/src/vectors_i32.rs b/crates/core_simd/src/vectors_i32.rs
new file mode 100644
index 000000000000..c3af621ae8a4
--- /dev/null
+++ b/crates/core_simd/src/vectors_i32.rs
@@ -0,0 +1,39 @@
+define_type! {
+    #[doc = "Vector of two `i32` values"]
+    struct i32x2([i32; 2]);
+}
+
+define_type! {
+    #[doc = "Vector of four `i32` values"]
+    struct i32x4([i32; 4]);
+}
+
+define_type! {
+    #[doc = "Vector of eight `i32` values"]
+    struct i32x8([i32; 8]);
+}
+
+define_type! {
+    #[doc = "Vector of 16 `i32` values"]
+    struct i32x16([i32; 16]);
+}
+
+#[cfg(target_arch = "x86")]
+from_aligned! { unsafe i32x4 |bidirectional| core::arch::x86::__m128i }
+
+#[cfg(target_arch = "x86_64")]
+from_aligned! { unsafe i32x4 |bidirectional| core::arch::x86_64::__m128i }
+
+#[cfg(target_arch = "x86")]
+from_aligned! { unsafe i32x8 |bidirectional| core::arch::x86::__m256i }
+
+#[cfg(target_arch = "x86_64")]
+from_aligned! { unsafe i32x8 |bidirectional| core::arch::x86_64::__m256i }
+
+/*
+#[cfg(target_arch = "x86")]
+from_aligned! { unsafe u32x16 |bidirectional| core::arch::x86::__m512i }
+
+#[cfg(target_arch = "x86_64")]
+from_aligned! { unsafe u32x16 |bidirectional| core::arch::x86_64::__m512i }
+*/
diff --git a/crates/core_simd/src/vectors_i64.rs b/crates/core_simd/src/vectors_i64.rs
new file mode 100644
index 000000000000..8c6c9d940b15
--- /dev/null
+++ b/crates/core_simd/src/vectors_i64.rs
@@ -0,0 +1,34 @@
+define_type! {
+    #[doc = "Vector of two `i64` values"]
+    struct i64x2([i64; 2]);
+}
+
+define_type! {
+    #[doc = "Vector of four `i64` values"]
+    struct i64x4([i64; 4]);
+}
+
+define_type! {
+    #[doc = "Vector of eight `i64` values"]
+    struct i64x8([i64; 8]);
+}
+
+#[cfg(target_arch = "x86")]
+from_aligned! { unsafe i64x2 |bidirectional| core::arch::x86::__m128i }
+
+#[cfg(target_arch = "x86_64")]
+from_aligned! { unsafe i64x2 |bidirectional| core::arch::x86_64::__m128i }
+
+#[cfg(target_arch = "x86")]
+from_aligned! { unsafe i64x4 |bidirectional| core::arch::x86::__m256i }
+
+#[cfg(target_arch = "x86_64")]
+from_aligned! { unsafe i64x4 |bidirectional| core::arch::x86_64::__m256i }
+
+/*
+#[cfg(target_arch = "x86")]
+from_aligned! { unsafe i64x8 |bidirectional| core::arch::x86::__m512i }
+
+#[cfg(target_arch = "x86_64")]
+from_aligned! { unsafe i64x8 |bidirectional| core::arch::x86_64::__m512i }
+*/
diff --git a/crates/core_simd/src/vectors_i8.rs b/crates/core_simd/src/vectors_i8.rs
new file mode 100644
index 000000000000..340025972d7b
--- /dev/null
+++ b/crates/core_simd/src/vectors_i8.rs
@@ -0,0 +1,49 @@
+define_type! {
+    #[doc = "Vector of two `i8` values"]
+    struct i8x2([i8; 2]);
+}
+
+define_type! {
+    #[doc = "Vector of four `i8` values"]
+    struct i8x4([i8; 4]);
+}
+
+define_type! {
+    #[doc = "Vector of eight `i8` values"]
+    struct i8x8([i8; 8]);
+}
+
+define_type! {
+    #[doc = "Vector of 16 `i8` values"]
+    struct i8x16([i8; 16]);
+}
+
+define_type! {
+    #[doc = "Vector of 32 `i8` values"]
+    struct i8x32([i8; 32]);
+}
+
+define_type! {
+    #[doc = "Vector of 64 `i8` values"]
+    struct i8x64([i8; 64]);
+}
+
+#[cfg(target_arch = "x86")]
+from_aligned! { unsafe i8x16 |bidirectional| core::arch::x86::__m128i }
+
+#[cfg(target_arch = "x86_64")]
+from_aligned! { unsafe i8x16 |bidirectional| core::arch::x86_64::__m128i }
+
+#[cfg(target_arch = "x86")]
+from_aligned! { unsafe i8x32 |bidirectional| core::arch::x86::__m256i }
+
+#[cfg(target_arch = "x86_64")]
+from_aligned! { unsafe i8x32 |bidirectional| core::arch::x86_64::__m256i }
+
+/*
+#[cfg(target_arch = "x86")]
+from_aligned! { unsafe u8x64 |bidirectional| core::arch::x86::__m512i }
+
+#[cfg(target_arch = "x86_64")]
+from_aligned! { unsafe u8x64 |bidirectional| core::arch::x86_64::__m512i }
+*/
diff --git a/crates/core_simd/src/vectors_isize.rs b/crates/core_simd/src/vectors_isize.rs
new file mode 100644
index 000000000000..60f4aad2066a
--- /dev/null
+++ b/crates/core_simd/src/vectors_isize.rs
@@ -0,0 +1,46 @@
+define_type! {
+    #[doc = "Vector of two `isize` values"]
+    struct isizex2([isize; 2]);
+}
+
+define_type! {
+    #[doc = "Vector of four `isize` values"]
+    struct isizex4([isize; 4]);
+}
+
+define_type! {
+    #[doc = "Vector of eight `isize` values"]
+    struct isizex8([isize; 8]);
+}
+
+#[cfg(all(target_arch = "x86", target_pointer_width = "32"))]
+from_aligned! { unsafe isizex4 |bidirectional| core::arch::x86::__m128i }
+
+#[cfg(all(target_arch = "x86_64", target_pointer_width = "32"))]
+from_aligned! { unsafe isizex4 |bidirectional| core::arch::x86_64::__m128i }
+
+#[cfg(all(target_arch = "x86", target_pointer_width = "32"))]
+from_aligned! { unsafe isizex8 |bidirectional| core::arch::x86::__m256i }
+
+#[cfg(all(target_arch = "x86_64", target_pointer_width = "32"))]
+from_aligned! { unsafe isizex8 |bidirectional| core::arch::x86_64::__m256i }
+
+#[cfg(all(target_arch = "x86", target_pointer_width = "64"))]
+from_aligned! { unsafe isizex2 |bidirectional| core::arch::x86::__m128i }
+
+#[cfg(all(target_arch = "x86_64", target_pointer_width = "64"))]
+from_aligned! { unsafe isizex2 |bidirectional| core::arch::x86_64::__m128i }
+
+#[cfg(all(target_arch = "x86", target_pointer_width = "64"))]
+from_aligned! { unsafe isizex4 |bidirectional| core::arch::x86::__m256i }
+
+#[cfg(all(target_arch = "x86_64", target_pointer_width = "64"))]
+from_aligned! { unsafe isizex4 |bidirectional| core::arch::x86_64::__m256i }
+
+/*
+#[cfg(all(target_arch = "x86", target_pointer_width = "64"))]
+from_aligned! { unsafe isizex8 |bidirectional| core::arch::x86::__m512i }
+
+#[cfg(all(target_arch = "x86_64", target_pointer_width = "64"))]
+from_aligned! { unsafe isizex8 |bidirectional| core::arch::x86_64::__m512i }
+*/
diff --git a/crates/core_simd/src/vectors_u128.rs b/crates/core_simd/src/vectors_u128.rs
new file mode 100644
index 000000000000..01a1a309ffa1
--- /dev/null
+++ b/crates/core_simd/src/vectors_u128.rs
@@ -0,0 +1,23 @@
+define_type! {
+    #[doc = "Vector of two `u128` values"]
+    struct u128x2([u128; 2]);
+}
+
+define_type! {
+    #[doc = "Vector of four `u128` values"]
+    struct u128x4([u128; 4]);
+}
+
+#[cfg(target_arch = "x86")]
+from_aligned! { unsafe u128x2 |bidirectional| core::arch::x86::__m256i }
+
+#[cfg(target_arch = "x86_64")]
+from_aligned! { unsafe u128x2 |bidirectional| core::arch::x86_64::__m256i }
+
+/*
+#[cfg(target_arch = "x86")]
+from_aligned! { unsafe u128x4 |bidirectional| core::arch::x86::__m512i }
+
+#[cfg(target_arch = "x86_64")]
+from_aligned! { unsafe u128x4 |bidirectional| core::arch::x86_64::__m512i }
+*/
diff --git a/crates/core_simd/src/vectors_u16.rs b/crates/core_simd/src/vectors_u16.rs
new file mode 100644
index 000000000000..00ae479c6db2
--- /dev/null
+++ b/crates/core_simd/src/vectors_u16.rs
@@ -0,0 +1,44 @@
+define_type! {
+    #[doc = "Vector of two `u16` values"]
+    struct u16x2([u16; 2]);
+}
+
+define_type! {
+    #[doc = "Vector of four `u16` values"]
+    struct u16x4([u16; 4]);
+}
+
+define_type! {
+    #[doc = "Vector of eight `u16` values"]
+    struct u16x8([u16; 8]);
+}
+
+define_type! {
+    #[doc = "Vector of 16 `u16` values"]
+    struct u16x16([u16; 16]);
+}
+
+define_type! {
+    #[doc = "Vector of 32 `u16` values"]
+    struct u16x32([u16; 32]);
+}
+
+#[cfg(target_arch = "x86")]
+from_aligned! { unsafe u16x8 |bidirectional| core::arch::x86::__m128i }
+
+#[cfg(target_arch = "x86_64")]
+from_aligned! { unsafe u16x8 |bidirectional| core::arch::x86_64::__m128i }
+
+#[cfg(target_arch = "x86")]
+from_aligned! { unsafe u16x16 |bidirectional| core::arch::x86::__m256i }
+
+#[cfg(target_arch = "x86_64")]
+from_aligned! { unsafe u16x16 |bidirectional| core::arch::x86_64::__m256i }
+
+/*
+#[cfg(target_arch = "x86")]
+from_aligned! { unsafe u16x32 |bidirectional| core::arch::x86::__m512i }
+
+#[cfg(target_arch = "x86_64")]
+from_aligned! { unsafe u16x32 |bidirectional| core::arch::x86_64::__m512i }
+*/
diff --git a/crates/core_simd/src/vectors_u32.rs b/crates/core_simd/src/vectors_u32.rs
new file mode 100644
index 000000000000..c45db3634c48
--- /dev/null
+++ b/crates/core_simd/src/vectors_u32.rs
@@ -0,0 +1,39 @@
+define_type! {
+    #[doc = "Vector of two `u32` values"]
+    struct u32x2([u32; 2]);
+}
+
+define_type! {
+    #[doc = "Vector of four `u32` values"]
+    struct u32x4([u32; 4]);
+}
+
+define_type! {
+    #[doc = "Vector of eight `u32` values"]
+    struct u32x8([u32; 8]);
+}
+
+define_type! {
+    #[doc = "Vector of 16 `u32` values"]
+    struct u32x16([u32; 16]);
+}
+
+#[cfg(target_arch = "x86")]
+from_aligned! { unsafe u32x4 |bidirectional| core::arch::x86::__m128i }
+
+#[cfg(target_arch = "x86_64")]
+from_aligned! { unsafe u32x4 |bidirectional| core::arch::x86_64::__m128i }
+
+#[cfg(target_arch = "x86")]
+from_aligned! { unsafe u32x8 |bidirectional| core::arch::x86::__m256i }
+
+#[cfg(target_arch = "x86_64")]
+from_aligned! { unsafe u32x8 |bidirectional| core::arch::x86_64::__m256i }
+
+/*
+#[cfg(target_arch = "x86")]
+from_aligned! { unsafe u32x16 |bidirectional| core::arch::x86::__m512i }
+
+#[cfg(target_arch = "x86_64")]
+from_aligned! { unsafe u32x16 |bidirectional| core::arch::x86_64::__m512i }
+*/
diff --git a/crates/core_simd/src/vectors_u64.rs b/crates/core_simd/src/vectors_u64.rs
new file mode 100644
index 000000000000..4c8b95177f55
--- /dev/null
+++ b/crates/core_simd/src/vectors_u64.rs
@@ -0,0 +1,34 @@
+define_type! {
+    #[doc = "Vector of two `u64` values"]
+    struct u64x2([u64; 2]);
+}
+
+define_type! {
+    #[doc = "Vector of four `u64` values"]
+    struct u64x4([u64; 4]);
+}
+
+define_type! {
+    #[doc = "Vector of eight `u64` values"]
+    struct u64x8([u64; 8]);
+}
+
+#[cfg(target_arch = "x86")]
+from_aligned! { unsafe u64x2 |bidirectional| core::arch::x86::__m128i }
+
+#[cfg(target_arch = "x86_64")]
+from_aligned! { unsafe u64x2 |bidirectional| core::arch::x86_64::__m128i }
+
+#[cfg(target_arch = "x86")]
+from_aligned! { unsafe u64x4 |bidirectional| core::arch::x86::__m256i }
+
+#[cfg(target_arch = "x86_64")]
+from_aligned! { unsafe u64x4 |bidirectional| core::arch::x86_64::__m256i }
+
+/*
+#[cfg(target_arch = "x86")]
+from_aligned! { unsafe u64x8 |bidirectional| core::arch::x86::__m512i }
+
+#[cfg(target_arch = "x86_64")]
+from_aligned! { unsafe u64x8 |bidirectional| core::arch::x86_64::__m512i }
+*/
diff --git a/crates/core_simd/src/vectors_u8.rs b/crates/core_simd/src/vectors_u8.rs
new file mode 100644
index 000000000000..d3f0893154ba
--- /dev/null
+++ b/crates/core_simd/src/vectors_u8.rs
@@ -0,0 +1,49 @@
+define_type! {
+    #[doc = "Vector of two `u8` values"]
+    struct u8x2([u8; 2]);
+}
+
+define_type! {
+    #[doc = "Vector of four `u8` values"]
+    struct u8x4([u8; 4]);
+}
+
+define_type! {
+    #[doc = "Vector of eight `u8` values"]
+    struct u8x8([u8; 8]);
+}
+
+define_type! {
+    #[doc = "Vector of 16 `u8` values"]
+    struct u8x16([u8; 16]);
+}
+
+define_type! {
+    #[doc = "Vector of 32 `u8` values"]
+    struct u8x32([u8; 32]);
+}
+
+define_type! {
+    #[doc = "Vector of 64 `u8` values"]
+    struct u8x64([u8; 64]);
+}
+
+#[cfg(target_arch = "x86")]
+from_aligned! { unsafe u8x16 |bidirectional| core::arch::x86::__m128i }
+
+#[cfg(target_arch = "x86_64")]
+from_aligned! { unsafe u8x16 |bidirectional| core::arch::x86_64::__m128i }
+
+#[cfg(target_arch = "x86")]
+from_aligned! { unsafe u8x32 |bidirectional| core::arch::x86::__m256i }
+
+#[cfg(target_arch = "x86_64")]
+from_aligned! { unsafe u8x32 |bidirectional| core::arch::x86_64::__m256i }
+
+/*
+#[cfg(target_arch = "x86")]
+from_aligned! { unsafe u8x64 |bidirectional| core::arch::x86::__m512i }
+
+#[cfg(target_arch = "x86_64")]
+from_aligned! { unsafe u8x64 |bidirectional| core::arch::x86_64::__m512i }
+*/
diff --git a/crates/core_simd/src/vectors_usize.rs b/crates/core_simd/src/vectors_usize.rs
new file mode 100644
index 000000000000..c997e7cd4e3d
--- /dev/null
+++ b/crates/core_simd/src/vectors_usize.rs
@@ -0,0 +1,46 @@
+define_type! {
+    #[doc = "Vector of two `usize` values"]
+    struct usizex2([usize; 2]);
+}
+
+define_type! {
+    #[doc = "Vector of four `usize` values"]
+    struct usizex4([usize; 4]);
+}
+
+define_type! {
+    #[doc = "Vector of eight `usize` values"]
+    struct usizex8([usize; 8]);
+}
+
+#[cfg(all(target_arch = "x86", target_pointer_width = "32"))]
+from_aligned! { unsafe usizex4 |bidirectional| core::arch::x86::__m128i }
+
+#[cfg(all(target_arch = "x86_64", target_pointer_width = "32"))]
+from_aligned! { unsafe usizex4 |bidirectional| core::arch::x86_64::__m128i }
+
+#[cfg(all(target_arch = "x86", target_pointer_width = "32"))]
+from_aligned! { unsafe usizex8 |bidirectional| core::arch::x86::__m256i }
+
+#[cfg(all(target_arch = "x86_64", target_pointer_width = "32"))]
+from_aligned! { unsafe usizex8 |bidirectional| core::arch::x86_64::__m256i }
+
+#[cfg(all(target_arch = "x86", target_pointer_width = "64"))]
+from_aligned! { unsafe usizex2 |bidirectional| core::arch::x86::__m128i }
+
+#[cfg(all(target_arch = "x86_64", target_pointer_width = "64"))]
+from_aligned! { unsafe usizex2 |bidirectional| core::arch::x86_64::__m128i }
+
+#[cfg(all(target_arch = "x86", target_pointer_width = "64"))]
+from_aligned! { unsafe usizex4 |bidirectional| core::arch::x86::__m256i }
+
+#[cfg(all(target_arch = "x86_64", target_pointer_width = "64"))]
+from_aligned! { unsafe usizex4 |bidirectional| core::arch::x86_64::__m256i }
+
+/*
+#[cfg(all(target_arch = "x86", target_pointer_width = "64"))]
+from_aligned! { unsafe usizex8 |bidirectional| core::arch::x86::__m512i }
+
+#[cfg(all(target_arch = "x86_64", target_pointer_width = "64"))]
+from_aligned! { unsafe usizex8 |bidirectional| core::arch::x86_64::__m512i }
+*/

From 872e8f62d6e776436cb830f3178beb4a853e7df8 Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Wed, 23 Sep 2020 10:42:12 -0400
Subject: [PATCH 009/249] Simplify transmutes

---
 crates/core_simd/src/macros.rs        | 37 ++++++++++---------------
 crates/core_simd/src/vectors_f32.rs   | 22 ++-------------
 crates/core_simd/src/vectors_f64.rs   | 22 ++-------------
 crates/core_simd/src/vectors_i128.rs  | 15 ++--------
 crates/core_simd/src/vectors_i16.rs   | 22 ++-------------
 crates/core_simd/src/vectors_i32.rs   | 22 ++-------------
 crates/core_simd/src/vectors_i64.rs   | 22 ++-------------
 crates/core_simd/src/vectors_i8.rs    | 22 ++-------------
 crates/core_simd/src/vectors_isize.rs | 40 +++++++--------------------
 crates/core_simd/src/vectors_u128.rs  | 15 ++--------
 crates/core_simd/src/vectors_u16.rs   | 22 ++-------------
 crates/core_simd/src/vectors_u32.rs   | 22 ++-------------
 crates/core_simd/src/vectors_u64.rs   | 22 ++-------------
 crates/core_simd/src/vectors_u8.rs    | 22 ++-------------
 crates/core_simd/src/vectors_usize.rs | 40 +++++++--------------------
 15 files changed, 69 insertions(+), 298 deletions(-)

diff --git a/crates/core_simd/src/macros.rs b/crates/core_simd/src/macros.rs
index 91c1071a2ee5..5a50b1a48ec1 100644
--- a/crates/core_simd/src/macros.rs
+++ b/crates/core_simd/src/macros.rs
@@ -1,29 +1,25 @@
-macro_rules! from_aligned {
-    { unsafe $from:ty => $to:ty } => {
+macro_rules! from_transmute {
+    { unsafe $a:ty => $b:ty } => {
+        from_transmute!{ @impl $a => $b }
+        from_transmute!{ @impl $b => $a }
+    };
+    { @impl $from:ty => $to:ty } => {
         impl core::convert::From<$from> for $to {
             #[inline]
             fn from(value: $from) -> $to {
-                assert_eq!(core::mem::size_of::<$from>(), core::mem::size_of::<$to>());
-                assert!(core::mem::align_of::<$from>() >= core::mem::align_of::<$to>());
                 unsafe { core::mem::transmute(value) }
             }
         }
     };
-    { unsafe $a:ty |bidirectional| $b:ty } => {
-        from_aligned!{ unsafe $a => $b }
-        from_aligned!{ unsafe $b => $a }
-    }
 }
 
-macro_rules! from_unaligned {
-    { unsafe $from:ty => $to:ty } => {
-        impl core::convert::From<$from> for $to {
-            #[inline]
-            fn from(value: $from) -> $to {
-                assert_eq!(core::mem::size_of::<$from>(), core::mem::size_of::<$to>());
-                unsafe { (&value as *const $from as *const $to).read_unaligned() }
-            }
-        }
+macro_rules! from_transmute_x86 {
+    { unsafe $generic:ty => $intel:ident } => {
+        #[cfg(target_arch = "x86")]
+        from_transmute! { unsafe $generic => core::arch::x86::$intel }
+
+        #[cfg(target_arch = "x86_64")]
+        from_transmute! { unsafe $generic => core::arch::x86_64::$intel }
     }
 }
 
@@ -61,11 +57,8 @@ macro_rules! define_type {
             }
         }
 
-        // vector to array
-        from_aligned! { unsafe $name => [$type; $lanes] }
-
-        // array to vector
-        from_unaligned! { unsafe [$type; $lanes] => $name }
+        // vector/array conversion
+        from_transmute! { unsafe $name => [$type; $lanes] }
 
         // splat
         impl From<$type> for $name {
diff --git a/crates/core_simd/src/vectors_f32.rs b/crates/core_simd/src/vectors_f32.rs
index fe24ec6b727e..3b38b26d6310 100644
--- a/crates/core_simd/src/vectors_f32.rs
+++ b/crates/core_simd/src/vectors_f32.rs
@@ -18,22 +18,6 @@ define_type! {
     struct f32x16([f32; 16]);
 }
 
-#[cfg(target_arch = "x86")]
-from_aligned! { unsafe f32x4 |bidirectional| core::arch::x86::__m128 }
-
-#[cfg(target_arch = "x86_64")]
-from_aligned! { unsafe f32x4 |bidirectional| core::arch::x86_64::__m128 }
-
-#[cfg(target_arch = "x86")]
-from_aligned! { unsafe f32x8 |bidirectional| core::arch::x86::__m256 }
-
-#[cfg(target_arch = "x86_64")]
-from_aligned! { unsafe f32x8 |bidirectional| core::arch::x86_64::__m256 }
-
-/*
-#[cfg(target_arch = "x86")]
-from_aligned! { unsafe f32x16 |bidirectional| core::arch::x86::__m512 }
-
-#[cfg(target_arch = "x86_64")]
-from_aligned! { unsafe f32x16 |bidirectional| core::arch::x86_64::__m512 }
-*/
+from_transmute_x86! { unsafe f32x4 => __m128 }
+from_transmute_x86! { unsafe f32x8 => __m256 }
+//from_transmute_x86! { unsafe f32x16 => __m512 }
diff --git a/crates/core_simd/src/vectors_f64.rs b/crates/core_simd/src/vectors_f64.rs
index 8934cec0517d..f627fffd46b4 100644
--- a/crates/core_simd/src/vectors_f64.rs
+++ b/crates/core_simd/src/vectors_f64.rs
@@ -13,22 +13,6 @@ define_type! {
     struct f64x8([f64; 8]);
 }
 
-#[cfg(target_arch = "x86")]
-from_aligned! { unsafe f64x2 |bidirectional| core::arch::x86::__m128d }
-
-#[cfg(target_arch = "x86_64")]
-from_aligned! { unsafe f64x2 |bidirectional| core::arch::x86_64::__m128d }
-
-#[cfg(target_arch = "x86")]
-from_aligned! { unsafe f64x4 |bidirectional| core::arch::x86::__m256d }
-
-#[cfg(target_arch = "x86_64")]
-from_aligned! { unsafe f64x4 |bidirectional| core::arch::x86_64::__m256d }
-
-/*
-#[cfg(target_arch = "x86")]
-from_aligned! { unsafe f64x8 |bidirectional| core::arch::x86::__m512d }
-
-#[cfg(target_arch = "x86_64")]
-from_aligned! { unsafe f64x8 |bidirectional| core::arch::x86_64::__m512d }
-*/
+from_transmute_x86! { unsafe f64x2 => __m128d }
+from_transmute_x86! { unsafe f64x4 => __m256d }
+//from_transmute_x86! { unsafe f64x8 => __m512d }
diff --git a/crates/core_simd/src/vectors_i128.rs b/crates/core_simd/src/vectors_i128.rs
index 1cdd3c91211e..b4f30f845fd1 100644
--- a/crates/core_simd/src/vectors_i128.rs
+++ b/crates/core_simd/src/vectors_i128.rs
@@ -8,16 +8,5 @@ define_type! {
     struct i128x4([i128; 4]);
 }
 
-#[cfg(target_arch = "x86")]
-from_aligned! { unsafe i128x2 |bidirectional| core::arch::x86::__m256i }
-
-#[cfg(target_arch = "x86_64")]
-from_aligned! { unsafe i128x2 |bidirectional| core::arch::x86_64::__m256i }
-
-/*
-#[cfg(target_arch = "x86")]
-from_aligned! { unsafe i128x4 |bidirectional| core::arch::x86::__m512i }
-
-#[cfg(target_arch = "x86_64")]
-from_aligned! { unsafe i128x4 |bidirectional| core::arch::x86_64::__m512i }
-*/
+from_transmute_x86! { unsafe i128x2 => __m256i }
+//from_transmute_x86! { unsafe i128x4 => __m512i }
diff --git a/crates/core_simd/src/vectors_i16.rs b/crates/core_simd/src/vectors_i16.rs
index 724fe7efd1a4..a87fa8c41cd5 100644
--- a/crates/core_simd/src/vectors_i16.rs
+++ b/crates/core_simd/src/vectors_i16.rs
@@ -23,22 +23,6 @@ define_type! {
     struct i16x32([i16; 32]);
 }
 
-#[cfg(target_arch = "x86")]
-from_aligned! { unsafe i16x8 |bidirectional| core::arch::x86::__m128i }
-
-#[cfg(target_arch = "x86_64")]
-from_aligned! { unsafe i16x8 |bidirectional| core::arch::x86_64::__m128i }
-
-#[cfg(target_arch = "x86")]
-from_aligned! { unsafe i16x16 |bidirectional| core::arch::x86::__m256i }
-
-#[cfg(target_arch = "x86_64")]
-from_aligned! { unsafe i16x16 |bidirectional| core::arch::x86_64::__m256i }
-
-/*
-#[cfg(target_arch = "x86")]
-from_aligned! { unsafe u8x32 |bidirectional| core::arch::x86::__m512i }
-
-#[cfg(target_arch = "x86_64")]
-from_aligned! { unsafe u8x32 |bidirectional| core::arch::x86_64::__m512i }
-*/
+from_transmute_x86! { unsafe i16x8 => __m128i }
+from_transmute_x86! { unsafe i16x16 => __m256i }
+//from_transmute_x86! { unsafe i16x32 => __m512i }
diff --git a/crates/core_simd/src/vectors_i32.rs b/crates/core_simd/src/vectors_i32.rs
index c3af621ae8a4..13c9299b7c6e 100644
--- a/crates/core_simd/src/vectors_i32.rs
+++ b/crates/core_simd/src/vectors_i32.rs
@@ -18,22 +18,6 @@ define_type! {
     struct i32x16([i32; 16]);
 }
 
-#[cfg(target_arch = "x86")]
-from_aligned! { unsafe i32x4 |bidirectional| core::arch::x86::__m128i }
-
-#[cfg(target_arch = "x86_64")]
-from_aligned! { unsafe i32x4 |bidirectional| core::arch::x86_64::__m128i }
-
-#[cfg(target_arch = "x86")]
-from_aligned! { unsafe i32x8 |bidirectional| core::arch::x86::__m256i }
-
-#[cfg(target_arch = "x86_64")]
-from_aligned! { unsafe i32x8 |bidirectional| core::arch::x86_64::__m256i }
-
-/*
-#[cfg(target_arch = "x86")]
-from_aligned! { unsafe u32x16 |bidirectional| core::arch::x86::__m512i }
-
-#[cfg(target_arch = "x86_64")]
-from_aligned! { unsafe u32x16 |bidirectional| core::arch::x86_64::__m512i }
-*/
+from_transmute_x86! { unsafe i32x4 => __m128i }
+from_transmute_x86! { unsafe i32x8 => __m256i }
+//from_transmute_x86! { unsafe i32x16 => __m512i }
diff --git a/crates/core_simd/src/vectors_i64.rs b/crates/core_simd/src/vectors_i64.rs
index 8c6c9d940b15..8d86501a4961 100644
--- a/crates/core_simd/src/vectors_i64.rs
+++ b/crates/core_simd/src/vectors_i64.rs
@@ -13,22 +13,6 @@ define_type! {
     struct i64x8([i64; 8]);
 }
 
-#[cfg(target_arch = "x86")]
-from_aligned! { unsafe i64x2 |bidirectional| core::arch::x86::__m128i }
-
-#[cfg(target_arch = "x86_64")]
-from_aligned! { unsafe i64x2 |bidirectional| core::arch::x86_64::__m128i }
-
-#[cfg(target_arch = "x86")]
-from_aligned! { unsafe i64x4 |bidirectional| core::arch::x86::__m256i }
-
-#[cfg(target_arch = "x86_64")]
-from_aligned! { unsafe i64x4 |bidirectional| core::arch::x86_64::__m256i }
-
-/*
-#[cfg(target_arch = "x86")]
-from_aligned! { unsafe i64x8 |bidirectional| core::arch::x86::__m512i }
-
-#[cfg(target_arch = "x86_64")]
-from_aligned! { unsafe i64x8 |bidirectional| core::arch::x86_64::__m512i }
-*/
+from_transmute_x86! { unsafe i64x2 => __m128i }
+from_transmute_x86! { unsafe i64x4 => __m256i }
+//from_transmute_x86! { unsafe i64x8 => __m512i }
diff --git a/crates/core_simd/src/vectors_i8.rs b/crates/core_simd/src/vectors_i8.rs
index 340025972d7b..f0d5ad55212d 100644
--- a/crates/core_simd/src/vectors_i8.rs
+++ b/crates/core_simd/src/vectors_i8.rs
@@ -28,22 +28,6 @@ define_type! {
     struct i8x64([i8; 64]);
 }
 
-#[cfg(target_arch = "x86")]
-from_aligned! { unsafe i8x16 |bidirectional| core::arch::x86::__m128i }
-
-#[cfg(target_arch = "x86_64")]
-from_aligned! { unsafe i8x16 |bidirectional| core::arch::x86_64::__m128i }
-
-#[cfg(target_arch = "x86")]
-from_aligned! { unsafe i8x32 |bidirectional| core::arch::x86::__m256i }
-
-#[cfg(target_arch = "x86_64")]
-from_aligned! { unsafe i8x32 |bidirectional| core::arch::x86_64::__m256i }
-
-/*
-#[cfg(target_arch = "x86")]
-from_aligned! { unsafe u8x64 |bidirectional| core::arch::x86::__m512i }
-
-#[cfg(target_arch = "x86_64")]
-from_aligned! { unsafe u8x64 |bidirectional| core::arch::x86_64::__m512i }
-*/
+from_transmute_x86! { unsafe i8x16 => __m128i }
+from_transmute_x86! { unsafe i8x32 => __m256i }
+//from_transmute_x86! { unsafe i8x64 => __m512i }
diff --git a/crates/core_simd/src/vectors_isize.rs b/crates/core_simd/src/vectors_isize.rs
index 60f4aad2066a..2028d1ecc710 100644
--- a/crates/core_simd/src/vectors_isize.rs
+++ b/crates/core_simd/src/vectors_isize.rs
@@ -13,34 +13,14 @@ define_type! {
     struct isizex8([isize; 8]);
 }
 
-#[cfg(all(target_arch = "x86", target_pointer_width = "32"))]
-from_aligned! { unsafe isizex4 |bidirectional| core::arch::x86::__m128i }
+#[cfg(target_pointer_width = "32")]
+from_transmute_x86! { unsafe isizex4 => __m128i }
+#[cfg(target_pointer_width = "32")]
+from_transmute_x86! { unsafe isizex8 => __m256i }
 
-#[cfg(all(target_arch = "x86_64", target_pointer_width = "32"))]
-from_aligned! { unsafe isizex4 |bidirectional| core::arch::x86_64::__m128i }
-
-#[cfg(all(target_arch = "x86", target_pointer_width = "32"))]
-from_aligned! { unsafe isizex8 |bidirectional| core::arch::x86::__m256i }
-
-#[cfg(all(target_arch = "x86_64", target_pointer_width = "32"))]
-from_aligned! { unsafe isizex8 |bidirectional| core::arch::x86_64::__m256i }
-
-#[cfg(all(target_arch = "x86", target_pointer_width = "64"))]
-from_aligned! { unsafe isizex2 |bidirectional| core::arch::x86::__m128i }
-
-#[cfg(all(target_arch = "x86_64", target_pointer_width = "64"))]
-from_aligned! { unsafe isizex2 |bidirectional| core::arch::x86_64::__m128i }
-
-#[cfg(all(target_arch = "x86", target_pointer_width = "64"))]
-from_aligned! { unsafe isizex4 |bidirectional| core::arch::x86::__m256i }
-
-#[cfg(all(target_arch = "x86_64", target_pointer_width = "64"))]
-from_aligned! { unsafe isizex4 |bidirectional| core::arch::x86_64::__m256i }
-
-/*
-#[cfg(all(target_arch = "x86", target_pointer_width = "64"))]
-from_aligned! { unsafe isizex8 |bidirectional| core::arch::x86::__m512i }
-
-#[cfg(all(target_arch = "x86_64", target_pointer_width = "64"))]
-from_aligned! { unsafe isizex8 |bidirectional| core::arch::x86_64::__m512i }
-*/
+#[cfg(target_pointer_width = "64")]
+from_transmute_x86! { unsafe isizex2 => __m128i }
+#[cfg(target_pointer_width = "64")]
+from_transmute_x86! { unsafe isizex4 => __m256i }
+//#[cfg(target_pointer_width = "64")]
+//from_transmute_x86! { unsafe isizex8 => __m512i }
diff --git a/crates/core_simd/src/vectors_u128.rs b/crates/core_simd/src/vectors_u128.rs
index 01a1a309ffa1..4b47fb2fa2ec 100644
--- a/crates/core_simd/src/vectors_u128.rs
+++ b/crates/core_simd/src/vectors_u128.rs
@@ -8,16 +8,5 @@ define_type! {
     struct u128x4([u128; 4]);
 }
 
-#[cfg(target_arch = "x86")]
-from_aligned! { unsafe u128x2 |bidirectional| core::arch::x86::__m256i }
-
-#[cfg(target_arch = "x86_64")]
-from_aligned! { unsafe u128x2 |bidirectional| core::arch::x86_64::__m256i }
-
-/*
-#[cfg(target_arch = "x86")]
-from_aligned! { unsafe u128x4 |bidirectional| core::arch::x86::__m512i }
-
-#[cfg(target_arch = "x86_64")]
-from_aligned! { unsafe u128x4 |bidirectional| core::arch::x86_64::__m512i }
-*/
+from_transmute_x86! { unsafe u128x2 => __m256i }
+//from_transmute_x86! { unsafe u128x4 => __m512i }
diff --git a/crates/core_simd/src/vectors_u16.rs b/crates/core_simd/src/vectors_u16.rs
index 00ae479c6db2..f637d2fe9f2c 100644
--- a/crates/core_simd/src/vectors_u16.rs
+++ b/crates/core_simd/src/vectors_u16.rs
@@ -23,22 +23,6 @@ define_type! {
     struct u16x32([u16; 32]);
 }
 
-#[cfg(target_arch = "x86")]
-from_aligned! { unsafe u16x8 |bidirectional| core::arch::x86::__m128i }
-
-#[cfg(target_arch = "x86_64")]
-from_aligned! { unsafe u16x8 |bidirectional| core::arch::x86_64::__m128i }
-
-#[cfg(target_arch = "x86")]
-from_aligned! { unsafe u16x16 |bidirectional| core::arch::x86::__m256i }
-
-#[cfg(target_arch = "x86_64")]
-from_aligned! { unsafe u16x16 |bidirectional| core::arch::x86_64::__m256i }
-
-/*
-#[cfg(target_arch = "x86")]
-from_aligned! { unsafe u16x32 |bidirectional| core::arch::x86::__m512i }
-
-#[cfg(target_arch = "x86_64")]
-from_aligned! { unsafe u16x32 |bidirectional| core::arch::x86_64::__m512i }
-*/
+from_transmute_x86! { unsafe u16x8 => __m128i }
+from_transmute_x86! { unsafe u16x16 => __m256i }
+//from_transmute_x86! { unsafe u16x32 => __m512i }
diff --git a/crates/core_simd/src/vectors_u32.rs b/crates/core_simd/src/vectors_u32.rs
index c45db3634c48..d2f5d7d241c5 100644
--- a/crates/core_simd/src/vectors_u32.rs
+++ b/crates/core_simd/src/vectors_u32.rs
@@ -18,22 +18,6 @@ define_type! {
     struct u32x16([u32; 16]);
 }
 
-#[cfg(target_arch = "x86")]
-from_aligned! { unsafe u32x4 |bidirectional| core::arch::x86::__m128i }
-
-#[cfg(target_arch = "x86_64")]
-from_aligned! { unsafe u32x4 |bidirectional| core::arch::x86_64::__m128i }
-
-#[cfg(target_arch = "x86")]
-from_aligned! { unsafe u32x8 |bidirectional| core::arch::x86::__m256i }
-
-#[cfg(target_arch = "x86_64")]
-from_aligned! { unsafe u32x8 |bidirectional| core::arch::x86_64::__m256i }
-
-/*
-#[cfg(target_arch = "x86")]
-from_aligned! { unsafe u32x16 |bidirectional| core::arch::x86::__m512i }
-
-#[cfg(target_arch = "x86_64")]
-from_aligned! { unsafe u32x16 |bidirectional| core::arch::x86_64::__m512i }
-*/
+from_transmute_x86! { unsafe u32x4 => __m128i }
+from_transmute_x86! { unsafe u32x8 => __m256i }
+//from_transmute_x86! { unsafe u32x16 => __m512i }
diff --git a/crates/core_simd/src/vectors_u64.rs b/crates/core_simd/src/vectors_u64.rs
index 4c8b95177f55..e6e21e62da10 100644
--- a/crates/core_simd/src/vectors_u64.rs
+++ b/crates/core_simd/src/vectors_u64.rs
@@ -13,22 +13,6 @@ define_type! {
     struct u64x8([u64; 8]);
 }
 
-#[cfg(target_arch = "x86")]
-from_aligned! { unsafe u64x2 |bidirectional| core::arch::x86::__m128i }
-
-#[cfg(target_arch = "x86_64")]
-from_aligned! { unsafe u64x2 |bidirectional| core::arch::x86_64::__m128i }
-
-#[cfg(target_arch = "x86")]
-from_aligned! { unsafe u64x4 |bidirectional| core::arch::x86::__m256i }
-
-#[cfg(target_arch = "x86_64")]
-from_aligned! { unsafe u64x4 |bidirectional| core::arch::x86_64::__m256i }
-
-/*
-#[cfg(target_arch = "x86")]
-from_aligned! { unsafe u64x8 |bidirectional| core::arch::x86::__m512i }
-
-#[cfg(target_arch = "x86_64")]
-from_aligned! { unsafe u64x8 |bidirectional| core::arch::x86_64::__m512i }
-*/
+from_transmute_x86! { unsafe u64x2 => __m128i }
+from_transmute_x86! { unsafe u64x4 => __m256i }
+//from_transmute_x86! { unsafe u64x8 => __m512i }
diff --git a/crates/core_simd/src/vectors_u8.rs b/crates/core_simd/src/vectors_u8.rs
index d3f0893154ba..f6246f87ecec 100644
--- a/crates/core_simd/src/vectors_u8.rs
+++ b/crates/core_simd/src/vectors_u8.rs
@@ -28,22 +28,6 @@ define_type! {
     struct u8x64([u8; 64]);
 }
 
-#[cfg(target_arch = "x86")]
-from_aligned! { unsafe u8x16 |bidirectional| core::arch::x86::__m128i }
-
-#[cfg(target_arch = "x86_64")]
-from_aligned! { unsafe u8x16 |bidirectional| core::arch::x86_64::__m128i }
-
-#[cfg(target_arch = "x86")]
-from_aligned! { unsafe u8x32 |bidirectional| core::arch::x86::__m256i }
-
-#[cfg(target_arch = "x86_64")]
-from_aligned! { unsafe u8x32 |bidirectional| core::arch::x86_64::__m256i }
-
-/*
-#[cfg(target_arch = "x86")]
-from_aligned! { unsafe u8x64 |bidirectional| core::arch::x86::__m512i }
-
-#[cfg(target_arch = "x86_64")]
-from_aligned! { unsafe u8x64 |bidirectional| core::arch::x86_64::__m512i }
-*/
+from_transmute_x86! { unsafe u8x16 => __m128i }
+from_transmute_x86! { unsafe u8x32 => __m256i }
+//from_transmute_x86! { unsafe u8x64 => __m512i }
diff --git a/crates/core_simd/src/vectors_usize.rs b/crates/core_simd/src/vectors_usize.rs
index c997e7cd4e3d..45da00562f60 100644
--- a/crates/core_simd/src/vectors_usize.rs
+++ b/crates/core_simd/src/vectors_usize.rs
@@ -13,34 +13,14 @@ define_type! {
     struct usizex8([usize; 8]);
 }
 
-#[cfg(all(target_arch = "x86", target_pointer_width = "32"))]
-from_aligned! { unsafe usizex4 |bidirectional| core::arch::x86::__m128i }
+#[cfg(target_pointer_width = "32")]
+from_transmute_x86! { unsafe usizex4 => __m128i }
+#[cfg(target_pointer_width = "32")]
+from_transmute_x86! { unsafe usizex8 => __m256i }
 
-#[cfg(all(target_arch = "x86_64", target_pointer_width = "32"))]
-from_aligned! { unsafe usizex4 |bidirectional| core::arch::x86_64::__m128i }
-
-#[cfg(all(target_arch = "x86", target_pointer_width = "32"))]
-from_aligned! { unsafe usizex8 |bidirectional| core::arch::x86::__m256i }
-
-#[cfg(all(target_arch = "x86_64", target_pointer_width = "32"))]
-from_aligned! { unsafe usizex8 |bidirectional| core::arch::x86_64::__m256i }
-
-#[cfg(all(target_arch = "x86", target_pointer_width = "64"))]
-from_aligned! { unsafe usizex2 |bidirectional| core::arch::x86::__m128i }
-
-#[cfg(all(target_arch = "x86_64", target_pointer_width = "64"))]
-from_aligned! { unsafe usizex2 |bidirectional| core::arch::x86_64::__m128i }
-
-#[cfg(all(target_arch = "x86", target_pointer_width = "64"))]
-from_aligned! { unsafe usizex4 |bidirectional| core::arch::x86::__m256i }
-
-#[cfg(all(target_arch = "x86_64", target_pointer_width = "64"))]
-from_aligned! { unsafe usizex4 |bidirectional| core::arch::x86_64::__m256i }
-
-/*
-#[cfg(all(target_arch = "x86", target_pointer_width = "64"))]
-from_aligned! { unsafe usizex8 |bidirectional| core::arch::x86::__m512i }
-
-#[cfg(all(target_arch = "x86_64", target_pointer_width = "64"))]
-from_aligned! { unsafe usizex8 |bidirectional| core::arch::x86_64::__m512i }
-*/
+#[cfg(target_pointer_width = "64")]
+from_transmute_x86! { unsafe usizex2 => __m128i }
+#[cfg(target_pointer_width = "64")]
+from_transmute_x86! { unsafe usizex4 => __m256i }
+//#[cfg(target_pointer_width = "64")]
+//from_transmute_x86! { unsafe usizex8 => __m512i }

From 3df72fddc267144a2052167c012b654974d84bb8 Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Thu, 24 Sep 2020 19:48:18 -0400
Subject: [PATCH 010/249] Add masks

---
 crates/core_simd/src/lib.rs              |  19 ++
 crates/core_simd/src/macros.rs           | 240 +++++++++++++++++------
 crates/core_simd/src/masks.rs            |  67 +++++++
 crates/core_simd/src/vectors_f32.rs      |   8 +-
 crates/core_simd/src/vectors_f64.rs      |   6 +-
 crates/core_simd/src/vectors_i128.rs     |   4 +-
 crates/core_simd/src/vectors_i16.rs      |  10 +-
 crates/core_simd/src/vectors_i32.rs      |   8 +-
 crates/core_simd/src/vectors_i64.rs      |   6 +-
 crates/core_simd/src/vectors_i8.rs       |  12 +-
 crates/core_simd/src/vectors_isize.rs    |   6 +-
 crates/core_simd/src/vectors_mask128.rs  |  11 ++
 crates/core_simd/src/vectors_mask16.rs   |  26 +++
 crates/core_simd/src/vectors_mask32.rs   |  21 ++
 crates/core_simd/src/vectors_mask64.rs   |  16 ++
 crates/core_simd/src/vectors_mask8.rs    |  31 +++
 crates/core_simd/src/vectors_masksize.rs |  16 ++
 crates/core_simd/src/vectors_u128.rs     |   4 +-
 crates/core_simd/src/vectors_u16.rs      |  10 +-
 crates/core_simd/src/vectors_u32.rs      |   8 +-
 crates/core_simd/src/vectors_u64.rs      |   6 +-
 crates/core_simd/src/vectors_u8.rs       |  12 +-
 crates/core_simd/src/vectors_usize.rs    |   6 +-
 23 files changed, 439 insertions(+), 114 deletions(-)
 create mode 100644 crates/core_simd/src/masks.rs
 create mode 100644 crates/core_simd/src/vectors_mask128.rs
 create mode 100644 crates/core_simd/src/vectors_mask16.rs
 create mode 100644 crates/core_simd/src/vectors_mask32.rs
 create mode 100644 crates/core_simd/src/vectors_mask64.rs
 create mode 100644 crates/core_simd/src/vectors_mask8.rs
 create mode 100644 crates/core_simd/src/vectors_masksize.rs

diff --git a/crates/core_simd/src/lib.rs b/crates/core_simd/src/lib.rs
index a5dae4b02792..1ee180a5e3a1 100644
--- a/crates/core_simd/src/lib.rs
+++ b/crates/core_simd/src/lib.rs
@@ -1,10 +1,14 @@
 #![no_std]
 #![feature(repr_simd)]
 #![warn(missing_docs)]
+//! Portable SIMD module.
 
 #[macro_use]
 mod macros;
 
+mod masks;
+pub use masks::*;
+
 mod vectors_u8;
 pub use vectors_u8::*;
 mod vectors_u16;
@@ -17,6 +21,7 @@ mod vectors_u128;
 pub use vectors_u128::*;
 mod vectors_usize;
 pub use vectors_usize::*;
+
 mod vectors_i8;
 pub use vectors_i8::*;
 mod vectors_i16;
@@ -29,7 +34,21 @@ mod vectors_i128;
 pub use vectors_i128::*;
 mod vectors_isize;
 pub use vectors_isize::*;
+
 mod vectors_f32;
 pub use vectors_f32::*;
 mod vectors_f64;
 pub use vectors_f64::*;
+
+mod vectors_mask8;
+pub use vectors_mask8::*;
+mod vectors_mask16;
+pub use vectors_mask16::*;
+mod vectors_mask32;
+pub use vectors_mask32::*;
+mod vectors_mask64;
+pub use vectors_mask64::*;
+mod vectors_mask128;
+pub use vectors_mask128::*;
+mod vectors_masksize;
+pub use vectors_masksize::*;
diff --git a/crates/core_simd/src/macros.rs b/crates/core_simd/src/macros.rs
index 5a50b1a48ec1..3b5d70695c61 100644
--- a/crates/core_simd/src/macros.rs
+++ b/crates/core_simd/src/macros.rs
@@ -1,3 +1,4 @@
+/// Provides implementations of `From<$a> for $b` and `From<$b> for $a` that transmutes the value.
 macro_rules! from_transmute {
     { unsafe $a:ty => $b:ty } => {
         from_transmute!{ @impl $a => $b }
@@ -13,6 +14,8 @@ macro_rules! from_transmute {
     };
 }
 
+/// Provides implementations of `From<$generic> for core::arch::{x86, x86_64}::$intel` and
+/// vice-versa that transmutes the value.
 macro_rules! from_transmute_x86 {
     { unsafe $generic:ty => $intel:ident } => {
         #[cfg(target_arch = "x86")]
@@ -23,10 +26,118 @@ macro_rules! from_transmute_x86 {
     }
 }
 
-macro_rules! define_type {
-    { $(#[$attr:meta])* struct $name:ident([$type:ty; $lanes:tt]); } => {
-        define_type! { @impl $(#[$attr])* | $name [$type; $lanes] }
+/// Calls a the macro `$mac` with the provided `$args` followed by `$repeat` repeated the specified
+/// number of times.
+macro_rules! call_repeat {
+    { 1 => $mac:path [$($repeat:tt)*] $($args:tt)* } => {
+        $mac! {
+            $($args)*
+            $($repeat)*
+        }
+    };
+    { 2 => $mac:path [$($repeat:tt)*] $($args:tt)* } => {
+        $mac! {
+            $($args)*
+            $($repeat)* $($repeat)*
+        }
+    };
+    { 4 => $mac:path [$($repeat:tt)*] $($args:tt)* } => {
+        $mac! {
+            $($args)*
+            $($repeat)* $($repeat)* $($repeat)* $($repeat)*
+        }
+    };
+    { 8 => $mac:path [$($repeat:tt)*] $($args:tt)* } => {
+        $mac! {
+            $($args)*
+            $($repeat)* $($repeat)* $($repeat)* $($repeat)* $($repeat)* $($repeat)* $($repeat)* $($repeat)*
+        }
+    };
+    { 16 => $mac:path [$($repeat:tt)*] $($args:tt)* } => {
+        $mac! {
+            $($args)*
+            $($repeat)* $($repeat)* $($repeat)* $($repeat)* $($repeat)* $($repeat)* $($repeat)* $($repeat)*
+            $($repeat)* $($repeat)* $($repeat)* $($repeat)* $($repeat)* $($repeat)* $($repeat)* $($repeat)*
+        }
+    };
+    { 32 => $mac:path [$($repeat:tt)*] $($args:tt)* } => {
+        $mac! {
+            $($args)*
+            $($repeat)* $($repeat)* $($repeat)* $($repeat)* $($repeat)* $($repeat)* $($repeat)* $($repeat)*
+            $($repeat)* $($repeat)* $($repeat)* $($repeat)* $($repeat)* $($repeat)* $($repeat)* $($repeat)*
+            $($repeat)* $($repeat)* $($repeat)* $($repeat)* $($repeat)* $($repeat)* $($repeat)* $($repeat)*
+            $($repeat)* $($repeat)* $($repeat)* $($repeat)* $($repeat)* $($repeat)* $($repeat)* $($repeat)*
+        }
+    };
+    { 64 => $mac:path [$($repeat:tt)*] $($args:tt)* } => {
+        $mac! {
+            $($args)*
+            $($repeat)* $($repeat)* $($repeat)* $($repeat)* $($repeat)* $($repeat)* $($repeat)* $($repeat)*
+            $($repeat)* $($repeat)* $($repeat)* $($repeat)* $($repeat)* $($repeat)* $($repeat)* $($repeat)*
+            $($repeat)* $($repeat)* $($repeat)* $($repeat)* $($repeat)* $($repeat)* $($repeat)* $($repeat)*
+            $($repeat)* $($repeat)* $($repeat)* $($repeat)* $($repeat)* $($repeat)* $($repeat)* $($repeat)*
+            $($repeat)* $($repeat)* $($repeat)* $($repeat)* $($repeat)* $($repeat)* $($repeat)* $($repeat)*
+            $($repeat)* $($repeat)* $($repeat)* $($repeat)* $($repeat)* $($repeat)* $($repeat)* $($repeat)*
+            $($repeat)* $($repeat)* $($repeat)* $($repeat)* $($repeat)* $($repeat)* $($repeat)* $($repeat)*
+            $($repeat)* $($repeat)* $($repeat)* $($repeat)* $($repeat)* $($repeat)* $($repeat)* $($repeat)*
+        }
+    };
+}
 
+/// Calls the macro `$mac` with the specified `$args` followed by the specified number of unique
+/// identifiers.
+macro_rules! call_counting_args {
+    { 1 => $mac:path => $($args:tt)* } => {
+        $mac! {
+            $($args)*
+            value
+        }
+    };
+    { 2 => $mac:path => $($args:tt)* } => {
+        $mac! {
+            $($args)*
+            v1 v2
+        }
+    };
+    { 4 => $mac:path => $($args:tt)* } => {
+        $mac! {
+            $($args)*
+            v0 v1 v2 v3
+        }
+    };
+    { 8 => $mac:path => $($args:tt)* } => {
+        $mac! {
+            $($args)*
+            v0 v1 v2 v3 v4 v5 v6 v7
+        }
+    };
+    { 16 => $mac:path => $($args:tt)* } => {
+        $mac! {
+            $($args)*
+            v0 v1 v2 v3 v4 v5 v6 v7 v8 v9 v10 v11 v12 v13 v14 v15
+        }
+    };
+    { 32 => $mac:path => $($args:tt)* } => {
+        $mac! {
+            $($args)*
+            v0  v1  v2  v3  v4  v5  v6  v7  v8  v9  v10 v11 v12 v13 v14 v15
+            v16 v17 v18 v19 v20 v21 v22 v23 v24 v25 v26 v27 v28 v29 v30 v31
+        }
+    };
+    { 64 => $mac:path => $($args:tt)* } => {
+        $mac! {
+            $($args)*
+            v0  v1  v2  v3  v4  v5  v6  v7  v8  v9  v10 v11 v12 v13 v14 v15
+            v16 v17 v18 v19 v20 v21 v22 v23 v24 v25 v26 v27 v28 v29 v30 v31
+            v32 v33 v34 v35 v36 v37 v38 v39 v40 v41 v42 v43 v44 v45 v46 v47
+            v48 v49 v50 v51 v52 v53 v54 v55 v56 v57 v58 v59 v60 v61 v62 v63
+        }
+    };
+}
+
+/// Implements common traits on the specified vector `$name`, holding multiple `$lanes` of `$type`.
+macro_rules! base_vector_traits {
+    { $name:path => [$type:ty; $lanes:literal] } => {
         // array references
         impl AsRef<[$type; $lanes]> for $name {
             #[inline]
@@ -67,71 +178,78 @@ macro_rules! define_type {
                 Self::splat(value)
             }
         }
-    };
-    { @impl $(#[$attr:meta])* | $name:ident [$type:ty; 1] } => {
-        define_type! { @def $(#[$attr])* | $name | $type | $type, | v0, }
-    };
-    { @impl $(#[$attr:meta])* | $name:ident [$type:ty; 2] } => {
-        define_type! { @def $(#[$attr])* | $name | $type | $type, $type, | v0, v1, }
-    };
-    { @impl $(#[$attr:meta])* | $name:ident [$type:ty; 4] } => {
-        define_type! { @def $(#[$attr])* | $name | $type |
-            $type, $type, $type, $type, |
-            v0, v1, v2, v3,
+    }
+}
+
+/// Defines a vector `$name` containing multiple `$lanes` of `$type`.
+macro_rules! define_vector {
+    { $(#[$attr:meta])* struct $name:ident([$type:ty; $lanes:tt]); } => {
+        call_repeat! { $lanes => define_vector [$type] def $(#[$attr])* | $name | }
+
+        impl $name {
+            call_repeat! { $lanes => define_vector [$type] splat $type | }
+            call_counting_args! { $lanes => define_vector => new $type | }
         }
+
+        base_vector_traits! { $name => [$type; $lanes] }
     };
-    { @impl $(#[$attr:meta])* | $name:ident [$type:ty; 8] } => {
-        define_type! { @def $(#[$attr])* | $name | $type |
-            $type, $type, $type, $type, $type, $type, $type, $type, |
-            v0, v1, v2, v3, v4, v5, v6, v7,
-        }
-    };
-    { @impl $(#[$attr:meta])* | $name:ident [$type:ty; 16] } => {
-        define_type! { @def $(#[$attr])* | $name | $type |
-            $type, $type, $type, $type, $type, $type, $type, $type, $type, $type, $type, $type, $type, $type, $type, $type, |
-            v0, v1, v2, v3, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13, v14, v15,
-        }
-    };
-    { @impl $(#[$attr:meta])* | $name:ident [$type:ty; 32] } => {
-        define_type! { @def $(#[$attr])* | $name | $type |
-            $type, $type, $type, $type, $type, $type, $type, $type, $type, $type, $type, $type, $type, $type, $type, $type,
-            $type, $type, $type, $type, $type, $type, $type, $type, $type, $type, $type, $type, $type, $type, $type, $type, |
-            v0,  v1,  v2,  v3,  v4,  v5,  v6,  v7,  v8,  v9,  v10, v11, v12, v13, v14, v15,
-            v16, v17, v18, v19, v20, v21, v22, v23, v24, v25, v26, v27, v28, v29, v30, v31,
-        }
-    };
-    { @impl $(#[$attr:meta])* | $name:ident [$type:ty; 64] } => {
-        define_type! { @def $(#[$attr])* | $name | $type |
-            $type, $type, $type, $type, $type, $type, $type, $type, $type, $type, $type, $type, $type, $type, $type, $type,
-            $type, $type, $type, $type, $type, $type, $type, $type, $type, $type, $type, $type, $type, $type, $type, $type,
-            $type, $type, $type, $type, $type, $type, $type, $type, $type, $type, $type, $type, $type, $type, $type, $type,
-            $type, $type, $type, $type, $type, $type, $type, $type, $type, $type, $type, $type, $type, $type, $type, $type, |
-            v0,  v1,  v2,  v3,  v4,  v5,  v6,  v7,  v8,  v9,  v10, v11, v12, v13, v14, v15,
-            v16, v17, v18, v19, v20, v21, v22, v23, v24, v25, v26, v27, v28, v29, v30, v31,
-            v32, v33, v34, v35, v36, v37, v38, v39, v40, v41, v42, v43, v44, v45, v46, v47,
-            v48, v49, v50, v51, v52, v53, v54, v55, v56, v57, v58, v59, v60, v61, v62, v63,
-        }
-    };
-    { @def $(#[$attr:meta])* | $name:ident | $type:ty | $($itype:ty,)* | $($ivar:ident,)* } => {
+    { def $(#[$attr:meta])* | $name:ident | $($itype:ty)* } => {
         $(#[$attr])*
         #[allow(non_camel_case_types)]
         #[derive(Copy, Clone, Debug, Default, PartialEq, PartialOrd)]
         #[repr(simd)]
         pub struct $name($($itype),*);
-
-        impl $name {
-            /// Construct a vector by setting all lanes to the given value.
-            #[inline]
-            pub const fn splat(value: $type) -> Self {
-                Self($(value as $itype),*)
-            }
-
-            /// Construct a vector by setting each lane to the given values.
-            #[allow(clippy::too_many_arguments)]
-            #[inline]
-            pub const fn new($($ivar: $itype),*) -> Self {
-                Self($($ivar),*)
-            }
+    };
+    { splat $type:ty | $($itype:ty)* } => {
+        /// Construct a vector by setting all lanes to the given value.
+        #[inline]
+        pub const fn splat(value: $type) -> Self {
+            Self($(value as $itype),*)
+        }
+    };
+    { new $type:ty | $($var:ident)* } => {
+        /// Construct a vector by setting each lane to the given values.
+        #[allow(clippy::too_many_arguments)]
+        #[inline]
+        pub const fn new($($var: $type),*) -> Self {
+            Self($($var),*)
+        }
+    }
+}
+
+/// Defines a mask vector `$name` containing multiple `$lanes` of `$type`, represented by the
+/// underlying type `$impl_type`.
+macro_rules! define_mask_vector {
+    { $(#[$attr:meta])* struct $name:ident([$impl_type:ty as $type:ty; $lanes:tt]); } => {
+        call_repeat! { $lanes => define_mask_vector [$impl_type] def $(#[$attr])* | $name | }
+
+        impl $name {
+            call_repeat! { $lanes => define_mask_vector [$impl_type] splat $type | }
+            call_counting_args! { $lanes => define_mask_vector => new $type | }
+        }
+
+        base_vector_traits! { $name => [$type; $lanes] }
+    };
+    { def $(#[$attr:meta])* | $name:ident | $($itype:ty)* } => {
+        $(#[$attr])*
+        #[allow(non_camel_case_types)]
+        #[derive(Copy, Clone, Debug, Default, PartialEq, PartialOrd)]
+        #[repr(simd)]
+        pub struct $name($($itype),*);
+    };
+    { splat $type:ty | $($itype:ty)* } => {
+        /// Construct a vector by setting all lanes to the given value.
+        #[inline]
+        pub const fn splat(value: $type) -> Self {
+            Self($(value.0 as $itype),*)
+        }
+    };
+    { new $type:ty | $($var:ident)* } => {
+        /// Construct a vector by setting each lane to the given values.
+        #[allow(clippy::too_many_arguments)]
+        #[inline]
+        pub const fn new($($var: $type),*) -> Self {
+            Self($($var.0),*)
         }
     }
 }
diff --git a/crates/core_simd/src/masks.rs b/crates/core_simd/src/masks.rs
new file mode 100644
index 000000000000..970047aabdef
--- /dev/null
+++ b/crates/core_simd/src/masks.rs
@@ -0,0 +1,67 @@
+macro_rules! define_mask {
+    { $(#[$attr:meta])* struct $name:ident($type:ty); } => {
+        $(#[$attr])*
+        #[allow(non_camel_case_types)]
+        #[derive(Copy, Clone, Debug, Default, PartialEq, PartialOrd, Eq, Ord, Hash)]
+        #[repr(transparent)]
+        pub struct $name(pub(crate) $type);
+
+        impl $name {
+            /// Construct a mask from the given value.
+            pub const fn new(value: bool) -> Self {
+                if value {
+                    Self(!0)
+                } else {
+                    Self(0)
+                }
+            }
+
+            /// Test if the mask is set.
+            pub const fn test(&self) -> bool {
+                self.0 != 0
+            }
+        }
+
+        impl core::convert::From<bool> for $name {
+            fn from(value: bool) -> Self {
+                Self::new(value)
+            }
+        }
+
+        impl core::convert::From<$name> for bool {
+            fn from(mask: $name) -> Self {
+                mask.test()
+            }
+        }
+    }
+}
+
+define_mask! {
+    #[doc = "8-bit mask"]
+    struct mask8(i8);
+}
+
+define_mask! {
+    #[doc = "16-bit mask"]
+    struct mask16(i16);
+}
+
+define_mask! {
+    #[doc = "32-bit mask"]
+    struct mask32(i32);
+}
+
+define_mask! {
+    #[doc = "64-bit mask"]
+    struct mask64(i64);
+}
+
+define_mask! {
+    #[doc = "128-bit mask"]
+    struct mask128(i128);
+}
+
+define_mask! {
+    #[doc = "`isize`-wide mask"]
+    struct masksize(isize);
+}
diff --git a/crates/core_simd/src/vectors_f32.rs b/crates/core_simd/src/vectors_f32.rs
index 3b38b26d6310..c2987a7f6314 100644
--- a/crates/core_simd/src/vectors_f32.rs
+++ b/crates/core_simd/src/vectors_f32.rs
@@ -1,19 +1,19 @@
-define_type! {
+define_vector! {
     #[doc = "Vector of two `f32` values"]
     struct f32x2([f32; 2]);
 }
 
-define_type! {
+define_vector! {
     #[doc = "Vector of four `f32` values"]
     struct f32x4([f32; 4]);
 }
 
-define_type! {
+define_vector! {
     #[doc = "Vector of eight `f32` values"]
     struct f32x8([f32; 8]);
 }
 
-define_type! {
+define_vector! {
     #[doc = "Vector of 16 `f32` values"]
     struct f32x16([f32; 16]);
 }
diff --git a/crates/core_simd/src/vectors_f64.rs b/crates/core_simd/src/vectors_f64.rs
index f627fffd46b4..4881df2b6dac 100644
--- a/crates/core_simd/src/vectors_f64.rs
+++ b/crates/core_simd/src/vectors_f64.rs
@@ -1,14 +1,14 @@
-define_type! {
+define_vector! {
     #[doc = "Vector of two `f64` values"]
     struct f64x2([f64; 2]);
 }
 
-define_type! {
+define_vector! {
     #[doc = "Vector of four `f64` values"]
     struct f64x4([f64; 4]);
 }
 
-define_type! {
+define_vector! {
     #[doc = "Vector of eight `f64` values"]
     struct f64x8([f64; 8]);
 }
diff --git a/crates/core_simd/src/vectors_i128.rs b/crates/core_simd/src/vectors_i128.rs
index b4f30f845fd1..939aeb8ce2e9 100644
--- a/crates/core_simd/src/vectors_i128.rs
+++ b/crates/core_simd/src/vectors_i128.rs
@@ -1,9 +1,9 @@
-define_type! {
+define_vector! {
     #[doc = "Vector of two `i128` values"]
     struct i128x2([i128; 2]);
 }
 
-define_type! {
+define_vector! {
     #[doc = "Vector of four `i128` values"]
     struct i128x4([i128; 4]);
 }
diff --git a/crates/core_simd/src/vectors_i16.rs b/crates/core_simd/src/vectors_i16.rs
index a87fa8c41cd5..b9089a3b4edf 100644
--- a/crates/core_simd/src/vectors_i16.rs
+++ b/crates/core_simd/src/vectors_i16.rs
@@ -1,24 +1,24 @@
-define_type! {
+define_vector! {
     #[doc = "Vector of two `i16` values"]
     struct i16x2([i16; 2]);
 }
 
-define_type! {
+define_vector! {
     #[doc = "Vector of four `i16` values"]
     struct i16x4([i16; 4]);
 }
 
-define_type! {
+define_vector! {
     #[doc = "Vector of eight `i16` values"]
     struct i16x8([i16; 8]);
 }
 
-define_type! {
+define_vector! {
     #[doc = "Vector of 16 `i16` values"]
     struct i16x16([i16; 16]);
 }
 
-define_type! {
+define_vector! {
     #[doc = "Vector of 32 `i16` values"]
     struct i16x32([i16; 32]);
 }
diff --git a/crates/core_simd/src/vectors_i32.rs b/crates/core_simd/src/vectors_i32.rs
index 13c9299b7c6e..686576baff18 100644
--- a/crates/core_simd/src/vectors_i32.rs
+++ b/crates/core_simd/src/vectors_i32.rs
@@ -1,19 +1,19 @@
-define_type! {
+define_vector! {
     #[doc = "Vector of two `i32` values"]
     struct i32x2([i32; 2]);
 }
 
-define_type! {
+define_vector! {
     #[doc = "Vector of four `i32` values"]
     struct i32x4([i32; 4]);
 }
 
-define_type! {
+define_vector! {
     #[doc = "Vector of eight `i32` values"]
     struct i32x8([i32; 8]);
 }
 
-define_type! {
+define_vector! {
     #[doc = "Vector of 16 `i32` values"]
     struct i32x16([i32; 16]);
 }
diff --git a/crates/core_simd/src/vectors_i64.rs b/crates/core_simd/src/vectors_i64.rs
index 8d86501a4961..0a34b4c9d7e8 100644
--- a/crates/core_simd/src/vectors_i64.rs
+++ b/crates/core_simd/src/vectors_i64.rs
@@ -1,14 +1,14 @@
-define_type! {
+define_vector! {
     #[doc = "Vector of two `i64` values"]
     struct i64x2([i64; 2]);
 }
 
-define_type! {
+define_vector! {
     #[doc = "Vector of four `i64` values"]
     struct i64x4([i64; 4]);
 }
 
-define_type! {
+define_vector! {
     #[doc = "Vector of eight `i64` values"]
     struct i64x8([i64; 8]);
 }
diff --git a/crates/core_simd/src/vectors_i8.rs b/crates/core_simd/src/vectors_i8.rs
index f0d5ad55212d..2deacfc1feb7 100644
--- a/crates/core_simd/src/vectors_i8.rs
+++ b/crates/core_simd/src/vectors_i8.rs
@@ -1,29 +1,29 @@
-define_type! {
+define_vector! {
     #[doc = "Vector of two `i8` values"]
     struct i8x2([i8; 2]);
 }
 
-define_type! {
+define_vector! {
     #[doc = "Vector of four `i8` values"]
     struct i8x4([i8; 4]);
 }
 
-define_type! {
+define_vector! {
     #[doc = "Vector of eight `i8` values"]
     struct i8x8([i8; 8]);
 }
 
-define_type! {
+define_vector! {
     #[doc = "Vector of 16 `i8` values"]
     struct i8x16([i8; 16]);
 }
 
-define_type! {
+define_vector! {
     #[doc = "Vector of 32 `i8` values"]
     struct i8x32([i8; 32]);
 }
 
-define_type! {
+define_vector! {
     #[doc = "Vector of 64 `i8` values"]
     struct i8x64([i8; 64]);
 }
diff --git a/crates/core_simd/src/vectors_isize.rs b/crates/core_simd/src/vectors_isize.rs
index 2028d1ecc710..e709d1758704 100644
--- a/crates/core_simd/src/vectors_isize.rs
+++ b/crates/core_simd/src/vectors_isize.rs
@@ -1,14 +1,14 @@
-define_type! {
+define_vector! {
     #[doc = "Vector of two `isize` values"]
     struct isizex2([isize; 2]);
 }
 
-define_type! {
+define_vector! {
     #[doc = "Vector of four `isize` values"]
     struct isizex4([isize; 4]);
 }
 
-define_type! {
+define_vector! {
     #[doc = "Vector of eight `isize` values"]
     struct isizex8([isize; 8]);
 }
diff --git a/crates/core_simd/src/vectors_mask128.rs b/crates/core_simd/src/vectors_mask128.rs
new file mode 100644
index 000000000000..3b1bacb7ad3c
--- /dev/null
+++ b/crates/core_simd/src/vectors_mask128.rs
@@ -0,0 +1,11 @@
+use crate::mask128;
+
+define_mask_vector! {
+    #[doc = "vector of two `mask128` values"]
+    struct mask128x2([i128 as mask128; 2]);
+}
+
+define_mask_vector! {
+    #[doc = "vector of four `mask128` values"]
+    struct mask128x4([i128 as mask128; 4]);
+}
diff --git a/crates/core_simd/src/vectors_mask16.rs b/crates/core_simd/src/vectors_mask16.rs
new file mode 100644
index 000000000000..8d076636caac
--- /dev/null
+++ b/crates/core_simd/src/vectors_mask16.rs
@@ -0,0 +1,26 @@
+use crate::mask16;
+
+define_mask_vector! {
+    #[doc = "vector of two `mask16` values"]
+    struct mask16x2([i16 as mask16; 2]);
+}
+
+define_mask_vector! {
+    #[doc = "vector of four `mask16` values"]
+    struct mask16x4([i16 as mask16; 4]);
+}
+
+define_mask_vector! {
+    #[doc = "vector of eight `mask16` values"]
+    struct mask16x8([i16 as mask16; 8]);
+}
+
+define_mask_vector! {
+    #[doc = "vector of 16 `mask16` values"]
+    struct mask16x16([i16 as mask16; 16]);
+}
+
+define_mask_vector! {
+    #[doc = "vector of 32 `mask16` values"]
+    struct mask16x32([i16 as mask16; 32]);
+}
diff --git a/crates/core_simd/src/vectors_mask32.rs b/crates/core_simd/src/vectors_mask32.rs
new file mode 100644
index 000000000000..64044bc4f578
--- /dev/null
+++ b/crates/core_simd/src/vectors_mask32.rs
@@ -0,0 +1,21 @@
+use crate::mask32;
+
+define_mask_vector! {
+    #[doc = "vector of two `mask32` values"]
+    struct mask32x2([i32 as mask32; 2]);
+}
+
+define_mask_vector! {
+    #[doc = "vector of four `mask32` values"]
+    struct mask32x4([i32 as mask32; 4]);
+}
+
+define_mask_vector! {
+    #[doc = "vector of eight `mask32` values"]
+    struct mask32x8([i32 as mask32; 8]);
+}
+
+define_mask_vector! {
+    #[doc = "vector of 16 `mask32` values"]
+    struct mask32x16([i32 as mask32; 16]);
+}
diff --git a/crates/core_simd/src/vectors_mask64.rs b/crates/core_simd/src/vectors_mask64.rs
new file mode 100644
index 000000000000..b0c62b225c9b
--- /dev/null
+++ b/crates/core_simd/src/vectors_mask64.rs
@@ -0,0 +1,16 @@
+use crate::mask64;
+
+define_mask_vector! {
+    #[doc = "vector of two `mask64` values"]
+    struct mask64x2([i64 as mask64; 2]);
+}
+
+define_mask_vector! {
+    #[doc = "vector of four `mask64` values"]
+    struct mask64x4([i64 as mask64; 4]);
+}
+
+define_mask_vector! {
+    #[doc = "vector of eight `mask64` values"]
+    struct mask64x8([i64 as mask64; 8]);
+}
diff --git a/crates/core_simd/src/vectors_mask8.rs b/crates/core_simd/src/vectors_mask8.rs
new file mode 100644
index 000000000000..c8f3cbac3c91
--- /dev/null
+++ b/crates/core_simd/src/vectors_mask8.rs
@@ -0,0 +1,31 @@
+use crate::mask8;
+
+define_mask_vector! {
+    #[doc = "vector of two `mask8` values"]
+    struct mask8x2([i8 as mask8; 2]);
+}
+
+define_mask_vector! {
+    #[doc = "vector of four `mask8` values"]
+    struct mask8x4([i8 as mask8; 4]);
+}
+
+define_mask_vector! {
+    #[doc = "vector of eight `mask8` values"]
+    struct mask8x8([i8 as mask8; 8]);
+}
+
+define_mask_vector! {
+    #[doc = "vector of 16 `mask8` values"]
+    struct mask8x16([i8 as mask8; 16]);
+}
+
+define_mask_vector! {
+    #[doc = "vector of 32 `mask8` values"]
+    struct mask8x32([i8 as mask8; 32]);
+}
+
+define_mask_vector! {
+    #[doc = "vector of 64 `mask8` values"]
+    struct mask8x64([i8 as mask8; 64]);
+}
diff --git a/crates/core_simd/src/vectors_masksize.rs b/crates/core_simd/src/vectors_masksize.rs
new file mode 100644
index 000000000000..1bf911caffc6
--- /dev/null
+++ b/crates/core_simd/src/vectors_masksize.rs
@@ -0,0 +1,16 @@
+use crate::masksize;
+
+define_mask_vector! {
+    #[doc = "vector of two `masksize` values"]
+    struct masksizex2([isize as masksize; 2]);
+}
+
+define_mask_vector! {
+    #[doc = "vector of four `masksize` values"]
+    struct masksizex4([isize as masksize; 4]);
+}
+
+define_mask_vector! {
+    #[doc = "vector of eight `masksize` values"]
+    struct masksizex8([isize as masksize; 8]);
+}
diff --git a/crates/core_simd/src/vectors_u128.rs b/crates/core_simd/src/vectors_u128.rs
index 4b47fb2fa2ec..f98026275c73 100644
--- a/crates/core_simd/src/vectors_u128.rs
+++ b/crates/core_simd/src/vectors_u128.rs
@@ -1,9 +1,9 @@
-define_type! {
+define_vector! {
     #[doc = "Vector of two `u128` values"]
     struct u128x2([u128; 2]);
 }
 
-define_type! {
+define_vector! {
     #[doc = "Vector of four `u128` values"]
     struct u128x4([u128; 4]);
 }
diff --git a/crates/core_simd/src/vectors_u16.rs b/crates/core_simd/src/vectors_u16.rs
index f637d2fe9f2c..e6ec15679f8c 100644
--- a/crates/core_simd/src/vectors_u16.rs
+++ b/crates/core_simd/src/vectors_u16.rs
@@ -1,24 +1,24 @@
-define_type! {
+define_vector! {
     #[doc = "Vector of two `u16` values"]
     struct u16x2([u16; 2]);
 }
 
-define_type! {
+define_vector! {
     #[doc = "Vector of four `u16` values"]
     struct u16x4([u16; 4]);
 }
 
-define_type! {
+define_vector! {
     #[doc = "Vector of eight `u16` values"]
     struct u16x8([u16; 8]);
 }
 
-define_type! {
+define_vector! {
     #[doc = "Vector of 16 `u16` values"]
     struct u16x16([u16; 16]);
 }
 
-define_type! {
+define_vector! {
     #[doc = "Vector of 32 `u16` values"]
     struct u16x32([u16; 32]);
 }
diff --git a/crates/core_simd/src/vectors_u32.rs b/crates/core_simd/src/vectors_u32.rs
index d2f5d7d241c5..e347509fa54b 100644
--- a/crates/core_simd/src/vectors_u32.rs
+++ b/crates/core_simd/src/vectors_u32.rs
@@ -1,19 +1,19 @@
-define_type! {
+define_vector! {
     #[doc = "Vector of two `u32` values"]
     struct u32x2([u32; 2]);
 }
 
-define_type! {
+define_vector! {
     #[doc = "Vector of four `u32` values"]
     struct u32x4([u32; 4]);
 }
 
-define_type! {
+define_vector! {
     #[doc = "Vector of eight `u32` values"]
     struct u32x8([u32; 8]);
 }
 
-define_type! {
+define_vector! {
     #[doc = "Vector of 16 `u32` values"]
     struct u32x16([u32; 16]);
 }
diff --git a/crates/core_simd/src/vectors_u64.rs b/crates/core_simd/src/vectors_u64.rs
index e6e21e62da10..a3b786230975 100644
--- a/crates/core_simd/src/vectors_u64.rs
+++ b/crates/core_simd/src/vectors_u64.rs
@@ -1,14 +1,14 @@
-define_type! {
+define_vector! {
     #[doc = "Vector of two `u64` values"]
     struct u64x2([u64; 2]);
 }
 
-define_type! {
+define_vector! {
     #[doc = "Vector of four `u64` values"]
     struct u64x4([u64; 4]);
 }
 
-define_type! {
+define_vector! {
     #[doc = "Vector of eight `u64` values"]
     struct u64x8([u64; 8]);
 }
diff --git a/crates/core_simd/src/vectors_u8.rs b/crates/core_simd/src/vectors_u8.rs
index f6246f87ecec..eac58dcc9634 100644
--- a/crates/core_simd/src/vectors_u8.rs
+++ b/crates/core_simd/src/vectors_u8.rs
@@ -1,29 +1,29 @@
-define_type! {
+define_vector! {
     #[doc = "Vector of two `u8` values"]
     struct u8x2([u8; 2]);
 }
 
-define_type! {
+define_vector! {
     #[doc = "Vector of four `u8` values"]
     struct u8x4([u8; 4]);
 }
 
-define_type! {
+define_vector! {
     #[doc = "Vector of eight `u8` values"]
     struct u8x8([u8; 8]);
 }
 
-define_type! {
+define_vector! {
     #[doc = "Vector of 16 `u8` values"]
     struct u8x16([u8; 16]);
 }
 
-define_type! {
+define_vector! {
     #[doc = "Vector of 32 `u8` values"]
     struct u8x32([u8; 32]);
 }
 
-define_type! {
+define_vector! {
     #[doc = "Vector of 64 `u8` values"]
     struct u8x64([u8; 64]);
 }
diff --git a/crates/core_simd/src/vectors_usize.rs b/crates/core_simd/src/vectors_usize.rs
index 45da00562f60..dc97715f0c8d 100644
--- a/crates/core_simd/src/vectors_usize.rs
+++ b/crates/core_simd/src/vectors_usize.rs
@@ -1,14 +1,14 @@
-define_type! {
+define_vector! {
     #[doc = "Vector of two `usize` values"]
     struct usizex2([usize; 2]);
 }
 
-define_type! {
+define_vector! {
     #[doc = "Vector of four `usize` values"]
     struct usizex4([usize; 4]);
 }
 
-define_type! {
+define_vector! {
     #[doc = "Vector of eight `usize` values"]
     struct usizex8([usize; 8]);
 }

From 613f2420885432aecafb5493e3d0cd3f9a4d67f1 Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Thu, 24 Sep 2020 21:15:31 -0400
Subject: [PATCH 011/249] Add pointer vectors and implement a few more traits

---
 crates/core_simd/src/lib.rs           |   3 +
 crates/core_simd/src/macros.rs        |  55 +++++++++++-
 crates/core_simd/src/pointers.rs      | 116 ++++++++++++++++++++++++++
 crates/core_simd/src/vectors_i128.rs  |   2 +
 crates/core_simd/src/vectors_i16.rs   |   5 ++
 crates/core_simd/src/vectors_i32.rs   |   4 +
 crates/core_simd/src/vectors_i64.rs   |   3 +
 crates/core_simd/src/vectors_i8.rs    |   6 ++
 crates/core_simd/src/vectors_isize.rs |   3 +
 crates/core_simd/src/vectors_u128.rs  |   2 +
 crates/core_simd/src/vectors_u16.rs   |   5 ++
 crates/core_simd/src/vectors_u32.rs   |   4 +
 crates/core_simd/src/vectors_u64.rs   |   3 +
 crates/core_simd/src/vectors_u8.rs    |   6 ++
 crates/core_simd/src/vectors_usize.rs |   3 +
 15 files changed, 218 insertions(+), 2 deletions(-)
 create mode 100644 crates/core_simd/src/pointers.rs

diff --git a/crates/core_simd/src/lib.rs b/crates/core_simd/src/lib.rs
index 1ee180a5e3a1..34f2b90f7673 100644
--- a/crates/core_simd/src/lib.rs
+++ b/crates/core_simd/src/lib.rs
@@ -9,6 +9,9 @@ mod macros;
 mod masks;
 pub use masks::*;
 
+mod pointers;
+pub use pointers::*;
+
 mod vectors_u8;
 pub use vectors_u8::*;
 mod vectors_u16;
diff --git a/crates/core_simd/src/macros.rs b/crates/core_simd/src/macros.rs
index 3b5d70695c61..6b5599f0170d 100644
--- a/crates/core_simd/src/macros.rs
+++ b/crates/core_simd/src/macros.rs
@@ -96,7 +96,7 @@ macro_rules! call_counting_args {
     { 2 => $mac:path => $($args:tt)* } => {
         $mac! {
             $($args)*
-            v1 v2
+            v0 v1
         }
     };
     { 4 => $mac:path => $($args:tt)* } => {
@@ -135,6 +135,57 @@ macro_rules! call_counting_args {
     };
 }
 
+/// Calls the macro `$mac` with the specified `$args` followed by counting values from 0 to the
+/// specified value.
+macro_rules! call_counting_values {
+    { 1 => $mac:path => $($args:tt)* } => {
+        $mac! {
+            $($args)*
+            0
+        }
+    };
+    { 2 => $mac:path => $($args:tt)* } => {
+        $mac! {
+            $($args)*
+            0 1
+        }
+    };
+    { 4 => $mac:path => $($args:tt)* } => {
+        $mac! {
+            $($args)*
+            0 1 2 3
+        }
+    };
+    { 8 => $mac:path => $($args:tt)* } => {
+        $mac! {
+            $($args)*
+            0 1 2 3 4 5 6 7
+        }
+    };
+    { 16 => $mac:path => $($args:tt)* } => {
+        $mac! {
+            $($args)*
+            0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
+        }
+    };
+    { 32 => $mac:path => $($args:tt)* } => {
+        $mac! {
+            $($args)*
+            0  1  2  3  4  5  6  7  8  9  10 11 12 13 14 15
+            16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31
+        }
+    };
+    { 64 => $mac:path => $($args:tt)* } => {
+        $mac! {
+            $($args)*
+            0  1  2  3  4  5  6  7  8  9  10 11 12 13 14 15
+            16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31
+            32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47
+            48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63
+        }
+    };
+}
+
 /// Implements common traits on the specified vector `$name`, holding multiple `$lanes` of `$type`.
 macro_rules! base_vector_traits {
     { $name:path => [$type:ty; $lanes:literal] } => {
@@ -233,7 +284,7 @@ macro_rules! define_mask_vector {
     { def $(#[$attr:meta])* | $name:ident | $($itype:ty)* } => {
         $(#[$attr])*
         #[allow(non_camel_case_types)]
-        #[derive(Copy, Clone, Debug, Default, PartialEq, PartialOrd)]
+        #[derive(Copy, Clone, Debug, Default, PartialEq, PartialOrd, Eq, Ord)]
         #[repr(simd)]
         pub struct $name($($itype),*);
     };
diff --git a/crates/core_simd/src/pointers.rs b/crates/core_simd/src/pointers.rs
new file mode 100644
index 000000000000..ecd78fb0cb17
--- /dev/null
+++ b/crates/core_simd/src/pointers.rs
@@ -0,0 +1,116 @@
+use core::marker::PhantomData;
+
+use crate::vectors_isize::*;
+
+macro_rules! define_pointer_vector {
+    { $(#[$attr:meta])* $name:ident => $underlying:ty => $lanes:tt, $mut:ident } => {
+        $(#[$attr])*
+        #[allow(non_camel_case_types)]
+        #[repr(C)]
+        pub struct $name<T>($underlying, PhantomData<T>);
+
+        impl<T> Copy for $name<T> {}
+
+        impl<T> Clone for $name<T> {
+            #[inline]
+            fn clone(&self) -> Self {
+                *self
+            }
+        }
+
+        impl<T> core::cmp::PartialEq for $name<T> {
+            #[inline]
+            fn eq(&self, other: &Self) -> bool {
+                self.0.eq(&other.0)
+            }
+        }
+
+        impl<T> core::cmp::Eq for $name<T> {}
+
+        impl<T> core::cmp::PartialOrd for $name<T> {
+            #[inline]
+            fn partial_cmp(&self, other: &Self) -> Option<core::cmp::Ordering> {
+                self.0.partial_cmp(&other.0)
+            }
+        }
+
+        impl<T> core::cmp::Ord for $name<T> {
+            fn cmp(&self, other: &Self) -> core::cmp::Ordering {
+                self.0.cmp(&other.0)
+            }
+        }
+
+        call_counting_values! { $lanes => define_pointer_vector => debug $name | *$mut T | }
+
+        impl<T> $name<T> {
+            /// Construct a vector by setting all lanes to the given value.
+            #[inline]
+            pub fn splat(value: *$mut T) -> Self {
+                Self(<$underlying>::splat(value as isize), PhantomData)
+            }
+            call_counting_args! { $lanes => define_pointer_vector => new $underlying | *$mut T | }
+        }
+
+        // array references
+        impl<T> AsRef<[*$mut T; $lanes]> for $name<T> {
+            #[inline]
+            fn as_ref(&self) -> &[*$mut T; $lanes] {
+                unsafe { &*(self as *const _ as *const _) }
+            }
+        }
+
+        impl<T> AsMut<[*$mut T; $lanes]> for $name<T> {
+            #[inline]
+            fn as_mut(&mut self) -> &mut [*$mut T; $lanes] {
+                unsafe { &mut *(self as *mut _ as *mut _) }
+            }
+        }
+
+        // slice references
+        impl<T> AsRef<[*$mut T]> for $name<T> {
+            #[inline]
+            fn as_ref(&self) -> &[*$mut T] {
+                AsRef::<[*$mut T; $lanes]>::as_ref(self)
+            }
+        }
+
+        impl<T> AsMut<[*$mut T]> for $name<T> {
+            #[inline]
+            fn as_mut(&mut self) -> &mut [*$mut T] {
+                AsMut::<[*$mut T; $lanes]>::as_mut(self)
+            }
+        }
+
+        // splat
+        impl<T> From<*$mut T> for $name<T> {
+            #[inline]
+            fn from(value: *$mut T) -> Self {
+                Self::splat(value)
+            }
+        }
+    };
+    { new $underlying:ty | $type:ty | $($var:ident)* } => {
+        /// Construct a vector by setting each lane to the given values.
+        #[allow(clippy::too_many_arguments)]
+        #[inline]
+        pub fn new($($var: $type),*) -> Self {
+            Self(<$underlying>::new($($var as isize),*), PhantomData)
+        }
+    };
+    { debug $name:ident | $type:ty | $($index:tt)* } => {
+        impl<T> core::fmt::Debug for $name<T> {
+            fn fmt(&self, f: &mut core::fmt::Formatter) -> core::fmt::Result {
+                f.debug_tuple(stringify!($name))
+                    $(.field(&(AsRef::<[isize]>::as_ref(&self.0)[$index] as $type)))*
+                    .finish()
+            }
+        }
+    }
+}
+
+define_pointer_vector! { #[doc = "Vector of two mutable pointers"] mptrx2 => isizex2 => 2, mut }
+define_pointer_vector! { #[doc = "Vector of four mutable pointers"] mptrx4 => isizex4 => 4, mut }
+define_pointer_vector! { #[doc = "Vector of eight mutable pointers"] mptrx8 => isizex8 => 8, mut }
+define_pointer_vector! { #[doc = "Vector of two const pointers"] cptrx2 => isizex2 => 2, const }
+define_pointer_vector! { #[doc = "Vector of four const pointers"] cptrx4 => isizex4 => 4, const }
+define_pointer_vector! { #[doc = "Vector of eight const pointers"] cptrx8 => isizex8 => 8, const }
diff --git a/crates/core_simd/src/vectors_i128.rs b/crates/core_simd/src/vectors_i128.rs
index 939aeb8ce2e9..9eb0a12b8032 100644
--- a/crates/core_simd/src/vectors_i128.rs
+++ b/crates/core_simd/src/vectors_i128.rs
@@ -1,10 +1,12 @@
 define_vector! {
     #[doc = "Vector of two `i128` values"]
+    #[derive(Eq, Ord, Hash)]
     struct i128x2([i128; 2]);
 }
 
 define_vector! {
     #[doc = "Vector of four `i128` values"]
+    #[derive(Eq, Ord, Hash)]
     struct i128x4([i128; 4]);
 }
 
diff --git a/crates/core_simd/src/vectors_i16.rs b/crates/core_simd/src/vectors_i16.rs
index b9089a3b4edf..682ee214eb50 100644
--- a/crates/core_simd/src/vectors_i16.rs
+++ b/crates/core_simd/src/vectors_i16.rs
@@ -1,25 +1,30 @@
 define_vector! {
     #[doc = "Vector of two `i16` values"]
+    #[derive(Eq, Ord, Hash)]
     struct i16x2([i16; 2]);
 }
 
 define_vector! {
     #[doc = "Vector of four `i16` values"]
+    #[derive(Eq, Ord, Hash)]
     struct i16x4([i16; 4]);
 }
 
 define_vector! {
     #[doc = "Vector of eight `i16` values"]
+    #[derive(Eq, Ord, Hash)]
     struct i16x8([i16; 8]);
 }
 
 define_vector! {
     #[doc = "Vector of 16 `i16` values"]
+    #[derive(Eq, Ord, Hash)]
     struct i16x16([i16; 16]);
 }
 
 define_vector! {
     #[doc = "Vector of 32 `i16` values"]
+    #[derive(Eq, Ord, Hash)]
     struct i16x32([i16; 32]);
 }
 
diff --git a/crates/core_simd/src/vectors_i32.rs b/crates/core_simd/src/vectors_i32.rs
index 686576baff18..d528fdbc2670 100644
--- a/crates/core_simd/src/vectors_i32.rs
+++ b/crates/core_simd/src/vectors_i32.rs
@@ -1,20 +1,24 @@
 define_vector! {
     #[doc = "Vector of two `i32` values"]
+    #[derive(Eq, Ord, Hash)]
     struct i32x2([i32; 2]);
 }
 
 define_vector! {
     #[doc = "Vector of four `i32` values"]
+    #[derive(Eq, Ord, Hash)]
     struct i32x4([i32; 4]);
 }
 
 define_vector! {
     #[doc = "Vector of eight `i32` values"]
+    #[derive(Eq, Ord, Hash)]
     struct i32x8([i32; 8]);
 }
 
 define_vector! {
     #[doc = "Vector of 16 `i32` values"]
+    #[derive(Eq, Ord, Hash)]
     struct i32x16([i32; 16]);
 }
 
diff --git a/crates/core_simd/src/vectors_i64.rs b/crates/core_simd/src/vectors_i64.rs
index 0a34b4c9d7e8..09dd27d94e34 100644
--- a/crates/core_simd/src/vectors_i64.rs
+++ b/crates/core_simd/src/vectors_i64.rs
@@ -1,15 +1,18 @@
 define_vector! {
     #[doc = "Vector of two `i64` values"]
+    #[derive(Eq, Ord, Hash)]
     struct i64x2([i64; 2]);
 }
 
 define_vector! {
     #[doc = "Vector of four `i64` values"]
+    #[derive(Eq, Ord, Hash)]
     struct i64x4([i64; 4]);
 }
 
 define_vector! {
     #[doc = "Vector of eight `i64` values"]
+    #[derive(Eq, Ord, Hash)]
     struct i64x8([i64; 8]);
 }
 
diff --git a/crates/core_simd/src/vectors_i8.rs b/crates/core_simd/src/vectors_i8.rs
index 2deacfc1feb7..8c9407c0c52c 100644
--- a/crates/core_simd/src/vectors_i8.rs
+++ b/crates/core_simd/src/vectors_i8.rs
@@ -1,30 +1,36 @@
 define_vector! {
     #[doc = "Vector of two `i8` values"]
+    #[derive(Eq, Ord, Hash)]
     struct i8x2([i8; 2]);
 }
 
 define_vector! {
     #[doc = "Vector of four `i8` values"]
+    #[derive(Eq, Ord, Hash)]
     struct i8x4([i8; 4]);
 }
 
 define_vector! {
     #[doc = "Vector of eight `i8` values"]
+    #[derive(Eq, Ord, Hash)]
     struct i8x8([i8; 8]);
 }
 
 define_vector! {
     #[doc = "Vector of 16 `i8` values"]
+    #[derive(Eq, Ord, Hash)]
     struct i8x16([i8; 16]);
 }
 
 define_vector! {
     #[doc = "Vector of 32 `i8` values"]
+    #[derive(Eq, Ord, Hash)]
     struct i8x32([i8; 32]);
 }
 
 define_vector! {
     #[doc = "Vector of 64 `i8` values"]
+    #[derive(Eq, Ord, Hash)]
     struct i8x64([i8; 64]);
 }
 
diff --git a/crates/core_simd/src/vectors_isize.rs b/crates/core_simd/src/vectors_isize.rs
index e709d1758704..4512d943d431 100644
--- a/crates/core_simd/src/vectors_isize.rs
+++ b/crates/core_simd/src/vectors_isize.rs
@@ -1,15 +1,18 @@
 define_vector! {
     #[doc = "Vector of two `isize` values"]
+    #[derive(Eq, Ord, Hash)]
     struct isizex2([isize; 2]);
 }
 
 define_vector! {
     #[doc = "Vector of four `isize` values"]
+    #[derive(Eq, Ord, Hash)]
     struct isizex4([isize; 4]);
 }
 
 define_vector! {
     #[doc = "Vector of eight `isize` values"]
+    #[derive(Eq, Ord, Hash)]
     struct isizex8([isize; 8]);
 }
 
diff --git a/crates/core_simd/src/vectors_u128.rs b/crates/core_simd/src/vectors_u128.rs
index f98026275c73..8d7418abe4c3 100644
--- a/crates/core_simd/src/vectors_u128.rs
+++ b/crates/core_simd/src/vectors_u128.rs
@@ -1,10 +1,12 @@
 define_vector! {
     #[doc = "Vector of two `u128` values"]
+    #[derive(Eq, Ord, Hash)]
     struct u128x2([u128; 2]);
 }
 
 define_vector! {
     #[doc = "Vector of four `u128` values"]
+    #[derive(Eq, Ord, Hash)]
     struct u128x4([u128; 4]);
 }
 
diff --git a/crates/core_simd/src/vectors_u16.rs b/crates/core_simd/src/vectors_u16.rs
index e6ec15679f8c..6b4f3c553924 100644
--- a/crates/core_simd/src/vectors_u16.rs
+++ b/crates/core_simd/src/vectors_u16.rs
@@ -1,25 +1,30 @@
 define_vector! {
     #[doc = "Vector of two `u16` values"]
+    #[derive(Eq, Ord, Hash)]
     struct u16x2([u16; 2]);
 }
 
 define_vector! {
     #[doc = "Vector of four `u16` values"]
+    #[derive(Eq, Ord, Hash)]
     struct u16x4([u16; 4]);
 }
 
 define_vector! {
     #[doc = "Vector of eight `u16` values"]
+    #[derive(Eq, Ord, Hash)]
     struct u16x8([u16; 8]);
 }
 
 define_vector! {
     #[doc = "Vector of 16 `u16` values"]
+    #[derive(Eq, Ord, Hash)]
     struct u16x16([u16; 16]);
 }
 
 define_vector! {
     #[doc = "Vector of 32 `u16` values"]
+    #[derive(Eq, Ord, Hash)]
     struct u16x32([u16; 32]);
 }
 
diff --git a/crates/core_simd/src/vectors_u32.rs b/crates/core_simd/src/vectors_u32.rs
index e347509fa54b..09aed52e6bb5 100644
--- a/crates/core_simd/src/vectors_u32.rs
+++ b/crates/core_simd/src/vectors_u32.rs
@@ -1,20 +1,24 @@
 define_vector! {
     #[doc = "Vector of two `u32` values"]
+    #[derive(Eq, Ord, Hash)]
     struct u32x2([u32; 2]);
 }
 
 define_vector! {
     #[doc = "Vector of four `u32` values"]
+    #[derive(Eq, Ord, Hash)]
     struct u32x4([u32; 4]);
 }
 
 define_vector! {
     #[doc = "Vector of eight `u32` values"]
+    #[derive(Eq, Ord, Hash)]
     struct u32x8([u32; 8]);
 }
 
 define_vector! {
     #[doc = "Vector of 16 `u32` values"]
+    #[derive(Eq, Ord, Hash)]
     struct u32x16([u32; 16]);
 }
 
diff --git a/crates/core_simd/src/vectors_u64.rs b/crates/core_simd/src/vectors_u64.rs
index a3b786230975..b19739e6657d 100644
--- a/crates/core_simd/src/vectors_u64.rs
+++ b/crates/core_simd/src/vectors_u64.rs
@@ -1,15 +1,18 @@
 define_vector! {
     #[doc = "Vector of two `u64` values"]
+    #[derive(Eq, Ord, Hash)]
     struct u64x2([u64; 2]);
 }
 
 define_vector! {
     #[doc = "Vector of four `u64` values"]
+    #[derive(Eq, Ord, Hash)]
     struct u64x4([u64; 4]);
 }
 
 define_vector! {
     #[doc = "Vector of eight `u64` values"]
+    #[derive(Eq, Ord, Hash)]
     struct u64x8([u64; 8]);
 }
 
diff --git a/crates/core_simd/src/vectors_u8.rs b/crates/core_simd/src/vectors_u8.rs
index eac58dcc9634..1c31578d0a7e 100644
--- a/crates/core_simd/src/vectors_u8.rs
+++ b/crates/core_simd/src/vectors_u8.rs
@@ -1,30 +1,36 @@
 define_vector! {
     #[doc = "Vector of two `u8` values"]
+    #[derive(Eq, Ord, Hash)]
     struct u8x2([u8; 2]);
 }
 
 define_vector! {
     #[doc = "Vector of four `u8` values"]
+    #[derive(Eq, Ord, Hash)]
     struct u8x4([u8; 4]);
 }
 
 define_vector! {
     #[doc = "Vector of eight `u8` values"]
+    #[derive(Eq, Ord, Hash)]
     struct u8x8([u8; 8]);
 }
 
 define_vector! {
     #[doc = "Vector of 16 `u8` values"]
+    #[derive(Eq, Ord, Hash)]
     struct u8x16([u8; 16]);
 }
 
 define_vector! {
     #[doc = "Vector of 32 `u8` values"]
+    #[derive(Eq, Ord, Hash)]
     struct u8x32([u8; 32]);
 }
 
 define_vector! {
     #[doc = "Vector of 64 `u8` values"]
+    #[derive(Eq, Ord, Hash)]
     struct u8x64([u8; 64]);
 }
 
diff --git a/crates/core_simd/src/vectors_usize.rs b/crates/core_simd/src/vectors_usize.rs
index dc97715f0c8d..30c4da8438b4 100644
--- a/crates/core_simd/src/vectors_usize.rs
+++ b/crates/core_simd/src/vectors_usize.rs
@@ -1,15 +1,18 @@
 define_vector! {
     #[doc = "Vector of two `usize` values"]
+    #[derive(Eq, Ord, Hash)]
     struct usizex2([usize; 2]);
 }
 
 define_vector! {
     #[doc = "Vector of four `usize` values"]
+    #[derive(Eq, Ord, Hash)]
     struct usizex4([usize; 4]);
 }
 
 define_vector! {
     #[doc = "Vector of eight `usize` values"]
+    #[derive(Eq, Ord, Hash)]
     struct usizex8([usize; 8]);
 }
 

From b7d1f3e797fbb12059203a640adce59f2a359dc4 Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Fri, 25 Sep 2020 00:44:48 -0400
Subject: [PATCH 012/249] Improve Debug implementation, add additional
 formatting traits

---
 crates/core_simd/src/fmt.rs      | 106 +++++++++++++++++++++++++++++++
 crates/core_simd/src/lib.rs      |   2 +
 crates/core_simd/src/macros.rs   |   4 +-
 crates/core_simd/src/masks.rs    |   8 ++-
 crates/core_simd/src/pointers.rs |   9 ++-
 5 files changed, 123 insertions(+), 6 deletions(-)
 create mode 100644 crates/core_simd/src/fmt.rs

diff --git a/crates/core_simd/src/fmt.rs b/crates/core_simd/src/fmt.rs
new file mode 100644
index 000000000000..62a21b442d26
--- /dev/null
+++ b/crates/core_simd/src/fmt.rs
@@ -0,0 +1,106 @@
+macro_rules! debug_wrapper {
+    { $($trait:ident => $name:ident,)* } => {
+        $(
+            pub(crate) fn $name<T: core::fmt::$trait>(slice: &[T], f: &mut core::fmt::Formatter) -> core::fmt::Result {
+                #[repr(transparent)]
+                struct Wrapper<'a, T: core::fmt::$trait>(&'a T);
+
+                impl<T: core::fmt::$trait> core::fmt::Debug for Wrapper<'_, T> {
+                    fn fmt(&self, f: &mut core::fmt::Formatter) -> core::fmt::Result {
+                        self.0.fmt(f)
+                    }
+                }
+
+                f.debug_list()
+                    .entries(slice.iter().map(|x| Wrapper(x)))
+                    .finish()
+            }
+        )*
+    }
+}
+
+debug_wrapper! {
+    Debug => format,
+    Binary => format_binary,
+    LowerExp => format_lower_exp,
+    UpperExp => format_upper_exp,
+    Octal => format_octal,
+    LowerHex => format_lower_hex,
+    UpperHex => format_upper_hex,
+    Pointer => format_pointer,
+}
+
+macro_rules! impl_fmt_trait {
+    { $($type:ty => $(($trait:ident, $format:ident)),*;)* } => {
+        $( // repeat type
+            $( // repeat trait
+                impl core::fmt::$trait for $type {
+                    fn fmt(&self, f: &mut core::fmt::Formatter) -> core::fmt::Result {
+                        $format(self.as_ref(), f)
+                    }
+                }
+            )*
+        )*
+    };
+    { integers: $($type:ty,)* } => {
+        impl_fmt_trait! {
+            $($type =>
+              (Debug, format),
+              (Binary, format_binary),
+              (LowerExp, format_lower_exp),
+              (UpperExp, format_upper_exp),
+              (Octal, format_octal),
+              (LowerHex, format_lower_hex),
+              (UpperHex, format_upper_hex);
+            )*
+        }
+    };
+    { floats: $($type:ty,)* } => {
+        impl_fmt_trait! {
+            $($type =>
+              (Debug, format),
+              (LowerExp, format_lower_exp),
+              (UpperExp, format_upper_exp);
+            )*
+        }
+    };
+    { masks: $($type:ty,)* } => {
+        impl_fmt_trait! {
+            $($type =>
+              (Debug, format);
+            )*
+        }
+    }
+}
+
+impl_fmt_trait! {
+    integers:
+        crate::u8x2,    crate::u8x4,    crate::u8x8,    crate::u8x16,    crate::u8x32,    crate::u8x64,
+        crate::i8x2,    crate::i8x4,    crate::i8x8,    crate::i8x16,    crate::i8x32,    crate::i8x64,
+        crate::u16x2,   crate::u16x4,   crate::u16x8,   crate::u16x16,   crate::u16x32,
+        crate::i16x2,   crate::i16x4,   crate::i16x8,   crate::i16x16,   crate::i16x32,
+        crate::u32x2,   crate::u32x4,   crate::u32x8,   crate::u32x16,
+        crate::i32x2,   crate::i32x4,   crate::i32x8,   crate::i32x16,
+        crate::u64x2,   crate::u64x4,   crate::u64x8,
+        crate::i64x2,   crate::i64x4,   crate::i64x8,
+        crate::u128x2,  crate::u128x4,
+        crate::i128x2,  crate::i128x4,
+        crate::usizex2, crate::usizex4, crate::usizex8,
+        crate::isizex2, crate::isizex4, crate::isizex8,
+}
+
+impl_fmt_trait! {
+    floats:
+        crate::f32x2, crate::f32x4, crate::f32x8, crate::f32x16,
+        crate::f64x2, crate::f64x4, crate::f64x8,
+}
+
+impl_fmt_trait! {
+    masks:
+        crate::mask8x2,    crate::mask8x4,    crate::mask8x8,    crate::mask8x16,    crate::mask8x32,    crate::mask8x64,
+        crate::mask16x2,   crate::mask16x4,   crate::mask16x8,   crate::mask16x16,   crate::mask16x32,
+        crate::mask32x2,   crate::mask32x4,   crate::mask32x8,   crate::mask32x16,
+        crate::mask64x2,   crate::mask64x4,   crate::mask64x8,
+        crate::mask128x2,  crate::mask128x4,
+        crate::masksizex2, crate::masksizex4, crate::masksizex8,
+}
diff --git a/crates/core_simd/src/lib.rs b/crates/core_simd/src/lib.rs
index 34f2b90f7673..f6ddc4c6cfda 100644
--- a/crates/core_simd/src/lib.rs
+++ b/crates/core_simd/src/lib.rs
@@ -6,6 +6,8 @@
 #[macro_use]
 mod macros;
 
+mod fmt;
+
 mod masks;
 pub use masks::*;
 
diff --git a/crates/core_simd/src/macros.rs b/crates/core_simd/src/macros.rs
index 6b5599f0170d..33541899ca36 100644
--- a/crates/core_simd/src/macros.rs
+++ b/crates/core_simd/src/macros.rs
@@ -247,7 +247,7 @@ macro_rules! define_vector {
     { def $(#[$attr:meta])* | $name:ident | $($itype:ty)* } => {
         $(#[$attr])*
         #[allow(non_camel_case_types)]
-        #[derive(Copy, Clone, Debug, Default, PartialEq, PartialOrd)]
+        #[derive(Copy, Clone, Default, PartialEq, PartialOrd)]
         #[repr(simd)]
         pub struct $name($($itype),*);
     };
@@ -284,7 +284,7 @@ macro_rules! define_mask_vector {
     { def $(#[$attr:meta])* | $name:ident | $($itype:ty)* } => {
         $(#[$attr])*
         #[allow(non_camel_case_types)]
-        #[derive(Copy, Clone, Debug, Default, PartialEq, PartialOrd, Eq, Ord)]
+        #[derive(Copy, Clone, Default, PartialEq, PartialOrd, Eq, Ord)]
         #[repr(simd)]
         pub struct $name($($itype),*);
     };
diff --git a/crates/core_simd/src/masks.rs b/crates/core_simd/src/masks.rs
index 970047aabdef..ceefbfc94447 100644
--- a/crates/core_simd/src/masks.rs
+++ b/crates/core_simd/src/masks.rs
@@ -2,7 +2,7 @@ macro_rules! define_mask {
     { $(#[$attr:meta])* struct $name:ident($type:ty); } => {
         $(#[$attr])*
         #[allow(non_camel_case_types)]
-        #[derive(Copy, Clone, Debug, Default, PartialEq, PartialOrd, Eq, Ord, Hash)]
+        #[derive(Copy, Clone, Default, PartialEq, PartialOrd, Eq, Ord, Hash)]
         #[repr(transparent)]
         pub struct $name(pub(crate) $type);
 
@@ -33,6 +33,12 @@ macro_rules! define_mask {
                 mask.test()
             }
         }
+
+        impl core::fmt::Debug for $name {
+            fn fmt(&self, f: &mut core::fmt::Formatter) -> core::fmt::Result {
+                self.test().fmt(f)
+            }
+        }
     }
 }
 
diff --git a/crates/core_simd/src/pointers.rs b/crates/core_simd/src/pointers.rs
index ecd78fb0cb17..a7c514aae7a3 100644
--- a/crates/core_simd/src/pointers.rs
+++ b/crates/core_simd/src/pointers.rs
@@ -100,9 +100,12 @@ macro_rules! define_pointer_vector {
     { debug $name:ident | $type:ty | $($index:tt)* } => {
         impl<T> core::fmt::Debug for $name<T> {
             fn fmt(&self, f: &mut core::fmt::Formatter) -> core::fmt::Result {
-                f.debug_tuple(stringify!($name))
-                    $(.field(&(AsRef::<[isize]>::as_ref(&self.0)[$index] as $type)))*
-                    .finish()
+                crate::fmt::format(self.as_ref(), f)
+            }
+        }
+        impl<T> core::fmt::Pointer for $name<T> {
+            fn fmt(&self, f: &mut core::fmt::Formatter) -> core::fmt::Result {
+                crate::fmt::format_pointer(self.as_ref(), f)
             }
         }
     }

From 2178409df574aa83c5e9e6e7bff62b66c32acf63 Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Fri, 25 Sep 2020 00:52:32 -0400
Subject: [PATCH 013/249] Remove some obsolete macros

---
 crates/core_simd/src/macros.rs   | 51 --------------------------------
 crates/core_simd/src/pointers.rs | 25 +++++++---------
 2 files changed, 11 insertions(+), 65 deletions(-)

diff --git a/crates/core_simd/src/macros.rs b/crates/core_simd/src/macros.rs
index 33541899ca36..591a85af08f1 100644
--- a/crates/core_simd/src/macros.rs
+++ b/crates/core_simd/src/macros.rs
@@ -135,57 +135,6 @@ macro_rules! call_counting_args {
     };
 }
 
-/// Calls the macro `$mac` with the specified `$args` followed by counting values from 0 to the
-/// specified value.
-macro_rules! call_counting_values {
-    { 1 => $mac:path => $($args:tt)* } => {
-        $mac! {
-            $($args)*
-            0
-        }
-    };
-    { 2 => $mac:path => $($args:tt)* } => {
-        $mac! {
-            $($args)*
-            0 1
-        }
-    };
-    { 4 => $mac:path => $($args:tt)* } => {
-        $mac! {
-            $($args)*
-            0 1 2 3
-        }
-    };
-    { 8 => $mac:path => $($args:tt)* } => {
-        $mac! {
-            $($args)*
-            0 1 2 3 4 5 6 7
-        }
-    };
-    { 16 => $mac:path => $($args:tt)* } => {
-        $mac! {
-            $($args)*
-            0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
-        }
-    };
-    { 32 => $mac:path => $($args:tt)* } => {
-        $mac! {
-            $($args)*
-            0  1  2  3  4  5  6  7  8  9  10 11 12 13 14 15
-            16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31
-        }
-    };
-    { 64 => $mac:path => $($args:tt)* } => {
-        $mac! {
-            $($args)*
-            0  1  2  3  4  5  6  7  8  9  10 11 12 13 14 15
-            16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31
-            32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47
-            48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63
-        }
-    };
-}
-
 /// Implements common traits on the specified vector `$name`, holding multiple `$lanes` of `$type`.
 macro_rules! base_vector_traits {
     { $name:path => [$type:ty; $lanes:literal] } => {
diff --git a/crates/core_simd/src/pointers.rs b/crates/core_simd/src/pointers.rs
index a7c514aae7a3..4c20eab99209 100644
--- a/crates/core_simd/src/pointers.rs
+++ b/crates/core_simd/src/pointers.rs
@@ -9,6 +9,17 @@ macro_rules! define_pointer_vector {
         #[repr(C)]
         pub struct $name<T>($underlying, PhantomData<T>);
 
+        impl<T> core::fmt::Debug for $name<T> {
+            fn fmt(&self, f: &mut core::fmt::Formatter) -> core::fmt::Result {
+                crate::fmt::format(self.as_ref(), f)
+            }
+        }
+        impl<T> core::fmt::Pointer for $name<T> {
+            fn fmt(&self, f: &mut core::fmt::Formatter) -> core::fmt::Result {
+                crate::fmt::format_pointer(self.as_ref(), f)
+            }
+        }
+
         impl<T> Copy for $name<T> {}
 
         impl<T> Clone for $name<T> {
@@ -40,8 +51,6 @@ macro_rules! define_pointer_vector {
             }
         }
 
-        call_counting_values! { $lanes => define_pointer_vector => debug $name | *$mut T | }
-
         impl<T> $name<T> {
             /// Construct a vector by setting all lanes to the given value.
             #[inline]
@@ -97,18 +106,6 @@ macro_rules! define_pointer_vector {
             Self(<$underlying>::new($($var as isize),*), PhantomData)
         }
     };
-    { debug $name:ident | $type:ty | $($index:tt)* } => {
-        impl<T> core::fmt::Debug for $name<T> {
-            fn fmt(&self, f: &mut core::fmt::Formatter) -> core::fmt::Result {
-                crate::fmt::format(self.as_ref(), f)
-            }
-        }
-        impl<T> core::fmt::Pointer for $name<T> {
-            fn fmt(&self, f: &mut core::fmt::Formatter) -> core::fmt::Result {
-                crate::fmt::format_pointer(self.as_ref(), f)
-            }
-        }
-    }
 }
 
 define_pointer_vector! { #[doc = "Vector of two mutable pointers"] mptrx2 => isizex2 => 2, mut }

From 992768709f555f716f6869646937d236111a40dd Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Fri, 25 Sep 2020 21:45:09 -0400
Subject: [PATCH 014/249] Remove pointer vectors

---
 crates/core_simd/src/fmt.rs      |   1 -
 crates/core_simd/src/lib.rs      |   3 -
 crates/core_simd/src/pointers.rs | 116 -------------------------------
 3 files changed, 120 deletions(-)
 delete mode 100644 crates/core_simd/src/pointers.rs

diff --git a/crates/core_simd/src/fmt.rs b/crates/core_simd/src/fmt.rs
index 62a21b442d26..e505947e1e65 100644
--- a/crates/core_simd/src/fmt.rs
+++ b/crates/core_simd/src/fmt.rs
@@ -27,7 +27,6 @@ debug_wrapper! {
     Octal => format_octal,
     LowerHex => format_lower_hex,
     UpperHex => format_upper_hex,
-    Pointer => format_pointer,
 }
 
 macro_rules! impl_fmt_trait {
diff --git a/crates/core_simd/src/lib.rs b/crates/core_simd/src/lib.rs
index f6ddc4c6cfda..d88f5b1eac45 100644
--- a/crates/core_simd/src/lib.rs
+++ b/crates/core_simd/src/lib.rs
@@ -11,9 +11,6 @@ mod fmt;
 mod masks;
 pub use masks::*;
 
-mod pointers;
-pub use pointers::*;
-
 mod vectors_u8;
 pub use vectors_u8::*;
 mod vectors_u16;
diff --git a/crates/core_simd/src/pointers.rs b/crates/core_simd/src/pointers.rs
deleted file mode 100644
index 4c20eab99209..000000000000
--- a/crates/core_simd/src/pointers.rs
+++ /dev/null
@@ -1,116 +0,0 @@
-use core::marker::PhantomData;
-
-use crate::vectors_isize::*;
-
-macro_rules! define_pointer_vector {
-    { $(#[$attr:meta])* $name:ident => $underlying:ty => $lanes:tt, $mut:ident } => {
-        $(#[$attr])*
-        #[allow(non_camel_case_types)]
-        #[repr(C)]
-        pub struct $name<T>($underlying, PhantomData<T>);
-
-        impl<T> core::fmt::Debug for $name<T> {
-            fn fmt(&self, f: &mut core::fmt::Formatter) -> core::fmt::Result {
-                crate::fmt::format(self.as_ref(), f)
-            }
-        }
-        impl<T> core::fmt::Pointer for $name<T> {
-            fn fmt(&self, f: &mut core::fmt::Formatter) -> core::fmt::Result {
-                crate::fmt::format_pointer(self.as_ref(), f)
-            }
-        }
-
-        impl<T> Copy for $name<T> {}
-
-        impl<T> Clone for $name<T> {
-            #[inline]
-            fn clone(&self) -> Self {
-                *self
-            }
-        }
-
-        impl<T> core::cmp::PartialEq for $name<T> {
-            #[inline]
-            fn eq(&self, other: &Self) -> bool {
-                self.0.eq(&other.0)
-            }
-        }
-
-        impl<T> core::cmp::Eq for $name<T> {}
-
-        impl<T> core::cmp::PartialOrd for $name<T> {
-            #[inline]
-            fn partial_cmp(&self, other: &Self) -> Option<core::cmp::Ordering> {
-                self.0.partial_cmp(&other.0)
-            }
-        }
-
-        impl<T> core::cmp::Ord for $name<T> {
-            fn cmp(&self, other: &Self) -> core::cmp::Ordering {
-                self.0.cmp(&other.0)
-            }
-        }
-
-        impl<T> $name<T> {
-            /// Construct a vector by setting all lanes to the given value.
-            #[inline]
-            pub fn splat(value: *$mut T) -> Self {
-                Self(<$underlying>::splat(value as isize), PhantomData)
-            }
-            call_counting_args! { $lanes => define_pointer_vector => new $underlying | *$mut T | }
-        }
-
-        // array references
-        impl<T> AsRef<[*$mut T; $lanes]> for $name<T> {
-            #[inline]
-            fn as_ref(&self) -> &[*$mut T; $lanes] {
-                unsafe { &*(self as *const _ as *const _) }
-            }
-        }
-
-        impl<T> AsMut<[*$mut T; $lanes]> for $name<T> {
-            #[inline]
-            fn as_mut(&mut self) -> &mut [*$mut T; $lanes] {
-                unsafe { &mut *(self as *mut _ as *mut _) }
-            }
-        }
-
-        // slice references
-        impl<T> AsRef<[*$mut T]> for $name<T> {
-            #[inline]
-            fn as_ref(&self) -> &[*$mut T] {
-                AsRef::<[*$mut T; $lanes]>::as_ref(self)
-            }
-        }
-
-        impl<T> AsMut<[*$mut T]> for $name<T> {
-            #[inline]
-            fn as_mut(&mut self) -> &mut [*$mut T] {
-                AsMut::<[*$mut T; $lanes]>::as_mut(self)
-            }
-        }
-
-        // splat
-        impl<T> From<*$mut T> for $name<T> {
-            #[inline]
-            fn from(value: *$mut T) -> Self {
-                Self::splat(value)
-            }
-        }
-    };
-    { new $underlying:ty | $type:ty | $($var:ident)* } => {
-        /// Construct a vector by setting each lane to the given values.
-        #[allow(clippy::too_many_arguments)]
-        #[inline]
-        pub fn new($($var: $type),*) -> Self {
-            Self(<$underlying>::new($($var as isize),*), PhantomData)
-        }
-    };
-}
-
-define_pointer_vector! { #[doc = "Vector of two mutable pointers"] mptrx2 => isizex2 => 2, mut }
-define_pointer_vector! { #[doc = "Vector of four mutable pointers"] mptrx4 => isizex4 => 4, mut }
-define_pointer_vector! { #[doc = "Vector of eight mutable pointers"] mptrx8 => isizex8 => 8, mut }
-define_pointer_vector! { #[doc = "Vector of two const pointers"] cptrx2 => isizex2 => 2, const }
-define_pointer_vector! { #[doc = "Vector of four const pointers"] cptrx4 => isizex4 => 4, const }
-define_pointer_vector! { #[doc = "Vector of eight const pointers"] cptrx8 => isizex8 => 8, const }

From 167c3c8100bfbe704ecf3743cda2141c6cb6d15a Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Fri, 25 Sep 2020 23:23:02 -0400
Subject: [PATCH 015/249] Change doc attributes to comments

---
 crates/core_simd/src/masks.rs            | 12 ++++++------
 crates/core_simd/src/vectors_f32.rs      |  8 ++++----
 crates/core_simd/src/vectors_f64.rs      |  6 +++---
 crates/core_simd/src/vectors_i128.rs     |  4 ++--
 crates/core_simd/src/vectors_i16.rs      | 10 +++++-----
 crates/core_simd/src/vectors_i32.rs      |  8 ++++----
 crates/core_simd/src/vectors_i64.rs      |  6 +++---
 crates/core_simd/src/vectors_i8.rs       | 12 ++++++------
 crates/core_simd/src/vectors_isize.rs    |  6 +++---
 crates/core_simd/src/vectors_mask128.rs  |  4 ++--
 crates/core_simd/src/vectors_mask16.rs   | 10 +++++-----
 crates/core_simd/src/vectors_mask32.rs   |  8 ++++----
 crates/core_simd/src/vectors_mask64.rs   |  6 +++---
 crates/core_simd/src/vectors_mask8.rs    | 12 ++++++------
 crates/core_simd/src/vectors_masksize.rs |  6 +++---
 crates/core_simd/src/vectors_u128.rs     |  4 ++--
 crates/core_simd/src/vectors_u16.rs      | 10 +++++-----
 crates/core_simd/src/vectors_u32.rs      |  8 ++++----
 crates/core_simd/src/vectors_u64.rs      |  6 +++---
 crates/core_simd/src/vectors_u8.rs       | 12 ++++++------
 crates/core_simd/src/vectors_usize.rs    |  6 +++---
 21 files changed, 82 insertions(+), 82 deletions(-)

diff --git a/crates/core_simd/src/masks.rs b/crates/core_simd/src/masks.rs
index ceefbfc94447..1fc281a310d3 100644
--- a/crates/core_simd/src/masks.rs
+++ b/crates/core_simd/src/masks.rs
@@ -43,31 +43,31 @@ macro_rules! define_mask {
 }
 
 define_mask! {
-    #[doc = "8-bit mask"]
+    /// 8-bit mask
     struct mask8(i8);
 }
 
 define_mask! {
-    #[doc = "16-bit mask"]
+    /// 16-bit mask
     struct mask16(i16);
 }
 
 define_mask! {
-    #[doc = "32-bit mask"]
+    /// 32-bit mask
     struct mask32(i32);
 }
 
 define_mask! {
-    #[doc = "64-bit mask"]
+    /// 64-bit mask
     struct mask64(i64);
 }
 
 define_mask! {
-    #[doc = "128-bit mask"]
+    /// 128-bit mask
     struct mask128(i128);
 }
 
 define_mask! {
-    #[doc = "`isize`-wide mask"]
+    /// `isize`-wide mask
     struct masksize(isize);
 }
diff --git a/crates/core_simd/src/vectors_f32.rs b/crates/core_simd/src/vectors_f32.rs
index c2987a7f6314..9fcbd9d53f0f 100644
--- a/crates/core_simd/src/vectors_f32.rs
+++ b/crates/core_simd/src/vectors_f32.rs
@@ -1,20 +1,20 @@
 define_vector! {
-    #[doc = "Vector of two `f32` values"]
+    /// Vector of two `f32` values
     struct f32x2([f32; 2]);
 }
 
 define_vector! {
-    #[doc = "Vector of four `f32` values"]
+    /// Vector of four `f32` values
     struct f32x4([f32; 4]);
 }
 
 define_vector! {
-    #[doc = "Vector of eight `f32` values"]
+    /// Vector of eight `f32` values
     struct f32x8([f32; 8]);
 }
 
 define_vector! {
-    #[doc = "Vector of 16 `f32` values"]
+    /// Vector of 16 `f32` values
     struct f32x16([f32; 16]);
 }
 
diff --git a/crates/core_simd/src/vectors_f64.rs b/crates/core_simd/src/vectors_f64.rs
index 4881df2b6dac..d741aabe88e0 100644
--- a/crates/core_simd/src/vectors_f64.rs
+++ b/crates/core_simd/src/vectors_f64.rs
@@ -1,15 +1,15 @@
 define_vector! {
-    #[doc = "Vector of two `f64` values"]
+    /// Vector of two `f64` values
     struct f64x2([f64; 2]);
 }
 
 define_vector! {
-    #[doc = "Vector of four `f64` values"]
+    /// Vector of four `f64` values
     struct f64x4([f64; 4]);
 }
 
 define_vector! {
-    #[doc = "Vector of eight `f64` values"]
+    /// Vector of eight `f64` values
     struct f64x8([f64; 8]);
 }
 
diff --git a/crates/core_simd/src/vectors_i128.rs b/crates/core_simd/src/vectors_i128.rs
index 9eb0a12b8032..588f4cffb909 100644
--- a/crates/core_simd/src/vectors_i128.rs
+++ b/crates/core_simd/src/vectors_i128.rs
@@ -1,11 +1,11 @@
 define_vector! {
-    #[doc = "Vector of two `i128` values"]
+    /// Vector of two `i128` values
     #[derive(Eq, Ord, Hash)]
     struct i128x2([i128; 2]);
 }
 
 define_vector! {
-    #[doc = "Vector of four `i128` values"]
+    /// Vector of four `i128` values
     #[derive(Eq, Ord, Hash)]
     struct i128x4([i128; 4]);
 }
diff --git a/crates/core_simd/src/vectors_i16.rs b/crates/core_simd/src/vectors_i16.rs
index 682ee214eb50..52d9ef8379b0 100644
--- a/crates/core_simd/src/vectors_i16.rs
+++ b/crates/core_simd/src/vectors_i16.rs
@@ -1,29 +1,29 @@
 define_vector! {
-    #[doc = "Vector of two `i16` values"]
+    /// Vector of two `i16` values
     #[derive(Eq, Ord, Hash)]
     struct i16x2([i16; 2]);
 }
 
 define_vector! {
-    #[doc = "Vector of four `i16` values"]
+    /// Vector of four `i16` values
     #[derive(Eq, Ord, Hash)]
     struct i16x4([i16; 4]);
 }
 
 define_vector! {
-    #[doc = "Vector of eight `i16` values"]
+    /// Vector of eight `i16` values
     #[derive(Eq, Ord, Hash)]
     struct i16x8([i16; 8]);
 }
 
 define_vector! {
-    #[doc = "Vector of 16 `i16` values"]
+    /// Vector of 16 `i16` values
     #[derive(Eq, Ord, Hash)]
     struct i16x16([i16; 16]);
 }
 
 define_vector! {
-    #[doc = "Vector of 32 `i16` values"]
+    /// Vector of 32 `i16` values
     #[derive(Eq, Ord, Hash)]
     struct i16x32([i16; 32]);
 }
diff --git a/crates/core_simd/src/vectors_i32.rs b/crates/core_simd/src/vectors_i32.rs
index d528fdbc2670..34ecc1694604 100644
--- a/crates/core_simd/src/vectors_i32.rs
+++ b/crates/core_simd/src/vectors_i32.rs
@@ -1,23 +1,23 @@
 define_vector! {
-    #[doc = "Vector of two `i32` values"]
+    /// Vector of two `i32` values
     #[derive(Eq, Ord, Hash)]
     struct i32x2([i32; 2]);
 }
 
 define_vector! {
-    #[doc = "Vector of four `i32` values"]
+    /// Vector of four `i32` values
     #[derive(Eq, Ord, Hash)]
     struct i32x4([i32; 4]);
 }
 
 define_vector! {
-    #[doc = "Vector of eight `i32` values"]
+    /// Vector of eight `i32` values
     #[derive(Eq, Ord, Hash)]
     struct i32x8([i32; 8]);
 }
 
 define_vector! {
-    #[doc = "Vector of 16 `i32` values"]
+    /// Vector of 16 `i32` values
     #[derive(Eq, Ord, Hash)]
     struct i32x16([i32; 16]);
 }
diff --git a/crates/core_simd/src/vectors_i64.rs b/crates/core_simd/src/vectors_i64.rs
index 09dd27d94e34..58893c0dfedc 100644
--- a/crates/core_simd/src/vectors_i64.rs
+++ b/crates/core_simd/src/vectors_i64.rs
@@ -1,17 +1,17 @@
 define_vector! {
-    #[doc = "Vector of two `i64` values"]
+    /// Vector of two `i64` values
     #[derive(Eq, Ord, Hash)]
     struct i64x2([i64; 2]);
 }
 
 define_vector! {
-    #[doc = "Vector of four `i64` values"]
+    /// Vector of four `i64` values
     #[derive(Eq, Ord, Hash)]
     struct i64x4([i64; 4]);
 }
 
 define_vector! {
-    #[doc = "Vector of eight `i64` values"]
+    /// Vector of eight `i64` values
     #[derive(Eq, Ord, Hash)]
     struct i64x8([i64; 8]);
 }
diff --git a/crates/core_simd/src/vectors_i8.rs b/crates/core_simd/src/vectors_i8.rs
index 8c9407c0c52c..c0551fc285e4 100644
--- a/crates/core_simd/src/vectors_i8.rs
+++ b/crates/core_simd/src/vectors_i8.rs
@@ -1,35 +1,35 @@
 define_vector! {
-    #[doc = "Vector of two `i8` values"]
+    /// Vector of two `i8` values
     #[derive(Eq, Ord, Hash)]
     struct i8x2([i8; 2]);
 }
 
 define_vector! {
-    #[doc = "Vector of four `i8` values"]
+    /// Vector of four `i8` values
     #[derive(Eq, Ord, Hash)]
     struct i8x4([i8; 4]);
 }
 
 define_vector! {
-    #[doc = "Vector of eight `i8` values"]
+    /// Vector of eight `i8` values
     #[derive(Eq, Ord, Hash)]
     struct i8x8([i8; 8]);
 }
 
 define_vector! {
-    #[doc = "Vector of 16 `i8` values"]
+    /// Vector of 16 `i8` values
     #[derive(Eq, Ord, Hash)]
     struct i8x16([i8; 16]);
 }
 
 define_vector! {
-    #[doc = "Vector of 32 `i8` values"]
+    /// Vector of 32 `i8` values
     #[derive(Eq, Ord, Hash)]
     struct i8x32([i8; 32]);
 }
 
 define_vector! {
-    #[doc = "Vector of 64 `i8` values"]
+    /// Vector of 64 `i8` values
     #[derive(Eq, Ord, Hash)]
     struct i8x64([i8; 64]);
 }
diff --git a/crates/core_simd/src/vectors_isize.rs b/crates/core_simd/src/vectors_isize.rs
index 4512d943d431..10a892bd0908 100644
--- a/crates/core_simd/src/vectors_isize.rs
+++ b/crates/core_simd/src/vectors_isize.rs
@@ -1,17 +1,17 @@
 define_vector! {
-    #[doc = "Vector of two `isize` values"]
+    /// Vector of two `isize` values
     #[derive(Eq, Ord, Hash)]
     struct isizex2([isize; 2]);
 }
 
 define_vector! {
-    #[doc = "Vector of four `isize` values"]
+    /// Vector of four `isize` values
     #[derive(Eq, Ord, Hash)]
     struct isizex4([isize; 4]);
 }
 
 define_vector! {
-    #[doc = "Vector of eight `isize` values"]
+    /// Vector of eight `isize` values
     #[derive(Eq, Ord, Hash)]
     struct isizex8([isize; 8]);
 }
diff --git a/crates/core_simd/src/vectors_mask128.rs b/crates/core_simd/src/vectors_mask128.rs
index 3b1bacb7ad3c..adf56a3684b3 100644
--- a/crates/core_simd/src/vectors_mask128.rs
+++ b/crates/core_simd/src/vectors_mask128.rs
@@ -1,11 +1,11 @@
 use crate::mask128;
 
 define_mask_vector! {
-    #[doc = "vector of two `mask128` values"]
+    /// Vector of two `mask128` values
     struct mask128x2([i128 as mask128; 2]);
 }
 
 define_mask_vector! {
-    #[doc = "vector of four `mask128` values"]
+    /// Vector of four `mask128` values
     struct mask128x4([i128 as mask128; 4]);
 }
diff --git a/crates/core_simd/src/vectors_mask16.rs b/crates/core_simd/src/vectors_mask16.rs
index 8d076636caac..8cbae6b7bbd2 100644
--- a/crates/core_simd/src/vectors_mask16.rs
+++ b/crates/core_simd/src/vectors_mask16.rs
@@ -1,26 +1,26 @@
 use crate::mask16;
 
 define_mask_vector! {
-    #[doc = "vector of two `mask16` values"]
+    /// Vector of two `mask16` values
     struct mask16x2([i16 as mask16; 2]);
 }
 
 define_mask_vector! {
-    #[doc = "vector of four `mask16` values"]
+    /// Vector of four `mask16` values
     struct mask16x4([i16 as mask16; 4]);
 }
 
 define_mask_vector! {
-    #[doc = "vector of eight `mask16` values"]
+    /// Vector of eight `mask16` values
     struct mask16x8([i16 as mask16; 8]);
 }
 
 define_mask_vector! {
-    #[doc = "vector of 16 `mask16` values"]
+    /// Vector of 16 `mask16` values
     struct mask16x16([i16 as mask16; 16]);
 }
 
 define_mask_vector! {
-    #[doc = "vector of 32 `mask16` values"]
+    /// Vector of 32 `mask16` values
     struct mask16x32([i16 as mask16; 32]);
 }
diff --git a/crates/core_simd/src/vectors_mask32.rs b/crates/core_simd/src/vectors_mask32.rs
index 64044bc4f578..fad191421f38 100644
--- a/crates/core_simd/src/vectors_mask32.rs
+++ b/crates/core_simd/src/vectors_mask32.rs
@@ -1,21 +1,21 @@
 use crate::mask32;
 
 define_mask_vector! {
-    #[doc = "vector of two `mask32` values"]
+    /// Vector of two `mask32` values
     struct mask32x2([i32 as mask32; 2]);
 }
 
 define_mask_vector! {
-    #[doc = "vector of four `mask32` values"]
+    /// Vector of four `mask32` values
     struct mask32x4([i32 as mask32; 4]);
 }
 
 define_mask_vector! {
-    #[doc = "vector of eight `mask32` values"]
+    /// Vector of eight `mask32` values
     struct mask32x8([i32 as mask32; 8]);
 }
 
 define_mask_vector! {
-    #[doc = "vector of 16 `mask32` values"]
+    /// Vector of 16 `mask32` values
     struct mask32x16([i32 as mask32; 16]);
 }
diff --git a/crates/core_simd/src/vectors_mask64.rs b/crates/core_simd/src/vectors_mask64.rs
index b0c62b225c9b..554e731ccf24 100644
--- a/crates/core_simd/src/vectors_mask64.rs
+++ b/crates/core_simd/src/vectors_mask64.rs
@@ -1,16 +1,16 @@
 use crate::mask64;
 
 define_mask_vector! {
-    #[doc = "vector of two `mask64` values"]
+    /// Vector of two `mask64` values
     struct mask64x2([i64 as mask64; 2]);
 }
 
 define_mask_vector! {
-    #[doc = "vector of four `mask64` values"]
+    /// Vector of four `mask64` values
     struct mask64x4([i64 as mask64; 4]);
 }
 
 define_mask_vector! {
-    #[doc = "vector of eight `mask64` values"]
+    /// Vector of eight `mask64` values
     struct mask64x8([i64 as mask64; 8]);
 }
diff --git a/crates/core_simd/src/vectors_mask8.rs b/crates/core_simd/src/vectors_mask8.rs
index c8f3cbac3c91..b6fbe5595189 100644
--- a/crates/core_simd/src/vectors_mask8.rs
+++ b/crates/core_simd/src/vectors_mask8.rs
@@ -1,31 +1,31 @@
 use crate::mask8;
 
 define_mask_vector! {
-    #[doc = "vector of two `mask8` values"]
+    /// Vector of two `mask8` values
     struct mask8x2([i8 as mask8; 2]);
 }
 
 define_mask_vector! {
-    #[doc = "vector of four `mask8` values"]
+    /// Vector of four `mask8` values
     struct mask8x4([i8 as mask8; 4]);
 }
 
 define_mask_vector! {
-    #[doc = "vector of eight `mask8` values"]
+    /// Vector of eight `mask8` values
     struct mask8x8([i8 as mask8; 8]);
 }
 
 define_mask_vector! {
-    #[doc = "vector of 16 `mask8` values"]
+    /// Vector of 16 `mask8` values
     struct mask8x16([i8 as mask8; 16]);
 }
 
 define_mask_vector! {
-    #[doc = "vector of 32 `mask8` values"]
+    /// Vector of 32 `mask8` values
     struct mask8x32([i8 as mask8; 32]);
 }
 
 define_mask_vector! {
-    #[doc = "vector of 64 `mask8` values"]
+    /// Vector of 64 `mask8` values
     struct mask8x64([i8 as mask8; 64]);
 }
diff --git a/crates/core_simd/src/vectors_masksize.rs b/crates/core_simd/src/vectors_masksize.rs
index 1bf911caffc6..a838aee51985 100644
--- a/crates/core_simd/src/vectors_masksize.rs
+++ b/crates/core_simd/src/vectors_masksize.rs
@@ -1,16 +1,16 @@
 use crate::masksize;
 
 define_mask_vector! {
-    #[doc = "vector of two `masksize` values"]
+    /// Vector of two `masksize` values
     struct masksizex2([isize as masksize; 2]);
 }
 
 define_mask_vector! {
-    #[doc = "vector of four `masksize` values"]
+    /// Vector of four `masksize` values
     struct masksizex4([isize as masksize; 4]);
 }
 
 define_mask_vector! {
-    #[doc = "vector of eight `masksize` values"]
+    /// Vector of eight `masksize` values
     struct masksizex8([isize as masksize; 8]);
 }
diff --git a/crates/core_simd/src/vectors_u128.rs b/crates/core_simd/src/vectors_u128.rs
index 8d7418abe4c3..1412dfdc85dc 100644
--- a/crates/core_simd/src/vectors_u128.rs
+++ b/crates/core_simd/src/vectors_u128.rs
@@ -1,11 +1,11 @@
 define_vector! {
-    #[doc = "Vector of two `u128` values"]
+    /// Vector of two `u128` values
     #[derive(Eq, Ord, Hash)]
     struct u128x2([u128; 2]);
 }
 
 define_vector! {
-    #[doc = "Vector of four `u128` values"]
+    /// Vector of four `u128` values
     #[derive(Eq, Ord, Hash)]
     struct u128x4([u128; 4]);
 }
diff --git a/crates/core_simd/src/vectors_u16.rs b/crates/core_simd/src/vectors_u16.rs
index 6b4f3c553924..6fd7c64f4b31 100644
--- a/crates/core_simd/src/vectors_u16.rs
+++ b/crates/core_simd/src/vectors_u16.rs
@@ -1,29 +1,29 @@
 define_vector! {
-    #[doc = "Vector of two `u16` values"]
+    /// Vector of two `u16` values
     #[derive(Eq, Ord, Hash)]
     struct u16x2([u16; 2]);
 }
 
 define_vector! {
-    #[doc = "Vector of four `u16` values"]
+    /// Vector of four `u16` values
     #[derive(Eq, Ord, Hash)]
     struct u16x4([u16; 4]);
 }
 
 define_vector! {
-    #[doc = "Vector of eight `u16` values"]
+    /// Vector of eight `u16` values
     #[derive(Eq, Ord, Hash)]
     struct u16x8([u16; 8]);
 }
 
 define_vector! {
-    #[doc = "Vector of 16 `u16` values"]
+    /// Vector of 16 `u16` values
     #[derive(Eq, Ord, Hash)]
     struct u16x16([u16; 16]);
 }
 
 define_vector! {
-    #[doc = "Vector of 32 `u16` values"]
+    /// Vector of 32 `u16` values
     #[derive(Eq, Ord, Hash)]
     struct u16x32([u16; 32]);
 }
diff --git a/crates/core_simd/src/vectors_u32.rs b/crates/core_simd/src/vectors_u32.rs
index 09aed52e6bb5..d4e1fd439ff3 100644
--- a/crates/core_simd/src/vectors_u32.rs
+++ b/crates/core_simd/src/vectors_u32.rs
@@ -1,23 +1,23 @@
 define_vector! {
-    #[doc = "Vector of two `u32` values"]
+    /// Vector of two `u32` values
     #[derive(Eq, Ord, Hash)]
     struct u32x2([u32; 2]);
 }
 
 define_vector! {
-    #[doc = "Vector of four `u32` values"]
+    /// Vector of four `u32` values
     #[derive(Eq, Ord, Hash)]
     struct u32x4([u32; 4]);
 }
 
 define_vector! {
-    #[doc = "Vector of eight `u32` values"]
+    /// Vector of eight `u32` values
     #[derive(Eq, Ord, Hash)]
     struct u32x8([u32; 8]);
 }
 
 define_vector! {
-    #[doc = "Vector of 16 `u32` values"]
+    /// Vector of 16 `u32` values
     #[derive(Eq, Ord, Hash)]
     struct u32x16([u32; 16]);
 }
diff --git a/crates/core_simd/src/vectors_u64.rs b/crates/core_simd/src/vectors_u64.rs
index b19739e6657d..6a2b0ef76309 100644
--- a/crates/core_simd/src/vectors_u64.rs
+++ b/crates/core_simd/src/vectors_u64.rs
@@ -1,17 +1,17 @@
 define_vector! {
-    #[doc = "Vector of two `u64` values"]
+    /// Vector of two `u64` values
     #[derive(Eq, Ord, Hash)]
     struct u64x2([u64; 2]);
 }
 
 define_vector! {
-    #[doc = "Vector of four `u64` values"]
+    /// Vector of four `u64` values
     #[derive(Eq, Ord, Hash)]
     struct u64x4([u64; 4]);
 }
 
 define_vector! {
-    #[doc = "Vector of eight `u64` values"]
+    /// Vector of eight `u64` values
     #[derive(Eq, Ord, Hash)]
     struct u64x8([u64; 8]);
 }
diff --git a/crates/core_simd/src/vectors_u8.rs b/crates/core_simd/src/vectors_u8.rs
index 1c31578d0a7e..487a25e2d66e 100644
--- a/crates/core_simd/src/vectors_u8.rs
+++ b/crates/core_simd/src/vectors_u8.rs
@@ -1,35 +1,35 @@
 define_vector! {
-    #[doc = "Vector of two `u8` values"]
+    /// Vector of two `u8` values
     #[derive(Eq, Ord, Hash)]
     struct u8x2([u8; 2]);
 }
 
 define_vector! {
-    #[doc = "Vector of four `u8` values"]
+    /// Vector of four `u8` values
     #[derive(Eq, Ord, Hash)]
     struct u8x4([u8; 4]);
 }
 
 define_vector! {
-    #[doc = "Vector of eight `u8` values"]
+    /// Vector of eight `u8` values
     #[derive(Eq, Ord, Hash)]
     struct u8x8([u8; 8]);
 }
 
 define_vector! {
-    #[doc = "Vector of 16 `u8` values"]
+    /// Vector of 16 `u8` values
     #[derive(Eq, Ord, Hash)]
     struct u8x16([u8; 16]);
 }
 
 define_vector! {
-    #[doc = "Vector of 32 `u8` values"]
+    /// Vector of 32 `u8` values
     #[derive(Eq, Ord, Hash)]
     struct u8x32([u8; 32]);
 }
 
 define_vector! {
-    #[doc = "Vector of 64 `u8` values"]
+    /// Vector of 64 `u8` values
     #[derive(Eq, Ord, Hash)]
     struct u8x64([u8; 64]);
 }
diff --git a/crates/core_simd/src/vectors_usize.rs b/crates/core_simd/src/vectors_usize.rs
index 30c4da8438b4..2318d63a0014 100644
--- a/crates/core_simd/src/vectors_usize.rs
+++ b/crates/core_simd/src/vectors_usize.rs
@@ -1,17 +1,17 @@
 define_vector! {
-    #[doc = "Vector of two `usize` values"]
+    /// Vector of two `usize` values
     #[derive(Eq, Ord, Hash)]
     struct usizex2([usize; 2]);
 }
 
 define_vector! {
-    #[doc = "Vector of four `usize` values"]
+    /// Vector of four `usize` values
     #[derive(Eq, Ord, Hash)]
     struct usizex4([usize; 4]);
 }
 
 define_vector! {
-    #[doc = "Vector of eight `usize` values"]
+    /// Vector of eight `usize` values
     #[derive(Eq, Ord, Hash)]
     struct usizex8([usize; 8]);
 }

From d817b56f1d7d485e7a2e466730adf2a618e3b077 Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Sat, 26 Sep 2020 14:12:30 -0400
Subject: [PATCH 016/249] Manually implement some traits, instead of derive

---
 crates/core_simd/src/macros.rs | 29 +++++++++++++++++++++++++++--
 1 file changed, 27 insertions(+), 2 deletions(-)

diff --git a/crates/core_simd/src/macros.rs b/crates/core_simd/src/macros.rs
index 591a85af08f1..ecf2b76ceb0d 100644
--- a/crates/core_simd/src/macros.rs
+++ b/crates/core_simd/src/macros.rs
@@ -138,6 +138,32 @@ macro_rules! call_counting_args {
 /// Implements common traits on the specified vector `$name`, holding multiple `$lanes` of `$type`.
 macro_rules! base_vector_traits {
     { $name:path => [$type:ty; $lanes:literal] } => {
+        impl Copy for $name {}
+
+        impl Clone for $name {
+            fn clone(&self) -> Self {
+                *self
+            }
+        }
+
+        impl Default for $name {
+            fn default() -> Self {
+                Self::splat(<$type>::default())
+            }
+        }
+
+        impl PartialEq for $name {
+            fn eq(&self, other: &Self) -> bool {
+                AsRef::<[$type]>::as_ref(self) == AsRef::<[$type]>::as_ref(other)
+            }
+        }
+
+        impl PartialOrd for $name {
+            fn partial_cmp(&self, other: &Self) -> Option<core::cmp::Ordering> {
+                AsRef::<[$type]>::as_ref(self).partial_cmp(AsRef::<[$type]>::as_ref(other))
+            }
+        }
+
         // array references
         impl AsRef<[$type; $lanes]> for $name {
             #[inline]
@@ -196,7 +222,6 @@ macro_rules! define_vector {
     { def $(#[$attr:meta])* | $name:ident | $($itype:ty)* } => {
         $(#[$attr])*
         #[allow(non_camel_case_types)]
-        #[derive(Copy, Clone, Default, PartialEq, PartialOrd)]
         #[repr(simd)]
         pub struct $name($($itype),*);
     };
@@ -233,7 +258,7 @@ macro_rules! define_mask_vector {
     { def $(#[$attr:meta])* | $name:ident | $($itype:ty)* } => {
         $(#[$attr])*
         #[allow(non_camel_case_types)]
-        #[derive(Copy, Clone, Default, PartialEq, PartialOrd, Eq, Ord)]
+        #[derive(Eq, Ord)]
         #[repr(simd)]
         pub struct $name($($itype),*);
     };

From b9bf9ef3c2dab2d38f964f13154e517515e8e2f3 Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Sat, 26 Sep 2020 14:30:00 -0400
Subject: [PATCH 017/249] Remove remaining derived traits

---
 crates/core_simd/src/macros.rs        | 36 ++++++++++++++++++++++++++-
 crates/core_simd/src/vectors_i128.rs  |  6 ++---
 crates/core_simd/src/vectors_i16.rs   | 15 ++++-------
 crates/core_simd/src/vectors_i32.rs   | 12 +++------
 crates/core_simd/src/vectors_i64.rs   |  9 +++----
 crates/core_simd/src/vectors_i8.rs    | 18 +++++---------
 crates/core_simd/src/vectors_isize.rs |  9 +++----
 crates/core_simd/src/vectors_u128.rs  |  6 ++---
 crates/core_simd/src/vectors_u16.rs   | 15 ++++-------
 crates/core_simd/src/vectors_u32.rs   | 12 +++------
 crates/core_simd/src/vectors_u64.rs   |  9 +++----
 crates/core_simd/src/vectors_u8.rs    | 18 +++++---------
 crates/core_simd/src/vectors_usize.rs |  9 +++----
 13 files changed, 81 insertions(+), 93 deletions(-)

diff --git a/crates/core_simd/src/macros.rs b/crates/core_simd/src/macros.rs
index ecf2b76ceb0d..165029cca5e9 100644
--- a/crates/core_simd/src/macros.rs
+++ b/crates/core_simd/src/macros.rs
@@ -207,6 +207,28 @@ macro_rules! base_vector_traits {
     }
 }
 
+/// Implements additional integer traits (Eq, Ord, Hash) on the specified vector `$name`, holding multiple `$lanes` of `$type`.
+macro_rules! integer_vector_traits {
+    { $name:path => [$type:ty; $lanes:literal] } => {
+        impl Eq for $name {}
+
+        impl Ord for $name {
+            fn cmp(&self, other: &Self) -> core::cmp::Ordering {
+                AsRef::<[$type]>::as_ref(self).cmp(AsRef::<[$type]>::as_ref(other))
+            }
+        }
+
+        impl core::hash::Hash for $name {
+            fn hash<H>(&self, state: &mut H)
+            where
+                H: core::hash::Hasher
+            {
+                AsRef::<[$type]>::as_ref(self).hash(state)
+            }
+        }
+    }
+}
+
 /// Defines a vector `$name` containing multiple `$lanes` of `$type`.
 macro_rules! define_vector {
     { $(#[$attr:meta])* struct $name:ident([$type:ty; $lanes:tt]); } => {
@@ -242,6 +264,18 @@ macro_rules! define_vector {
     }
 }
 
+/// Defines an integer vector `$name` containing multiple `$lanes` of integer `$type`.
+macro_rules! define_integer_vector {
+    { $(#[$attr:meta])* struct $name:ident([$type:ty; $lanes:tt]); } => {
+        define_vector! {
+            $(#[$attr])*
+            struct $name([$type; $lanes]);
+        }
+
+        integer_vector_traits! { $name => [$type; $lanes] }
+    }
+}
+
 /// Defines a mask vector `$name` containing multiple `$lanes` of `$type`, represented by the
 /// underlying type `$impl_type`.
 macro_rules! define_mask_vector {
@@ -254,11 +288,11 @@ macro_rules! define_mask_vector {
         }
 
         base_vector_traits! { $name => [$type; $lanes] }
+        integer_vector_traits! { $name => [$type; $lanes] }
     };
     { def $(#[$attr:meta])* | $name:ident | $($itype:ty)* } => {
         $(#[$attr])*
         #[allow(non_camel_case_types)]
-        #[derive(Eq, Ord)]
         #[repr(simd)]
         pub struct $name($($itype),*);
     };
diff --git a/crates/core_simd/src/vectors_i128.rs b/crates/core_simd/src/vectors_i128.rs
index 588f4cffb909..5c8354070e81 100644
--- a/crates/core_simd/src/vectors_i128.rs
+++ b/crates/core_simd/src/vectors_i128.rs
@@ -1,12 +1,10 @@
-define_vector! {
+define_integer_vector! {
     /// Vector of two `i128` values
-    #[derive(Eq, Ord, Hash)]
     struct i128x2([i128; 2]);
 }
 
-define_vector! {
+define_integer_vector! {
     /// Vector of four `i128` values
-    #[derive(Eq, Ord, Hash)]
     struct i128x4([i128; 4]);
 }
 
diff --git a/crates/core_simd/src/vectors_i16.rs b/crates/core_simd/src/vectors_i16.rs
index 52d9ef8379b0..011820e19c71 100644
--- a/crates/core_simd/src/vectors_i16.rs
+++ b/crates/core_simd/src/vectors_i16.rs
@@ -1,30 +1,25 @@
-define_vector! {
+define_integer_vector! {
     /// Vector of two `i16` values
-    #[derive(Eq, Ord, Hash)]
     struct i16x2([i16; 2]);
 }
 
-define_vector! {
+define_integer_vector! {
     /// Vector of four `i16` values
-    #[derive(Eq, Ord, Hash)]
     struct i16x4([i16; 4]);
 }
 
-define_vector! {
+define_integer_vector! {
     /// Vector of eight `i16` values
-    #[derive(Eq, Ord, Hash)]
     struct i16x8([i16; 8]);
 }
 
-define_vector! {
+define_integer_vector! {
     /// Vector of 16 `i16` values
-    #[derive(Eq, Ord, Hash)]
     struct i16x16([i16; 16]);
 }
 
-define_vector! {
+define_integer_vector! {
     /// Vector of 32 `i16` values
-    #[derive(Eq, Ord, Hash)]
     struct i16x32([i16; 32]);
 }
 
diff --git a/crates/core_simd/src/vectors_i32.rs b/crates/core_simd/src/vectors_i32.rs
index 34ecc1694604..9aa9bc8e9dc8 100644
--- a/crates/core_simd/src/vectors_i32.rs
+++ b/crates/core_simd/src/vectors_i32.rs
@@ -1,24 +1,20 @@
-define_vector! {
+define_integer_vector! {
     /// Vector of two `i32` values
-    #[derive(Eq, Ord, Hash)]
     struct i32x2([i32; 2]);
 }
 
-define_vector! {
+define_integer_vector! {
     /// Vector of four `i32` values
-    #[derive(Eq, Ord, Hash)]
     struct i32x4([i32; 4]);
 }
 
-define_vector! {
+define_integer_vector! {
     /// Vector of eight `i32` values
-    #[derive(Eq, Ord, Hash)]
     struct i32x8([i32; 8]);
 }
 
-define_vector! {
+define_integer_vector! {
     /// Vector of 16 `i32` values
-    #[derive(Eq, Ord, Hash)]
     struct i32x16([i32; 16]);
 }
 
diff --git a/crates/core_simd/src/vectors_i64.rs b/crates/core_simd/src/vectors_i64.rs
index 58893c0dfedc..ba66aba2095d 100644
--- a/crates/core_simd/src/vectors_i64.rs
+++ b/crates/core_simd/src/vectors_i64.rs
@@ -1,18 +1,15 @@
-define_vector! {
+define_integer_vector! {
     /// Vector of two `i64` values
-    #[derive(Eq, Ord, Hash)]
     struct i64x2([i64; 2]);
 }
 
-define_vector! {
+define_integer_vector! {
     /// Vector of four `i64` values
-    #[derive(Eq, Ord, Hash)]
     struct i64x4([i64; 4]);
 }
 
-define_vector! {
+define_integer_vector! {
     /// Vector of eight `i64` values
-    #[derive(Eq, Ord, Hash)]
     struct i64x8([i64; 8]);
 }
 
diff --git a/crates/core_simd/src/vectors_i8.rs b/crates/core_simd/src/vectors_i8.rs
index c0551fc285e4..fb739bc44fa8 100644
--- a/crates/core_simd/src/vectors_i8.rs
+++ b/crates/core_simd/src/vectors_i8.rs
@@ -1,36 +1,30 @@
-define_vector! {
+define_integer_vector! {
     /// Vector of two `i8` values
-    #[derive(Eq, Ord, Hash)]
     struct i8x2([i8; 2]);
 }
 
-define_vector! {
+define_integer_vector! {
     /// Vector of four `i8` values
-    #[derive(Eq, Ord, Hash)]
     struct i8x4([i8; 4]);
 }
 
-define_vector! {
+define_integer_vector! {
     /// Vector of eight `i8` values
-    #[derive(Eq, Ord, Hash)]
     struct i8x8([i8; 8]);
 }
 
-define_vector! {
+define_integer_vector! {
     /// Vector of 16 `i8` values
-    #[derive(Eq, Ord, Hash)]
     struct i8x16([i8; 16]);
 }
 
-define_vector! {
+define_integer_vector! {
     /// Vector of 32 `i8` values
-    #[derive(Eq, Ord, Hash)]
     struct i8x32([i8; 32]);
 }
 
-define_vector! {
+define_integer_vector! {
     /// Vector of 64 `i8` values
-    #[derive(Eq, Ord, Hash)]
     struct i8x64([i8; 64]);
 }
 
diff --git a/crates/core_simd/src/vectors_isize.rs b/crates/core_simd/src/vectors_isize.rs
index 10a892bd0908..35dac8bcbd45 100644
--- a/crates/core_simd/src/vectors_isize.rs
+++ b/crates/core_simd/src/vectors_isize.rs
@@ -1,18 +1,15 @@
-define_vector! {
+define_integer_vector! {
     /// Vector of two `isize` values
-    #[derive(Eq, Ord, Hash)]
     struct isizex2([isize; 2]);
 }
 
-define_vector! {
+define_integer_vector! {
     /// Vector of four `isize` values
-    #[derive(Eq, Ord, Hash)]
     struct isizex4([isize; 4]);
 }
 
-define_vector! {
+define_integer_vector! {
     /// Vector of eight `isize` values
-    #[derive(Eq, Ord, Hash)]
     struct isizex8([isize; 8]);
 }
 
diff --git a/crates/core_simd/src/vectors_u128.rs b/crates/core_simd/src/vectors_u128.rs
index 1412dfdc85dc..eec7bde1722f 100644
--- a/crates/core_simd/src/vectors_u128.rs
+++ b/crates/core_simd/src/vectors_u128.rs
@@ -1,12 +1,10 @@
-define_vector! {
+define_integer_vector! {
     /// Vector of two `u128` values
-    #[derive(Eq, Ord, Hash)]
     struct u128x2([u128; 2]);
 }
 
-define_vector! {
+define_integer_vector! {
     /// Vector of four `u128` values
-    #[derive(Eq, Ord, Hash)]
     struct u128x4([u128; 4]);
 }
 
diff --git a/crates/core_simd/src/vectors_u16.rs b/crates/core_simd/src/vectors_u16.rs
index 6fd7c64f4b31..9e846a32efad 100644
--- a/crates/core_simd/src/vectors_u16.rs
+++ b/crates/core_simd/src/vectors_u16.rs
@@ -1,30 +1,25 @@
-define_vector! {
+define_integer_vector! {
     /// Vector of two `u16` values
-    #[derive(Eq, Ord, Hash)]
     struct u16x2([u16; 2]);
 }
 
-define_vector! {
+define_integer_vector! {
     /// Vector of four `u16` values
-    #[derive(Eq, Ord, Hash)]
     struct u16x4([u16; 4]);
 }
 
-define_vector! {
+define_integer_vector! {
     /// Vector of eight `u16` values
-    #[derive(Eq, Ord, Hash)]
     struct u16x8([u16; 8]);
 }
 
-define_vector! {
+define_integer_vector! {
     /// Vector of 16 `u16` values
-    #[derive(Eq, Ord, Hash)]
     struct u16x16([u16; 16]);
 }
 
-define_vector! {
+define_integer_vector! {
     /// Vector of 32 `u16` values
-    #[derive(Eq, Ord, Hash)]
     struct u16x32([u16; 32]);
 }
 
diff --git a/crates/core_simd/src/vectors_u32.rs b/crates/core_simd/src/vectors_u32.rs
index d4e1fd439ff3..b00c63d9058f 100644
--- a/crates/core_simd/src/vectors_u32.rs
+++ b/crates/core_simd/src/vectors_u32.rs
@@ -1,24 +1,20 @@
-define_vector! {
+define_integer_vector! {
     /// Vector of two `u32` values
-    #[derive(Eq, Ord, Hash)]
     struct u32x2([u32; 2]);
 }
 
-define_vector! {
+define_integer_vector! {
     /// Vector of four `u32` values
-    #[derive(Eq, Ord, Hash)]
     struct u32x4([u32; 4]);
 }
 
-define_vector! {
+define_integer_vector! {
     /// Vector of eight `u32` values
-    #[derive(Eq, Ord, Hash)]
     struct u32x8([u32; 8]);
 }
 
-define_vector! {
+define_integer_vector! {
     /// Vector of 16 `u32` values
-    #[derive(Eq, Ord, Hash)]
     struct u32x16([u32; 16]);
 }
 
diff --git a/crates/core_simd/src/vectors_u64.rs b/crates/core_simd/src/vectors_u64.rs
index 6a2b0ef76309..0bcf28ebc265 100644
--- a/crates/core_simd/src/vectors_u64.rs
+++ b/crates/core_simd/src/vectors_u64.rs
@@ -1,18 +1,15 @@
-define_vector! {
+define_integer_vector! {
     /// Vector of two `u64` values
-    #[derive(Eq, Ord, Hash)]
     struct u64x2([u64; 2]);
 }
 
-define_vector! {
+define_integer_vector! {
     /// Vector of four `u64` values
-    #[derive(Eq, Ord, Hash)]
     struct u64x4([u64; 4]);
 }
 
-define_vector! {
+define_integer_vector! {
     /// Vector of eight `u64` values
-    #[derive(Eq, Ord, Hash)]
     struct u64x8([u64; 8]);
 }
 
diff --git a/crates/core_simd/src/vectors_u8.rs b/crates/core_simd/src/vectors_u8.rs
index 487a25e2d66e..d70de1a24c9d 100644
--- a/crates/core_simd/src/vectors_u8.rs
+++ b/crates/core_simd/src/vectors_u8.rs
@@ -1,36 +1,30 @@
-define_vector! {
+define_integer_vector! {
     /// Vector of two `u8` values
-    #[derive(Eq, Ord, Hash)]
     struct u8x2([u8; 2]);
 }
 
-define_vector! {
+define_integer_vector! {
     /// Vector of four `u8` values
-    #[derive(Eq, Ord, Hash)]
     struct u8x4([u8; 4]);
 }
 
-define_vector! {
+define_integer_vector! {
     /// Vector of eight `u8` values
-    #[derive(Eq, Ord, Hash)]
     struct u8x8([u8; 8]);
 }
 
-define_vector! {
+define_integer_vector! {
     /// Vector of 16 `u8` values
-    #[derive(Eq, Ord, Hash)]
     struct u8x16([u8; 16]);
 }
 
-define_vector! {
+define_integer_vector! {
     /// Vector of 32 `u8` values
-    #[derive(Eq, Ord, Hash)]
     struct u8x32([u8; 32]);
 }
 
-define_vector! {
+define_integer_vector! {
     /// Vector of 64 `u8` values
-    #[derive(Eq, Ord, Hash)]
     struct u8x64([u8; 64]);
 }
 
diff --git a/crates/core_simd/src/vectors_usize.rs b/crates/core_simd/src/vectors_usize.rs
index 2318d63a0014..84a4b8e509b3 100644
--- a/crates/core_simd/src/vectors_usize.rs
+++ b/crates/core_simd/src/vectors_usize.rs
@@ -1,18 +1,15 @@
-define_vector! {
+define_integer_vector! {
     /// Vector of two `usize` values
-    #[derive(Eq, Ord, Hash)]
     struct usizex2([usize; 2]);
 }
 
-define_vector! {
+define_integer_vector! {
     /// Vector of four `usize` values
-    #[derive(Eq, Ord, Hash)]
     struct usizex4([usize; 4]);
 }
 
-define_vector! {
+define_integer_vector! {
     /// Vector of eight `usize` values
-    #[derive(Eq, Ord, Hash)]
     struct usizex8([usize; 8]);
 }
 

From 88bfbb001e291ba97b3b6d6cea3456bbd4fbf640 Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Sat, 26 Sep 2020 14:41:01 -0400
Subject: [PATCH 018/249] Mark trait methods inline

---
 crates/core_simd/src/macros.rs | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/crates/core_simd/src/macros.rs b/crates/core_simd/src/macros.rs
index 165029cca5e9..2f93db190352 100644
--- a/crates/core_simd/src/macros.rs
+++ b/crates/core_simd/src/macros.rs
@@ -141,24 +141,28 @@ macro_rules! base_vector_traits {
         impl Copy for $name {}
 
         impl Clone for $name {
+            #[inline]
             fn clone(&self) -> Self {
                 *self
             }
         }
 
         impl Default for $name {
+            #[inline]
             fn default() -> Self {
                 Self::splat(<$type>::default())
             }
         }
 
         impl PartialEq for $name {
+            #[inline]
             fn eq(&self, other: &Self) -> bool {
                 AsRef::<[$type]>::as_ref(self) == AsRef::<[$type]>::as_ref(other)
             }
         }
 
         impl PartialOrd for $name {
+            #[inline]
             fn partial_cmp(&self, other: &Self) -> Option<core::cmp::Ordering> {
                 AsRef::<[$type]>::as_ref(self).partial_cmp(AsRef::<[$type]>::as_ref(other))
             }
@@ -213,12 +217,14 @@ macro_rules! integer_vector_traits {
         impl Eq for $name {}
 
         impl Ord for $name {
+            #[inline]
             fn cmp(&self, other: &Self) -> core::cmp::Ordering {
                 AsRef::<[$type]>::as_ref(self).cmp(AsRef::<[$type]>::as_ref(other))
             }
         }
 
         impl core::hash::Hash for $name {
+            #[inline]
             fn hash<H>(&self, state: &mut H)
             where
                 H: core::hash::Hasher

From cfda50a82a7eb1c04c8ed18600c69e587af3556e Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Sat, 26 Sep 2020 19:31:14 -0400
Subject: [PATCH 019/249] Remove vectors under 64-bit width

---
 crates/core_simd/src/fmt.rs            | 12 ++++++------
 crates/core_simd/src/vectors_i16.rs    |  5 -----
 crates/core_simd/src/vectors_i8.rs     | 10 ----------
 crates/core_simd/src/vectors_mask16.rs |  5 -----
 crates/core_simd/src/vectors_mask8.rs  | 10 ----------
 crates/core_simd/src/vectors_u16.rs    |  5 -----
 crates/core_simd/src/vectors_u8.rs     | 10 ----------
 7 files changed, 6 insertions(+), 51 deletions(-)

diff --git a/crates/core_simd/src/fmt.rs b/crates/core_simd/src/fmt.rs
index e505947e1e65..c634e0546bc1 100644
--- a/crates/core_simd/src/fmt.rs
+++ b/crates/core_simd/src/fmt.rs
@@ -74,10 +74,10 @@ macro_rules! impl_fmt_trait {
 
 impl_fmt_trait! {
     integers:
-        crate::u8x2,    crate::u8x4,    crate::u8x8,    crate::u8x16,    crate::u8x32,    crate::u8x64,
-        crate::i8x2,    crate::i8x4,    crate::i8x8,    crate::i8x16,    crate::i8x32,    crate::i8x64,
-        crate::u16x2,   crate::u16x4,   crate::u16x8,   crate::u16x16,   crate::u16x32,
-        crate::i16x2,   crate::i16x4,   crate::i16x8,   crate::i16x16,   crate::i16x32,
+        crate::u8x8,    crate::u8x16,    crate::u8x32,    crate::u8x64,
+        crate::i8x8,    crate::i8x16,    crate::i8x32,    crate::i8x64,
+        crate::u16x4,   crate::u16x8,   crate::u16x16,   crate::u16x32,
+        crate::i16x4,   crate::i16x8,   crate::i16x16,   crate::i16x32,
         crate::u32x2,   crate::u32x4,   crate::u32x8,   crate::u32x16,
         crate::i32x2,   crate::i32x4,   crate::i32x8,   crate::i32x16,
         crate::u64x2,   crate::u64x4,   crate::u64x8,
@@ -96,8 +96,8 @@ impl_fmt_trait! {
 
 impl_fmt_trait! {
     masks:
-        crate::mask8x2,    crate::mask8x4,    crate::mask8x8,    crate::mask8x16,    crate::mask8x32,    crate::mask8x64,
-        crate::mask16x2,   crate::mask16x4,   crate::mask16x8,   crate::mask16x16,   crate::mask16x32,
+        crate::mask8x8,    crate::mask8x16,    crate::mask8x32,    crate::mask8x64,
+        crate::mask16x4,   crate::mask16x8,   crate::mask16x16,   crate::mask16x32,
         crate::mask32x2,   crate::mask32x4,   crate::mask32x8,   crate::mask32x16,
         crate::mask64x2,   crate::mask64x4,   crate::mask64x8,
         crate::mask128x2,  crate::mask128x4,
diff --git a/crates/core_simd/src/vectors_i16.rs b/crates/core_simd/src/vectors_i16.rs
index 011820e19c71..8aabd136b104 100644
--- a/crates/core_simd/src/vectors_i16.rs
+++ b/crates/core_simd/src/vectors_i16.rs
@@ -1,8 +1,3 @@
-define_integer_vector! {
-    /// Vector of two `i16` values
-    struct i16x2([i16; 2]);
-}
-
 define_integer_vector! {
     /// Vector of four `i16` values
     struct i16x4([i16; 4]);
diff --git a/crates/core_simd/src/vectors_i8.rs b/crates/core_simd/src/vectors_i8.rs
index fb739bc44fa8..3e52d894cc22 100644
--- a/crates/core_simd/src/vectors_i8.rs
+++ b/crates/core_simd/src/vectors_i8.rs
@@ -1,13 +1,3 @@
-define_integer_vector! {
-    /// Vector of two `i8` values
-    struct i8x2([i8; 2]);
-}
-
-define_integer_vector! {
-    /// Vector of four `i8` values
-    struct i8x4([i8; 4]);
-}
-
 define_integer_vector! {
     /// Vector of eight `i8` values
     struct i8x8([i8; 8]);
diff --git a/crates/core_simd/src/vectors_mask16.rs b/crates/core_simd/src/vectors_mask16.rs
index 8cbae6b7bbd2..406d7255a11e 100644
--- a/crates/core_simd/src/vectors_mask16.rs
+++ b/crates/core_simd/src/vectors_mask16.rs
@@ -1,10 +1,5 @@
 use crate::mask16;
 
-define_mask_vector! {
-    /// Vector of two `mask16` values
-    struct mask16x2([i16 as mask16; 2]);
-}
-
 define_mask_vector! {
     /// Vector of four `mask16` values
     struct mask16x4([i16 as mask16; 4]);
diff --git a/crates/core_simd/src/vectors_mask8.rs b/crates/core_simd/src/vectors_mask8.rs
index b6fbe5595189..d038b3361044 100644
--- a/crates/core_simd/src/vectors_mask8.rs
+++ b/crates/core_simd/src/vectors_mask8.rs
@@ -1,15 +1,5 @@
 use crate::mask8;
 
-define_mask_vector! {
-    /// Vector of two `mask8` values
-    struct mask8x2([i8 as mask8; 2]);
-}
-
-define_mask_vector! {
-    /// Vector of four `mask8` values
-    struct mask8x4([i8 as mask8; 4]);
-}
-
 define_mask_vector! {
     /// Vector of eight `mask8` values
     struct mask8x8([i8 as mask8; 8]);
diff --git a/crates/core_simd/src/vectors_u16.rs b/crates/core_simd/src/vectors_u16.rs
index 9e846a32efad..809ab10383cd 100644
--- a/crates/core_simd/src/vectors_u16.rs
+++ b/crates/core_simd/src/vectors_u16.rs
@@ -1,8 +1,3 @@
-define_integer_vector! {
-    /// Vector of two `u16` values
-    struct u16x2([u16; 2]);
-}
-
 define_integer_vector! {
     /// Vector of four `u16` values
     struct u16x4([u16; 4]);
diff --git a/crates/core_simd/src/vectors_u8.rs b/crates/core_simd/src/vectors_u8.rs
index d70de1a24c9d..a187bc6f7b42 100644
--- a/crates/core_simd/src/vectors_u8.rs
+++ b/crates/core_simd/src/vectors_u8.rs
@@ -1,13 +1,3 @@
-define_integer_vector! {
-    /// Vector of two `u8` values
-    struct u8x2([u8; 2]);
-}
-
-define_integer_vector! {
-    /// Vector of four `u8` values
-    struct u8x4([u8; 4]);
-}
-
 define_integer_vector! {
     /// Vector of eight `u8` values
     struct u8x8([u8; 8]);

From 9f4fd1e68febc0f1cbaf2bd6772bd33658bc195b Mon Sep 17 00:00:00 2001
From: Lokathor <zefria@gmail.com>
Date: Sat, 26 Sep 2020 20:57:32 -0600
Subject: [PATCH 020/249] Update README.md

---
 README.md | 17 ++++++++++++++++-
 1 file changed, 16 insertions(+), 1 deletion(-)

diff --git a/README.md b/README.md
index a4efc29293d6..5fe074093f25 100644
--- a/README.md
+++ b/README.md
@@ -2,4 +2,19 @@
 
 Code repository for the [Portable SIMD Project Group](https://github.com/rust-lang/project-portable-simd).
 
-More coming soon!
+## Code Organization
+
+Currently the crate is organized so that each element type is a file, and then the 64-bit, 128-bit, 256-bit, and 512-bit vectors using those types are contained in said file.
+
+All types are then exported as a single, flat
+
+Depending on the size of the primitive type, the number of lanes the vector will have varies. For example, 128-bit vectors have four `f32` lanes and two `f64` lanes.
+
+The supported element types are as follows:
+* **Floating Point:** `f32`, `f64`
+* **Signed Integers:** `i8`, `i16`, `i32`, `i64`, `i128`, `isize`
+* **Unsigned Integers:** `u8`, `u16`, `u32`, `u64`, `u128`, `usize`
+* **Masks:** `mask8`, `mask16`, `mask32`, `mask64`, `masksize`
+
+Floating point, signed integers, and unsigned integers are the [primitive types](https://doc.rust-lang.org/core/primitive/index.html) you're already used to.
+The `mask` types are "truthy" values, but they use the number of bits in their name instead of just 1 bit like a normal `bool` uses.

From 639e2d2cff6b311c39a9befa9cc79017350dc77d Mon Sep 17 00:00:00 2001
From: Lokathor <zefria@gmail.com>
Date: Sat, 26 Sep 2020 20:58:34 -0600
Subject: [PATCH 021/249] missing word.

---
 README.md | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/README.md b/README.md
index 5fe074093f25..5c841a27163a 100644
--- a/README.md
+++ b/README.md
@@ -6,7 +6,7 @@ Code repository for the [Portable SIMD Project Group](https://github.com/rust-la
 
 Currently the crate is organized so that each element type is a file, and then the 64-bit, 128-bit, 256-bit, and 512-bit vectors using those types are contained in said file.
 
-All types are then exported as a single, flat
+All types are then exported as a single, flat module.
 
 Depending on the size of the primitive type, the number of lanes the vector will have varies. For example, 128-bit vectors have four `f32` lanes and two `f64` lanes.
 

From 272c9461fc2c5b2a2876324efa83d0106b463f6a Mon Sep 17 00:00:00 2001
From: Lokathor <zefria@gmail.com>
Date: Sat, 26 Sep 2020 21:20:34 -0600
Subject: [PATCH 022/249] missed a type

---
 README.md | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/README.md b/README.md
index 5c841a27163a..976ec644e415 100644
--- a/README.md
+++ b/README.md
@@ -14,7 +14,7 @@ The supported element types are as follows:
 * **Floating Point:** `f32`, `f64`
 * **Signed Integers:** `i8`, `i16`, `i32`, `i64`, `i128`, `isize`
 * **Unsigned Integers:** `u8`, `u16`, `u32`, `u64`, `u128`, `usize`
-* **Masks:** `mask8`, `mask16`, `mask32`, `mask64`, `masksize`
+* **Masks:** `mask8`, `mask16`, `mask32`, `mask64`, `mask128`, `masksize`
 
 Floating point, signed integers, and unsigned integers are the [primitive types](https://doc.rust-lang.org/core/primitive/index.html) you're already used to.
 The `mask` types are "truthy" values, but they use the number of bits in their name instead of just 1 bit like a normal `bool` uses.

From 970307035a33c9b72da08403798e5259c1d5f325 Mon Sep 17 00:00:00 2001
From: Lokathor <zefria@gmail.com>
Date: Sat, 26 Sep 2020 23:37:52 -0600
Subject: [PATCH 023/249] end of draft 1

---
 beginners-guide.md | 37 +++++++++++++++++++++++++++++++++++++
 1 file changed, 37 insertions(+)

diff --git a/beginners-guide.md b/beginners-guide.md
index d5010e9847ef..bf0592236d05 100644
--- a/beginners-guide.md
+++ b/beginners-guide.md
@@ -26,3 +26,40 @@ SIMD has a few special vocabulary terms you should know:
 * **Vertical:** When an operation is "vertical", each lane processes individually without regard to the other lanes in the same vector. For example, a "vertical add" between two vectors would add lane 0 in `a` with lane 0 in `b`, with the total in lane 0 of `out`, and then the same thing for lanes 1, 2, etc. Most SIMD operations are vertical operations, so if your problem is a vertical problem then you can probably solve it with SIMD.
 
 * **Horizontal:** When an operation is "horizontal", the lanes within a single vector interact in some way. A "horizontal add" might add up lane 0 of `a` with lane 1 of `a`, with the total in lane 0 of `out`.
+
+* **Target Feature:** Rust calls a CPU architecture extension a `target_feature`. Proper SIMD requires various CPU extensions to be enabled (details below). Don't confuse this with `feature`, which is a Cargo crate concept.
+
+## Target Features
+
+When using SIMD, you should be familiar with the CPU feature set that you're targeting.
+
+On `arm` and `aarch64` it's fairly simple. There's just one CPU feature that controls if SIMD is available: `neon` (or "NEON", all caps, as the ARM docs often put it). Neon registers are 128-bit, but they can also operate as 64-bit (the high lanes are just zeroed out).
+
+> By default, the `aarch64`, `arm`, and `thumb` Rust targets generally do not enable `neon` unless it's in the target string.
+
+On `x86` and `x86_64` it's slightly more complicated. The SIMD support is split into many levels:
+* 128-bit: `sse`, `sse2`, `sse3`, `ssse3` (not a typo!), `sse4.1`, `sse4.2`, `sse4a` (AMD only)
+* 256-bit (mostly): `avx`, `avx2`, `fma`
+* 512-bit (mostly): a *wide* range of `avx512` variations
+
+> By default, the `i686` and `x86_64` Rust targets enable `sse` and `sse2`.
+
+### Selecting Additional Target Features
+
+If you want to enable support for a target feature within your build, generally you should use a [target-feature](https://rust-lang.github.io/packed_simd/perf-guide/target-feature/rustflags.html#target-feature) setting within you `RUSTFLAGS` setting.
+
+If you know that you're targeting a specific CPU you can instead use the [target-cpu](https://rust-lang.github.io/packed_simd/perf-guide/target-feature/rustflags.html#target-cpu) flag and the compiler will enable the correct set of features for that CPU.
+
+The [Steam Hardware Survey](https://store.steampowered.com/hwsurvey/Steam-Hardware-Software-Survey-Welcome-to-Steam) is one of the few places with data on how common various CPU features are. The dataset is limited to "the kinds of computers owned by people who play computer games", so the info only covers `x86`/`x86_64`, and it also probably skews to slightly higher quality computers than average. Still, we can see that the `sse` levels have very high support, `avx` and `avx2` are quite common as well, and the `avx-512` family is still so early in adoption you can barely find it in consumer grade stuff.
+
+## Running a program compiled for a CPU feature level that the CPU doesn't support is automatic undefined behavior.
+
+This means that if you build your program with `avx` support enabled and run it on a CPU without `avx` support, it's **instantly** undefined behavior.
+
+Even without an `unsafe` block in sight.
+
+This is no bug in Rust, or soundness hole in the type system. You just plain can't make a CPU do what it doesn't know how to do.
+
+This is why the various Rust targets *don't* enable many CPU feature flags by default: requiring a more advanced CPU makes the final binary *less* portable.
+
+So please select an appropriate CPU feature level when building your programs.

From 0ac2ee32cff3351b7cd6c6482247ecadcaff2a98 Mon Sep 17 00:00:00 2001
From: Lokathor <zefria@gmail.com>
Date: Sun, 27 Sep 2020 10:25:30 -0600
Subject: [PATCH 024/249] resolve
 https://github.com/rust-lang/stdsimd/pull/8#discussion_r495582339

---
 beginners-guide.md | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/beginners-guide.md b/beginners-guide.md
index bf0592236d05..f39cc9e7c3c3 100644
--- a/beginners-guide.md
+++ b/beginners-guide.md
@@ -7,7 +7,7 @@ Because SIMD is a subject that many programmers haven't worked with before, we t
 
 ## Quick Background
 
-**SIMD** stands for *Single Instruction, Multiple Data*. In other words, SIMD is when the CPU performs a single action on more that one logical pieces of data at the same time. Instead of adding two registers that each contain one `f32` value and getting an `f32` as the result, you might add two registers that each contain `f32x4` (128 bits of data) and then you get an `f32x4` as the output.
+**SIMD** stands for *Single Instruction, Multiple Data*. In other words, SIMD is when the CPU performs a single action on more than one logical piece of data at the same time. Instead of adding two registers that each contain one `f32` value and getting an `f32` as the result, you might add two registers that each contain `f32x4` (128 bits of data) and then you get an `f32x4` as the output.
 
 This might seem a tiny bit weird at first, but there's a good reason for it. Back in the day, as CPUs got faster and faster, eventually they got so fast that the CPU would just melt itself. The heat management (heat sinks, fans, etc) simply couldn't keep up with how much electricity was going through the metal. Two main strategies were developed to help get around the limits of physics.
 * One of them you're probably familiar with: Multi-core processors. By giving a processor more than one core, each core can do its own work, and because they're physically distant (at least on the CPU's scale) the heat can still be managed. Unfortunately, not all tasks can just be split up across cores in an efficient way.

From 656312e33c529ba78a483e1532a6c6ae044a2003 Mon Sep 17 00:00:00 2001
From: Lokathor <zefria@gmail.com>
Date: Sun, 27 Sep 2020 10:27:24 -0600
Subject: [PATCH 025/249] Resolve
 https://github.com/rust-lang/stdsimd/pull/8#discussion_r495583060

---
 beginners-guide.md | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/beginners-guide.md b/beginners-guide.md
index f39cc9e7c3c3..835f4e39298c 100644
--- a/beginners-guide.md
+++ b/beginners-guide.md
@@ -19,7 +19,7 @@ SIMD has a few special vocabulary terms you should know:
 
 * **Vector:** A SIMD value is called a vector. This shouldn't be confused with the `Vec<T>` type. A SIMD vector has a fixed size, known at compile time. All of the elements within the vector are of the same type. This makes vectors *similar to* arrays. One difference is that a vector is generally aligned to its *entire* size (eg: 16 bytes, 32 bytes, etc), not just the size of an individual element. Sometimes vector data is called "packed" data.
 
-* **Lane:** A single element position within a vector is called a lane. If you have `N` lanes available then they're numbered from `0` to `N-1` when referring to them, again like an array. The biggest difference between an array element and a vector lane is that it is *relatively costly* to access an individual lane value. Generally, the vector has to be pushed out of register onto the stack, then an individual lane is accessed while it's on the stack. For this reason, when working with SIMD you should avoid reading or writing the value of an individual lane during hot loops.
+* **Lane:** A single element position within a vector is called a lane. If you have `N` lanes available then they're numbered from `0` to `N-1` when referring to them, again like an array. The biggest difference between an array element and a vector lane is that in general is *relatively costly* to access an individual lane value. On most architectures, the vector has to be pushed out of the SIMD register onto the stack, then an individual lane is accessed while it's on the stack (and possibly the stack value is read back into a register). For this reason, when working with SIMD you should avoid reading or writing the value of an individual lane during hot loops.
 
 * **Bit Widths:** When talking about SIMD, the bit widths used are the bit size of the vectors involved, *not* the individual elements. So "128-bit SIMD" has 128-bit vectors, and that might be `f32x4`, `i32x4`, `i16x8`, or other variations. While 128-bit SIMD is the most common, there's also 64-bit, 256-bit, and even 512-bit on the newest CPUs.
 

From fdfbf7c68b7372474003fe3fe0abff0c22103bd8 Mon Sep 17 00:00:00 2001
From: Lokathor <zefria@gmail.com>
Date: Sun, 27 Sep 2020 10:55:49 -0600
Subject: [PATCH 026/249] resolve
 https://github.com/rust-lang/stdsimd/pull/8#discussion_r495584133

---
 beginners-guide.md | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/beginners-guide.md b/beginners-guide.md
index 835f4e39298c..a39243170fc5 100644
--- a/beginners-guide.md
+++ b/beginners-guide.md
@@ -33,7 +33,7 @@ SIMD has a few special vocabulary terms you should know:
 
 When using SIMD, you should be familiar with the CPU feature set that you're targeting.
 
-On `arm` and `aarch64` it's fairly simple. There's just one CPU feature that controls if SIMD is available: `neon` (or "NEON", all caps, as the ARM docs often put it). Neon registers are 128-bit, but they can also operate as 64-bit (the high lanes are just zeroed out).
+On `arm` and `aarch64` it's fairly simple. There's just one CPU feature that controls if SIMD is available: `neon` (or "NEON", all caps, as the ARM docs often put it). Neon registers can be used as 64-bit or 128-bit. When doing 128-bit operations it just uses two 64-bit registers as a single 128-bit register.
 
 > By default, the `aarch64`, `arm`, and `thumb` Rust targets generally do not enable `neon` unless it's in the target string.
 
@@ -42,6 +42,8 @@ On `x86` and `x86_64` it's slightly more complicated. The SIMD support is split
 * 256-bit (mostly): `avx`, `avx2`, `fma`
 * 512-bit (mostly): a *wide* range of `avx512` variations
 
+The list notes the bit widths available at each feature level, though the operations of the more advanced features can generally be used with the smaller register sizes as well. For example, new operations introduced in `avx` generally have a 128-bit form as well as a 256-bit form. This means that even if you only do 128-bit work you can still benefit from the later feature levels.
+
 > By default, the `i686` and `x86_64` Rust targets enable `sse` and `sse2`.
 
 ### Selecting Additional Target Features

From 965edaecdc1d552005150abf5d544dfeb0acff09 Mon Sep 17 00:00:00 2001
From: Ashley Mannix <=>
Date: Mon, 28 Sep 2020 19:52:28 +1000
Subject: [PATCH 027/249] add initial travis CI

---
 .travis.yml | 158 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 158 insertions(+)
 create mode 100644 .travis.yml

diff --git a/.travis.yml b/.travis.yml
new file mode 100644
index 000000000000..fff11d97f6f7
--- /dev/null
+++ b/.travis.yml
@@ -0,0 +1,158 @@
+language: rust
+rust:
+  - nightly
+
+matrix:
+  fast_finish: true
+  include:
+    # Linux (x86_64)
+    - name: "x86_64-unknown-linux-gnu"
+      os: linux
+      arch: amd64
+      env:
+        - TARGET=x86_64-unknown-linux-gnu
+    - name: "x86_64-unknown-linux-gnu+sse"
+      os: linux
+      arch: amd64
+      env:
+        - TARGET=x86_64-unknown-linux-gnu
+        - TARGET_FEATURE=sse
+    - name: "x86_64-unknown-linux-gnu+sse2"
+      os: linux
+      arch: amd64
+      env:
+        - TARGET=x86_64-unknown-linux-gnu
+        - TARGET_FEATURE=sse2
+    - name: "x86_64-unknown-linux-gnu+sse3"
+      os: linux
+      arch: amd64
+      env:
+        - TARGET=x86_64-unknown-linux-gnu
+        - TARGET_FEATURE=sse3
+    - name: "x86_64-unknown-linux-gnu+sse4.1"
+      os: linux
+      arch: amd64
+      env:
+        - TARGET=x86_64-unknown-linux-gnu
+        - TARGET_FEATURE=sse4.1
+    - name: "x86_64-unknown-linux-gnu+sse4.2"
+      os: linux
+      arch: amd64
+      env:
+        - TARGET=x86_64-unknown-linux-gnu
+        - TARGET_FEATURE=sse4.2
+    - name: "x86_64-unknown-linux-gnu+avx"
+      os: linux
+      arch: amd64
+      env:
+        - TARGET=x86_64-unknown-linux-gnu
+        - TARGET_FEATURE=avx
+    - name: "x86_64-unknown-linux-gnu+avx2"
+      os: linux
+      arch: amd64
+      env:
+        - TARGET=x86_64-unknown-linux-gnu
+        - TARGET_FEATURE=avx2
+    - name: "x86_64-unknown-linux-gnu+avx512vl"
+      os: linux
+      arch: amd64
+      env:
+        - TARGET=x86_64-unknown-linux-gnu
+        - TARGET_FEATURE=avx512vl
+
+    # Linux (aarch64)
+    - name: "aarch64-unknown-linux-gnu"
+      os: linux
+      arch: arm64
+      env:
+        - TARGET=aarch64-unknown-linux-gnu
+    - name: "aarch64-unknown-linux-gnu+neon"
+      os: linux
+      arch: arm64
+      env:
+        - TARGET=aarch64-unknown-linux-gnu
+        - TARGET_FEATURE=neon
+    - name: "aarch64-unknown-linux-gnu+sve"
+      os: linux
+      arch: arm64
+      env:
+        - TARGET=aarch64-unknown-linux-gnu
+        - TARGET_FEATURE=sve
+
+    # Linux (powerpc64)
+    - name: "powerpc64le-unknown-linux-gnu"
+      os: linux
+      arch: ppc64le
+      env:
+        - TARGET=powerpc64le-unknown-linux-gnu
+    - name: "powerpc64le-unknown-linux-gnu+vsx"
+      os: linux
+      arch: ppc64le
+      env:
+        - TARGET=powerpc64le-unknown-linux-gnu
+        - TARGET_FEATURE=vsx
+
+    # Windows (x86_64)
+    - name: "x86_64-pc-windows-msvc"
+      os: windows
+      arch: amd64
+      env: TARGET=x86_64-pc-windows-msvc
+    
+    # Windows (i686)
+    - name: "i686-pc-windows-msvc"
+      os: windows
+      env: TARGET=i686-pc-windows-msvc
+    - name: "i686-pc-windows-msvc+sse"
+      os: windows
+      arch: amd64
+      env:
+        - TARGET=i686-pc-windows-msvc
+        - TARGET_FEATURE=sse
+    - name: "i686-pc-windows-msvc+sse2"
+      os: windows
+      arch: amd64
+      env:
+        - TARGET=i686-pc-windows-msvc
+        - TARGET_FEATURE=sse2
+    - name: "i686-pc-windows-msvc+sse3"
+      os: windows
+      arch: amd64
+      env:
+        - TARGET=i686-pc-windows-msvc
+        - TARGET_FEATURE=sse3
+    - name: "i686-pc-windows-msvc+sse4.1"
+      os: windows
+      arch: amd64
+      env:
+        - TARGET=i686-pc-windows-msvc
+        - TARGET_FEATURE=sse4.1
+    - name: "i686-pc-windows-msvc+sse4.2"
+      os: windows
+      arch: amd64
+      env:
+        - TARGET=i686-pc-windows-msvc
+        - TARGET_FEATURE=sse4.2
+    - name: "i686-pc-windows-msvc+avx"
+      os: windows
+      arch: amd64
+      env:
+        - TARGET=i686-pc-windows-msvc
+        - TARGET_FEATURE=avx
+    - name: "i686-pc-windows-msvc+avx2"
+      os: windows
+      arch: amd64
+      env:
+        - TARGET=i686-pc-windows-msvc
+        - TARGET_FEATURE=avx2
+
+    # OSX (x86_64)
+    - name: "x86_64-apple-darwin"
+      os: osx
+      arch: amd64
+      env:
+        - TARGET=x86_64-apple-darwin
+
+script:
+  - rustup target add $TARGET
+  - if [ -n "$TARGET_FEATURE" ]; then RUSTFLAGS="-C target-feature=+$TARGET_FEATURE"; fi
+  - cargo test -v --target $TARGET

From ea8f511f7f919748b3559e00ba3c48634e6fdd19 Mon Sep 17 00:00:00 2001
From: Ashley Mannix <=>
Date: Tue, 29 Sep 2020 09:02:16 +1000
Subject: [PATCH 028/249] add i586 targets

---
 .travis.yml | 79 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 79 insertions(+)

diff --git a/.travis.yml b/.travis.yml
index fff11d97f6f7..0bd9760ab0ae 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -6,53 +6,62 @@ matrix:
   fast_finish: true
   include:
     # Linux (x86_64)
+
     - name: "x86_64-unknown-linux-gnu"
       os: linux
       arch: amd64
       env:
         - TARGET=x86_64-unknown-linux-gnu
+
     - name: "x86_64-unknown-linux-gnu+sse"
       os: linux
       arch: amd64
       env:
         - TARGET=x86_64-unknown-linux-gnu
         - TARGET_FEATURE=sse
+
     - name: "x86_64-unknown-linux-gnu+sse2"
       os: linux
       arch: amd64
       env:
         - TARGET=x86_64-unknown-linux-gnu
         - TARGET_FEATURE=sse2
+
     - name: "x86_64-unknown-linux-gnu+sse3"
       os: linux
       arch: amd64
       env:
         - TARGET=x86_64-unknown-linux-gnu
         - TARGET_FEATURE=sse3
+
     - name: "x86_64-unknown-linux-gnu+sse4.1"
       os: linux
       arch: amd64
       env:
         - TARGET=x86_64-unknown-linux-gnu
         - TARGET_FEATURE=sse4.1
+
     - name: "x86_64-unknown-linux-gnu+sse4.2"
       os: linux
       arch: amd64
       env:
         - TARGET=x86_64-unknown-linux-gnu
         - TARGET_FEATURE=sse4.2
+
     - name: "x86_64-unknown-linux-gnu+avx"
       os: linux
       arch: amd64
       env:
         - TARGET=x86_64-unknown-linux-gnu
         - TARGET_FEATURE=avx
+
     - name: "x86_64-unknown-linux-gnu+avx2"
       os: linux
       arch: amd64
       env:
         - TARGET=x86_64-unknown-linux-gnu
         - TARGET_FEATURE=avx2
+
     - name: "x86_64-unknown-linux-gnu+avx512vl"
       os: linux
       arch: amd64
@@ -61,17 +70,20 @@ matrix:
         - TARGET_FEATURE=avx512vl
 
     # Linux (aarch64)
+
     - name: "aarch64-unknown-linux-gnu"
       os: linux
       arch: arm64
       env:
         - TARGET=aarch64-unknown-linux-gnu
+
     - name: "aarch64-unknown-linux-gnu+neon"
       os: linux
       arch: arm64
       env:
         - TARGET=aarch64-unknown-linux-gnu
         - TARGET_FEATURE=neon
+
     - name: "aarch64-unknown-linux-gnu+sve"
       os: linux
       arch: arm64
@@ -80,11 +92,13 @@ matrix:
         - TARGET_FEATURE=sve
 
     # Linux (powerpc64)
+
     - name: "powerpc64le-unknown-linux-gnu"
       os: linux
       arch: ppc64le
       env:
         - TARGET=powerpc64le-unknown-linux-gnu
+
     - name: "powerpc64le-unknown-linux-gnu+vsx"
       os: linux
       arch: ppc64le
@@ -93,51 +107,60 @@ matrix:
         - TARGET_FEATURE=vsx
 
     # Windows (x86_64)
+
     - name: "x86_64-pc-windows-msvc"
       os: windows
       arch: amd64
       env: TARGET=x86_64-pc-windows-msvc
     
     # Windows (i686)
+
     - name: "i686-pc-windows-msvc"
       os: windows
       env: TARGET=i686-pc-windows-msvc
+
     - name: "i686-pc-windows-msvc+sse"
       os: windows
       arch: amd64
       env:
         - TARGET=i686-pc-windows-msvc
         - TARGET_FEATURE=sse
+
     - name: "i686-pc-windows-msvc+sse2"
       os: windows
       arch: amd64
       env:
         - TARGET=i686-pc-windows-msvc
         - TARGET_FEATURE=sse2
+
     - name: "i686-pc-windows-msvc+sse3"
       os: windows
       arch: amd64
       env:
         - TARGET=i686-pc-windows-msvc
         - TARGET_FEATURE=sse3
+
     - name: "i686-pc-windows-msvc+sse4.1"
       os: windows
       arch: amd64
       env:
         - TARGET=i686-pc-windows-msvc
         - TARGET_FEATURE=sse4.1
+
     - name: "i686-pc-windows-msvc+sse4.2"
       os: windows
       arch: amd64
       env:
         - TARGET=i686-pc-windows-msvc
         - TARGET_FEATURE=sse4.2
+
     - name: "i686-pc-windows-msvc+avx"
       os: windows
       arch: amd64
       env:
         - TARGET=i686-pc-windows-msvc
         - TARGET_FEATURE=avx
+
     - name: "i686-pc-windows-msvc+avx2"
       os: windows
       arch: amd64
@@ -145,7 +168,63 @@ matrix:
         - TARGET=i686-pc-windows-msvc
         - TARGET_FEATURE=avx2
 
+    # Windows (i586)
+
+    - name: "i586-pc-windows-msvc"
+      os: windows
+      env: TARGET=i586-pc-windows-msvc
+
+    - name: "i586-pc-windows-msvc+sse"
+      os: windows
+      arch: amd64
+      env:
+        - TARGET=i586-pc-windows-msvc
+        - TARGET_FEATURE=sse
+
+    - name: "i586-pc-windows-msvc+sse2"
+      os: windows
+      arch: amd64
+      env:
+        - TARGET=i586-pc-windows-msvc
+        - TARGET_FEATURE=sse2
+
+    - name: "i586-pc-windows-msvc+sse3"
+      os: windows
+      arch: amd64
+      env:
+        - TARGET=i586-pc-windows-msvc
+        - TARGET_FEATURE=sse3
+
+    - name: "i586-pc-windows-msvc+sse4.1"
+      os: windows
+      arch: amd64
+      env:
+        - TARGET=i586-pc-windows-msvc
+        - TARGET_FEATURE=sse4.1
+
+    - name: "i586-pc-windows-msvc+sse4.2"
+      os: windows
+      arch: amd64
+      env:
+        - TARGET=i586-pc-windows-msvc
+        - TARGET_FEATURE=sse4.2
+
+    - name: "i586-pc-windows-msvc+avx"
+      os: windows
+      arch: amd64
+      env:
+        - TARGET=i586-pc-windows-msvc
+        - TARGET_FEATURE=avx
+
+    - name: "i586-pc-windows-msvc+avx2"
+      os: windows
+      arch: amd64
+      env:
+        - TARGET=i586-pc-windows-msvc
+        - TARGET_FEATURE=avx2
+
     # OSX (x86_64)
+
     - name: "x86_64-apple-darwin"
       os: osx
       arch: amd64

From 43dabd1aeabe02c3404c76a8c62f29b55f1d1c7c Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Thu, 1 Oct 2020 22:50:15 -0400
Subject: [PATCH 029/249] Implement `core::ops` (#10)

* Add vector-vector arithmetic ops
* Add operators and integer conversions for masks
* Add unary traits
* Implement Index and IndexMut
* Implement by-ref ops for masks
* Document intrinsics
* Implement format traits for masks
* Add floating point ops tests
* Add integer tests
* Add mask tests
---
 crates/core_simd/src/intrinsics.rs            |  39 ++
 crates/core_simd/src/lib.rs                   |   4 +-
 crates/core_simd/src/masks.rs                 |  51 ++
 crates/core_simd/src/ops.rs                   | 628 ++++++++++++++++++
 crates/core_simd/tests/helpers/biteq.rs       | 127 ++++
 crates/core_simd/tests/helpers/lanewise.rs    |  49 ++
 crates/core_simd/tests/helpers/mod.rs         |   4 +
 crates/core_simd/tests/ops.rs                 |   1 +
 crates/core_simd/tests/ops_impl/f32.rs        |   6 +
 crates/core_simd/tests/ops_impl/f64.rs        |   5 +
 .../core_simd/tests/ops_impl/float_macros.rs  | 237 +++++++
 crates/core_simd/tests/ops_impl/i128.rs       |   4 +
 crates/core_simd/tests/ops_impl/i16.rs        |   6 +
 crates/core_simd/tests/ops_impl/i32.rs        |   6 +
 crates/core_simd/tests/ops_impl/i64.rs        |   5 +
 crates/core_simd/tests/ops_impl/i8.rs         |   6 +
 crates/core_simd/tests/ops_impl/int_macros.rs | 388 +++++++++++
 crates/core_simd/tests/ops_impl/isize.rs      |   5 +
 crates/core_simd/tests/ops_impl/mask128.rs    |   4 +
 crates/core_simd/tests/ops_impl/mask16.rs     |   6 +
 crates/core_simd/tests/ops_impl/mask32.rs     |   6 +
 crates/core_simd/tests/ops_impl/mask64.rs     |   5 +
 crates/core_simd/tests/ops_impl/mask8.rs      |   6 +
 .../core_simd/tests/ops_impl/mask_macros.rs   | 179 +++++
 crates/core_simd/tests/ops_impl/masksize.rs   |   5 +
 crates/core_simd/tests/ops_impl/mod.rs        |  39 ++
 crates/core_simd/tests/ops_impl/u128.rs       |   4 +
 crates/core_simd/tests/ops_impl/u16.rs        |   6 +
 crates/core_simd/tests/ops_impl/u32.rs        |   6 +
 crates/core_simd/tests/ops_impl/u64.rs        |   5 +
 crates/core_simd/tests/ops_impl/u8.rs         |   6 +
 .../core_simd/tests/ops_impl/uint_macros.rs   | 381 +++++++++++
 crates/core_simd/tests/ops_impl/usize.rs      |   5 +
 33 files changed, 2233 insertions(+), 1 deletion(-)
 create mode 100644 crates/core_simd/src/intrinsics.rs
 create mode 100644 crates/core_simd/src/ops.rs
 create mode 100644 crates/core_simd/tests/helpers/biteq.rs
 create mode 100644 crates/core_simd/tests/helpers/lanewise.rs
 create mode 100644 crates/core_simd/tests/helpers/mod.rs
 create mode 100644 crates/core_simd/tests/ops.rs
 create mode 100644 crates/core_simd/tests/ops_impl/f32.rs
 create mode 100644 crates/core_simd/tests/ops_impl/f64.rs
 create mode 100644 crates/core_simd/tests/ops_impl/float_macros.rs
 create mode 100644 crates/core_simd/tests/ops_impl/i128.rs
 create mode 100644 crates/core_simd/tests/ops_impl/i16.rs
 create mode 100644 crates/core_simd/tests/ops_impl/i32.rs
 create mode 100644 crates/core_simd/tests/ops_impl/i64.rs
 create mode 100644 crates/core_simd/tests/ops_impl/i8.rs
 create mode 100644 crates/core_simd/tests/ops_impl/int_macros.rs
 create mode 100644 crates/core_simd/tests/ops_impl/isize.rs
 create mode 100644 crates/core_simd/tests/ops_impl/mask128.rs
 create mode 100644 crates/core_simd/tests/ops_impl/mask16.rs
 create mode 100644 crates/core_simd/tests/ops_impl/mask32.rs
 create mode 100644 crates/core_simd/tests/ops_impl/mask64.rs
 create mode 100644 crates/core_simd/tests/ops_impl/mask8.rs
 create mode 100644 crates/core_simd/tests/ops_impl/mask_macros.rs
 create mode 100644 crates/core_simd/tests/ops_impl/masksize.rs
 create mode 100644 crates/core_simd/tests/ops_impl/mod.rs
 create mode 100644 crates/core_simd/tests/ops_impl/u128.rs
 create mode 100644 crates/core_simd/tests/ops_impl/u16.rs
 create mode 100644 crates/core_simd/tests/ops_impl/u32.rs
 create mode 100644 crates/core_simd/tests/ops_impl/u64.rs
 create mode 100644 crates/core_simd/tests/ops_impl/u8.rs
 create mode 100644 crates/core_simd/tests/ops_impl/uint_macros.rs
 create mode 100644 crates/core_simd/tests/ops_impl/usize.rs

diff --git a/crates/core_simd/src/intrinsics.rs b/crates/core_simd/src/intrinsics.rs
new file mode 100644
index 000000000000..c2cef778560f
--- /dev/null
+++ b/crates/core_simd/src/intrinsics.rs
@@ -0,0 +1,39 @@
+//! This module contains the LLVM intrinsics bindings that provide the functionality for this
+//! crate.
+//!
+//! The LLVM assembly language is documented here: https://llvm.org/docs/LangRef.html
+
+/// These intrinsics aren't linked directly from LLVM and are mostly undocumented, however they are
+/// simply lowered to the matching LLVM instructions by the compiler.  The associated instruction
+/// is documented alongside each intrinsic.
+extern "platform-intrinsic" {
+    /// add/fadd
+    pub(crate) fn simd_add<T>(x: T, y: T) -> T;
+
+    /// sub/fsub
+    pub(crate) fn simd_sub<T>(x: T, y: T) -> T;
+
+    /// mul/fmul
+    pub(crate) fn simd_mul<T>(x: T, y: T) -> T;
+
+    /// udiv/sdiv/fdiv
+    pub(crate) fn simd_div<T>(x: T, y: T) -> T;
+
+    /// urem/srem/frem
+    pub(crate) fn simd_rem<T>(x: T, y: T) -> T;
+
+    /// shl
+    pub(crate) fn simd_shl<T>(x: T, y: T) -> T;
+
+    /// lshr/ashr
+    pub(crate) fn simd_shr<T>(x: T, y: T) -> T;
+
+    /// and
+    pub(crate) fn simd_and<T>(x: T, y: T) -> T;
+
+    /// or
+    pub(crate) fn simd_or<T>(x: T, y: T) -> T;
+
+    /// xor
+    pub(crate) fn simd_xor<T>(x: T, y: T) -> T;
+}
diff --git a/crates/core_simd/src/lib.rs b/crates/core_simd/src/lib.rs
index d88f5b1eac45..d08ef400f9bb 100644
--- a/crates/core_simd/src/lib.rs
+++ b/crates/core_simd/src/lib.rs
@@ -1,5 +1,5 @@
 #![no_std]
-#![feature(repr_simd)]
+#![feature(repr_simd, platform_intrinsics)]
 #![warn(missing_docs)]
 //! Portable SIMD module.
 
@@ -7,6 +7,8 @@
 mod macros;
 
 mod fmt;
+mod intrinsics;
+mod ops;
 
 mod masks;
 pub use masks::*;
diff --git a/crates/core_simd/src/masks.rs b/crates/core_simd/src/masks.rs
index 1fc281a310d3..cba76b6a2a35 100644
--- a/crates/core_simd/src/masks.rs
+++ b/crates/core_simd/src/masks.rs
@@ -1,3 +1,13 @@
+/// The error type returned when converting an integer to a mask fails.
+#[derive(Debug, Copy, Clone, PartialEq, Eq)]
+pub struct TryFromMaskError(());
+
+impl core::fmt::Display for TryFromMaskError {
+    fn fmt(&self, f: &mut core::fmt::Formatter) -> core::fmt::Result {
+        write!(f, "mask must have all bits set or unset")
+    }
+}
+
 macro_rules! define_mask {
     { $(#[$attr:meta])* struct $name:ident($type:ty); } => {
         $(#[$attr])*
@@ -34,11 +44,52 @@ macro_rules! define_mask {
             }
         }
 
+        impl core::convert::TryFrom<$type> for $name {
+            type Error = TryFromMaskError;
+            fn try_from(value: $type) -> Result<Self, Self::Error> {
+                if value == 0 || !value == 0 {
+                    Ok(Self(value))
+                } else {
+                    Err(TryFromMaskError(()))
+                }
+            }
+        }
+
+        impl core::convert::From<$name> for $type {
+            fn from(value: $name) -> Self {
+                value.0
+            }
+        }
+
         impl core::fmt::Debug for $name {
             fn fmt(&self, f: &mut core::fmt::Formatter) -> core::fmt::Result {
                 self.test().fmt(f)
             }
         }
+
+        impl core::fmt::Binary for $name {
+            fn fmt(&self, f: &mut core::fmt::Formatter) -> core::fmt::Result {
+                <$type as core::fmt::Binary>::fmt(&self.0, f)
+            }
+        }
+
+        impl core::fmt::Octal for $name {
+            fn fmt(&self, f: &mut core::fmt::Formatter) -> core::fmt::Result {
+                <$type as core::fmt::Octal>::fmt(&self.0, f)
+            }
+        }
+
+        impl core::fmt::LowerHex for $name {
+            fn fmt(&self, f: &mut core::fmt::Formatter) -> core::fmt::Result {
+                <$type as core::fmt::LowerHex>::fmt(&self.0, f)
+            }
+        }
+
+        impl core::fmt::UpperHex for $name {
+            fn fmt(&self, f: &mut core::fmt::Formatter) -> core::fmt::Result {
+                <$type as core::fmt::UpperHex>::fmt(&self.0, f)
+            }
+        }
     }
 }
 
diff --git a/crates/core_simd/src/ops.rs b/crates/core_simd/src/ops.rs
new file mode 100644
index 000000000000..5af10a4e1886
--- /dev/null
+++ b/crates/core_simd/src/ops.rs
@@ -0,0 +1,628 @@
+/// Checks if the right-hand side argument of a left- or right-shift would cause overflow.
+fn invalid_shift_rhs<T>(rhs: T) -> bool
+where
+    T: Default + PartialOrd + core::convert::TryFrom<usize>,
+    <T as core::convert::TryFrom<usize>>::Error: core::fmt::Debug,
+{
+    let bits_in_type = T::try_from(8 * core::mem::size_of::<T>()).unwrap();
+    rhs < T::default() || rhs >= bits_in_type
+}
+
+/// Automatically implements operators over references in addition to the provided operator.
+macro_rules! impl_ref_ops {
+    // binary op
+    {
+        impl core::ops::$trait:ident<$rhs:ty> for $type:ty {
+            type Output = $output:ty;
+
+            $(#[$attrs:meta])*
+            fn $fn:ident($self_tok:ident, $rhs_arg:ident: $rhs_arg_ty:ty) -> Self::Output $body:tt
+        }
+    } => {
+        impl core::ops::$trait<$rhs> for $type {
+            type Output = $output;
+
+            $(#[$attrs])*
+            fn $fn($self_tok, $rhs_arg: $rhs_arg_ty) -> Self::Output $body
+        }
+
+        impl core::ops::$trait<&'_ $rhs> for $type {
+            type Output = <$type as core::ops::$trait<$rhs>>::Output;
+
+            $(#[$attrs])*
+            fn $fn($self_tok, $rhs_arg: &$rhs) -> Self::Output {
+                core::ops::$trait::$fn($self_tok, *$rhs_arg)
+            }
+        }
+
+        impl core::ops::$trait<$rhs> for &'_ $type {
+            type Output = <$type as core::ops::$trait<$rhs>>::Output;
+
+            $(#[$attrs])*
+            fn $fn($self_tok, $rhs_arg: $rhs) -> Self::Output {
+                core::ops::$trait::$fn(*$self_tok, $rhs_arg)
+            }
+        }
+
+        impl core::ops::$trait<&'_ $rhs> for &'_ $type {
+            type Output = <$type as core::ops::$trait<$rhs>>::Output;
+
+            $(#[$attrs])*
+            fn $fn($self_tok, $rhs_arg: &$rhs) -> Self::Output {
+                core::ops::$trait::$fn(*$self_tok, *$rhs_arg)
+            }
+        }
+    };
+
+    // binary assignment op
+    {
+        impl core::ops::$trait:ident<$rhs:ty> for $type:ty {
+            $(#[$attrs:meta])*
+            fn $fn:ident(&mut $self_tok:ident, $rhs_arg:ident: $rhs_arg_ty:ty) $body:tt
+        }
+    } => {
+        impl core::ops::$trait<$rhs> for $type {
+            $(#[$attrs])*
+            fn $fn(&mut $self_tok, $rhs_arg: $rhs_arg_ty) $body
+        }
+
+        impl core::ops::$trait<&'_ $rhs> for $type {
+            $(#[$attrs])*
+            fn $fn(&mut $self_tok, $rhs_arg: &$rhs_arg_ty) {
+                core::ops::$trait::$fn($self_tok, *$rhs_arg)
+            }
+        }
+    };
+
+    // unary op
+    {
+        impl core::ops::$trait:ident for $type:ty {
+            type Output = $output:ty;
+            fn $fn:ident($self_tok:ident) -> Self::Output $body:tt
+        }
+    } => {
+        impl core::ops::$trait for $type {
+            type Output = $output;
+            fn $fn($self_tok) -> Self::Output $body
+        }
+
+        impl core::ops::$trait for &'_ $type {
+            type Output = <$type as core::ops::$trait>::Output;
+            fn $fn($self_tok) -> Self::Output {
+                core::ops::$trait::$fn(*$self_tok)
+            }
+        }
+    }
+}
+
+/// Implements op traits for masks
+macro_rules! impl_mask_ops {
+    { $($mask:ty),* } => {
+        $(
+            impl_ref_ops! {
+                impl core::ops::BitAnd<$mask> for $mask {
+                    type Output = Self;
+                    fn bitand(self, rhs: Self) -> Self::Output {
+                        Self(self.0 & rhs.0)
+                    }
+                }
+            }
+
+            impl_ref_ops! {
+                impl core::ops::BitAndAssign<$mask> for $mask {
+                    fn bitand_assign(&mut self, rhs: Self) {
+                        *self = *self & rhs;
+                    }
+                }
+            }
+
+            impl_ref_ops! {
+                impl core::ops::BitOr<$mask> for $mask {
+                    type Output = Self;
+                    fn bitor(self, rhs: Self) -> Self::Output {
+                        Self(self.0 | rhs.0)
+                    }
+                }
+            }
+
+            impl_ref_ops! {
+                impl core::ops::BitOrAssign<$mask> for $mask {
+                    fn bitor_assign(&mut self, rhs: Self) {
+                        *self = *self | rhs;
+                    }
+                }
+            }
+
+            impl_ref_ops! {
+                impl core::ops::BitXor<$mask> for $mask {
+                    type Output = Self;
+                    fn bitxor(self, rhs: Self) -> Self::Output {
+                        Self(self.0 ^ rhs.0)
+                    }
+                }
+            }
+
+            impl_ref_ops! {
+                impl core::ops::BitXorAssign<$mask> for $mask {
+                    fn bitxor_assign(&mut self, rhs: Self) {
+                        *self = *self ^ rhs;
+                    }
+                }
+            }
+
+            impl_ref_ops! {
+                impl core::ops::Not for $mask {
+                    type Output = Self;
+                    fn not(self) -> Self::Output {
+                        Self(!self.0)
+                    }
+                }
+            }
+        )*
+    }
+}
+impl_mask_ops! { crate::mask8, crate::mask16, crate::mask32, crate::mask64, crate::mask128, crate::masksize }
+
+/// Automatically implements operators over vectors and scalars for a particular vector.
+macro_rules! impl_op {
+    { impl Add for $type:ty, $scalar:ty } => {
+        impl_op! { @binary $type, $scalar, Add::add, AddAssign::add_assign, simd_add }
+    };
+    { impl Sub for $type:ty, $scalar:ty } => {
+        impl_op! { @binary $type, $scalar, Sub::sub, SubAssign::sub_assign, simd_sub }
+    };
+    { impl Mul for $type:ty, $scalar:ty } => {
+        impl_op! { @binary $type, $scalar, Mul::mul, MulAssign::mul_assign, simd_mul }
+    };
+    { impl Div for $type:ty, $scalar:ty } => {
+        impl_op! { @binary $type, $scalar, Div::div, DivAssign::div_assign, simd_div }
+    };
+    { impl Rem for $type:ty, $scalar:ty } => {
+        impl_op! { @binary $type, $scalar, Rem::rem, RemAssign::rem_assign, simd_rem }
+    };
+    { impl Shl for $type:ty, $scalar:ty } => {
+        impl_op! { @binary $type, $scalar, Shl::shl, ShlAssign::shl_assign, simd_shl }
+    };
+    { impl Shr for $type:ty, $scalar:ty } => {
+        impl_op! { @binary $type, $scalar, Shr::shr, ShrAssign::shr_assign, simd_shr }
+    };
+    { impl BitAnd for $type:ty, $scalar:ty } => {
+        impl_op! { @binary $type, $scalar, BitAnd::bitand, BitAndAssign::bitand_assign, simd_and }
+    };
+    { impl BitOr for $type:ty, $scalar:ty } => {
+        impl_op! { @binary $type, $scalar, BitOr::bitor, BitOrAssign::bitor_assign, simd_or }
+    };
+    { impl BitXor for $type:ty, $scalar:ty } => {
+        impl_op! { @binary $type, $scalar, BitXor::bitxor, BitXorAssign::bitxor_assign, simd_xor }
+    };
+
+    { impl Not for $type:ty, $scalar:ty } => {
+        impl_ref_ops! {
+            impl core::ops::Not for $type {
+                type Output = Self;
+                fn not(self) -> Self::Output {
+                    self ^ <$type>::splat(!<$scalar>::default())
+                }
+            }
+        }
+    };
+
+    { impl Neg for $type:ty, $scalar:ty } => {
+        impl_ref_ops! {
+            impl core::ops::Neg for $type {
+                type Output = Self;
+                fn neg(self) -> Self::Output {
+                    <$type>::splat(-<$scalar>::default()) - self
+                }
+            }
+        }
+    };
+
+    { impl Index for $type:ty, $scalar:ty } => {
+        impl<I> core::ops::Index<I> for $type
+        where
+            I: core::slice::SliceIndex<[$scalar]>,
+        {
+            type Output = I::Output;
+            fn index(&self, index: I) -> &Self::Output {
+                let slice: &[_] = self.as_ref();
+                &slice[index]
+            }
+        }
+
+        impl<I> core::ops::IndexMut<I> for $type
+        where
+            I: core::slice::SliceIndex<[$scalar]>,
+        {
+            fn index_mut(&mut self, index: I) -> &mut Self::Output {
+                let slice: &mut [_] = self.as_mut();
+                &mut slice[index]
+            }
+        }
+    };
+
+    // generic binary op with assignment when output is `Self`
+    { @binary $type:ty, $scalar:ty, $trait:ident :: $trait_fn:ident, $assign_trait:ident :: $assign_trait_fn:ident, $intrinsic:ident } => {
+        impl_ref_ops! {
+            impl core::ops::$trait<$type> for $type {
+                type Output = $type;
+
+                #[inline]
+                fn $trait_fn(self, rhs: $type) -> Self::Output {
+                    unsafe {
+                        crate::intrinsics::$intrinsic(self, rhs)
+                    }
+                }
+            }
+        }
+
+        impl_ref_ops! {
+            impl core::ops::$trait<$scalar> for $type {
+                type Output = $type;
+
+                #[inline]
+                fn $trait_fn(self, rhs: $scalar) -> Self::Output {
+                    core::ops::$trait::$trait_fn(self, <$type>::splat(rhs))
+                }
+            }
+        }
+
+        impl_ref_ops! {
+            impl core::ops::$trait<$type> for $scalar {
+                type Output = $type;
+
+                #[inline]
+                fn $trait_fn(self, rhs: $type) -> Self::Output {
+                    core::ops::$trait::$trait_fn(<$type>::splat(self), rhs)
+                }
+            }
+        }
+
+        impl_ref_ops! {
+            impl core::ops::$assign_trait<$type> for $type {
+                #[inline]
+                fn $assign_trait_fn(&mut self, rhs: $type) {
+                    unsafe {
+                        *self = crate::intrinsics::$intrinsic(*self, rhs);
+                    }
+                }
+            }
+        }
+
+        impl_ref_ops! {
+            impl core::ops::$assign_trait<$scalar> for $type {
+                #[inline]
+                fn $assign_trait_fn(&mut self, rhs: $scalar) {
+                    core::ops::$assign_trait::$assign_trait_fn(self, <$type>::splat(rhs));
+                }
+            }
+        }
+    };
+}
+
+/// Implements floating-point operators for the provided types.
+macro_rules! impl_float_ops {
+    { $($scalar:ty => $($vector:ty),*;)* } => {
+        $( // scalar
+            $( // vector
+                impl_op! { impl Add for $vector, $scalar }
+                impl_op! { impl Sub for $vector, $scalar }
+                impl_op! { impl Mul for $vector, $scalar }
+                impl_op! { impl Div for $vector, $scalar }
+                impl_op! { impl Rem for $vector, $scalar }
+                impl_op! { impl Neg for $vector, $scalar }
+                impl_op! { impl Index for $vector, $scalar }
+            )*
+        )*
+    };
+}
+
+/// Implements mask operators for the provided types.
+macro_rules! impl_mask_ops {
+    { $($scalar:ty => $($vector:ty),*;)* } => {
+        $( // scalar
+            $( // vector
+                impl_op! { impl BitAnd for $vector, $scalar }
+                impl_op! { impl BitOr  for $vector, $scalar }
+                impl_op! { impl BitXor for $vector, $scalar }
+                impl_op! { impl Not for $vector, $scalar }
+                impl_op! { impl Index for $vector, $scalar }
+            )*
+        )*
+    };
+}
+
+/// Implements unsigned integer operators for the provided types.
+macro_rules! impl_unsigned_int_ops {
+    { $($scalar:ty => $($vector:ty),*;)* } => {
+        $( // scalar
+            $( // vector
+                impl_op! { impl Add for $vector, $scalar }
+                impl_op! { impl Sub for $vector, $scalar }
+                impl_op! { impl Mul for $vector, $scalar }
+                impl_op! { impl BitAnd for $vector, $scalar }
+                impl_op! { impl BitOr  for $vector, $scalar }
+                impl_op! { impl BitXor for $vector, $scalar }
+                impl_op! { impl Not for $vector, $scalar }
+                impl_op! { impl Index for $vector, $scalar }
+
+                // Integers panic on divide by 0
+                impl_ref_ops! {
+                    impl core::ops::Div<$vector> for $vector {
+                        type Output = Self;
+
+                        #[inline]
+                        fn div(self, rhs: $vector) -> Self::Output {
+                            // TODO there is probably a better way of doing this
+                            if AsRef::<[$scalar]>::as_ref(&rhs)
+                                .iter()
+                                .any(|x| *x == 0)
+                            {
+                                panic!("attempt to divide by zero");
+                            }
+                            unsafe { crate::intrinsics::simd_div(self, rhs) }
+                        }
+                    }
+                }
+
+                impl_ref_ops! {
+                    impl core::ops::Div<$scalar> for $vector {
+                        type Output = $vector;
+
+                        #[inline]
+                        fn div(self, rhs: $scalar) -> Self::Output {
+                            if rhs == 0 {
+                                panic!("attempt to divide by zero");
+                            }
+                            let rhs = Self::splat(rhs);
+                            unsafe { crate::intrinsics::simd_div(self, rhs) }
+                        }
+                    }
+                }
+
+                impl_ref_ops! {
+                    impl core::ops::Div<$vector> for $scalar {
+                        type Output = $vector;
+
+                        #[inline]
+                        fn div(self, rhs: $vector) -> Self::Output {
+                            <$vector>::splat(self) / rhs
+                        }
+                    }
+                }
+
+                impl_ref_ops! {
+                    impl core::ops::DivAssign<$vector> for $vector {
+                        #[inline]
+                        fn div_assign(&mut self, rhs: Self) {
+                            *self = *self / rhs;
+                        }
+                    }
+                }
+
+                impl_ref_ops! {
+                    impl core::ops::DivAssign<$scalar> for $vector {
+                        #[inline]
+                        fn div_assign(&mut self, rhs: $scalar) {
+                            *self = *self / rhs;
+                        }
+                    }
+                }
+
+                // remainder panics on zero divisor
+                impl_ref_ops! {
+                    impl core::ops::Rem<$vector> for $vector {
+                        type Output = Self;
+
+                        #[inline]
+                        fn rem(self, rhs: $vector) -> Self::Output {
+                            // TODO there is probably a better way of doing this
+                            if AsRef::<[$scalar]>::as_ref(&rhs)
+                                .iter()
+                                .any(|x| *x == 0)
+                            {
+                                panic!("attempt to calculate the remainder with a divisor of zero");
+                            }
+                            unsafe { crate::intrinsics::simd_rem(self, rhs) }
+                        }
+                    }
+                }
+
+                impl_ref_ops! {
+                    impl core::ops::Rem<$scalar> for $vector {
+                        type Output = $vector;
+
+                        #[inline]
+                        fn rem(self, rhs: $scalar) -> Self::Output {
+                            if rhs == 0 {
+                                panic!("attempt to calculate the remainder with a divisor of zero");
+                            }
+                            let rhs = Self::splat(rhs);
+                            unsafe { crate::intrinsics::simd_rem(self, rhs) }
+                        }
+                    }
+                }
+
+                impl_ref_ops! {
+                    impl core::ops::Rem<$vector> for $scalar {
+                        type Output = $vector;
+
+                        #[inline]
+                        fn rem(self, rhs: $vector) -> Self::Output {
+                            <$vector>::splat(self) % rhs
+                        }
+                    }
+                }
+
+                impl_ref_ops! {
+                    impl core::ops::RemAssign<$vector> for $vector {
+                        #[inline]
+                        fn rem_assign(&mut self, rhs: Self) {
+                            *self = *self % rhs;
+                        }
+                    }
+                }
+
+                impl_ref_ops! {
+                    impl core::ops::RemAssign<$scalar> for $vector {
+                        #[inline]
+                        fn rem_assign(&mut self, rhs: $scalar) {
+                            *self = *self % rhs;
+                        }
+                    }
+                }
+
+                // shifts panic on overflow
+                impl_ref_ops! {
+                    impl core::ops::Shl<$vector> for $vector {
+                        type Output = Self;
+
+                        #[inline]
+                        fn shl(self, rhs: $vector) -> Self::Output {
+                            // TODO there is probably a better way of doing this
+                            if AsRef::<[$scalar]>::as_ref(&rhs)
+                                .iter()
+                                .copied()
+                                .any(invalid_shift_rhs)
+                            {
+                                panic!("attempt to shift left with overflow");
+                            }
+                            unsafe { crate::intrinsics::simd_shl(self, rhs) }
+                        }
+                    }
+                }
+
+                impl_ref_ops! {
+                    impl core::ops::Shl<$scalar> for $vector {
+                        type Output = $vector;
+
+                        #[inline]
+                        fn shl(self, rhs: $scalar) -> Self::Output {
+                            if invalid_shift_rhs(rhs) {
+                                panic!("attempt to shift left with overflow");
+                            }
+                            let rhs = Self::splat(rhs);
+                            unsafe { crate::intrinsics::simd_shl(self, rhs) }
+                        }
+                    }
+                }
+
+
+                impl_ref_ops! {
+                    impl core::ops::ShlAssign<$vector> for $vector {
+                        #[inline]
+                        fn shl_assign(&mut self, rhs: Self) {
+                            *self = *self << rhs;
+                        }
+                    }
+                }
+
+                impl_ref_ops! {
+                    impl core::ops::ShlAssign<$scalar> for $vector {
+                        #[inline]
+                        fn shl_assign(&mut self, rhs: $scalar) {
+                            *self = *self << rhs;
+                        }
+                    }
+                }
+
+                impl_ref_ops! {
+                    impl core::ops::Shr<$vector> for $vector {
+                        type Output = Self;
+
+                        #[inline]
+                        fn shr(self, rhs: $vector) -> Self::Output {
+                            // TODO there is probably a better way of doing this
+                            if AsRef::<[$scalar]>::as_ref(&rhs)
+                                .iter()
+                                .copied()
+                                .any(invalid_shift_rhs)
+                            {
+                                panic!("attempt to shift with overflow");
+                            }
+                            unsafe { crate::intrinsics::simd_shr(self, rhs) }
+                        }
+                    }
+                }
+
+                impl_ref_ops! {
+                    impl core::ops::Shr<$scalar> for $vector {
+                        type Output = $vector;
+
+                        #[inline]
+                        fn shr(self, rhs: $scalar) -> Self::Output {
+                            if invalid_shift_rhs(rhs) {
+                                panic!("attempt to shift with overflow");
+                            }
+                            let rhs = Self::splat(rhs);
+                            unsafe { crate::intrinsics::simd_shr(self, rhs) }
+                        }
+                    }
+                }
+
+
+                impl_ref_ops! {
+                    impl core::ops::ShrAssign<$vector> for $vector {
+                        #[inline]
+                        fn shr_assign(&mut self, rhs: Self) {
+                            *self = *self >> rhs;
+                        }
+                    }
+                }
+
+                impl_ref_ops! {
+                    impl core::ops::ShrAssign<$scalar> for $vector {
+                        #[inline]
+                        fn shr_assign(&mut self, rhs: $scalar) {
+                            *self = *self >> rhs;
+                        }
+                    }
+                }
+            )*
+        )*
+    };
+}
+
+/// Implements unsigned integer operators for the provided types.
+macro_rules! impl_signed_int_ops {
+    { $($scalar:ty => $($vector:ty),*;)* } => {
+        impl_unsigned_int_ops! { $($scalar => $($vector),*;)* }
+        $( // scalar
+            $( // vector
+                impl_op! { impl Neg for $vector, $scalar }
+            )*
+        )*
+    };
+}
+
+impl_unsigned_int_ops! {
+    u8    => crate::u8x8,    crate::u8x16,   crate::u8x32,   crate::u8x64;
+    u16   => crate::u16x4,   crate::u16x8,   crate::u16x16,  crate::u16x32;
+    u32   => crate::u32x2,   crate::u32x4,   crate::u32x8,   crate::u32x16;
+    u64   => crate::u64x2,   crate::u64x4,   crate::u64x8;
+    u128  => crate::u128x2,  crate::u128x4;
+    usize => crate::usizex2, crate::usizex4, crate::usizex8;
+}
+
+impl_signed_int_ops! {
+    i8    => crate::i8x8,    crate::i8x16,   crate::i8x32,   crate::i8x64;
+    i16   => crate::i16x4,   crate::i16x8,   crate::i16x16,  crate::i16x32;
+    i32   => crate::i32x2,   crate::i32x4,   crate::i32x8,   crate::i32x16;
+    i64   => crate::i64x2,   crate::i64x4,   crate::i64x8;
+    i128  => crate::i128x2,  crate::i128x4;
+    isize => crate::isizex2, crate::isizex4, crate::isizex8;
+}
+
+impl_float_ops! {
+    f32 => crate::f32x2, crate::f32x4, crate::f32x8, crate::f32x16;
+    f64 => crate::f64x2, crate::f64x4, crate::f64x8;
+}
+
+impl_mask_ops! {
+    crate::mask8    => crate::mask8x8,    crate::mask8x16,   crate::mask8x32,   crate::mask8x64;
+    crate::mask16   => crate::mask16x4,   crate::mask16x8,   crate::mask16x16,  crate::mask16x32;
+    crate::mask32   => crate::mask32x2,   crate::mask32x4,   crate::mask32x8,   crate::mask32x16;
+    crate::mask64   => crate::mask64x2,   crate::mask64x4,   crate::mask64x8;
+    crate::mask128  => crate::mask128x2,  crate::mask128x4;
+    crate::masksize => crate::masksizex2, crate::masksizex4, crate::masksizex8;
+}
diff --git a/crates/core_simd/tests/helpers/biteq.rs b/crates/core_simd/tests/helpers/biteq.rs
new file mode 100644
index 000000000000..f932eba907c3
--- /dev/null
+++ b/crates/core_simd/tests/helpers/biteq.rs
@@ -0,0 +1,127 @@
+pub(crate) trait BitEq {
+    fn biteq(&self, other: &Self) -> bool;
+    fn fmt(&self, f: &mut core::fmt::Formatter) -> core::fmt::Result;
+}
+
+macro_rules! impl_biteq {
+    { integer impl BitEq for $($type:ty,)* } => {
+        $(
+        impl BitEq for $type {
+            fn biteq(&self, other: &Self) -> bool {
+                self == other
+            }
+
+            fn fmt(&self, f: &mut core::fmt::Formatter) -> core::fmt::Result {
+                write!(f, "{:?} ({:x})", self, self)
+            }
+        }
+        )*
+    };
+    { float impl BitEq for $($type:ty,)* } => {
+        $(
+        impl BitEq for $type {
+            fn biteq(&self, other: &Self) -> bool {
+                self.to_bits() == other.to_bits()
+            }
+
+            fn fmt(&self, f: &mut core::fmt::Formatter) -> core::fmt::Result {
+                write!(f, "{:?} ({:x})", self, self.to_bits())
+            }
+        }
+        )*
+    };
+    { vector impl BitEq for $($type:ty,)* } => {
+        $(
+        impl BitEq for $type {
+            fn biteq(&self, other: &Self) -> bool {
+                let a: &[_] = self.as_ref();
+                let b: &[_] = other.as_ref();
+                if a.len() == b.len() {
+                    a.iter().zip(b.iter()).fold(true, |value, (left, right)| {
+                        value && left.biteq(right)
+                    })
+                } else {
+                    false
+                }
+            }
+
+            fn fmt(&self, f: &mut core::fmt::Formatter) -> core::fmt::Result {
+                #[repr(transparent)]
+                struct Wrapper<'a, T: BitEq>(&'a T);
+
+                impl<T: BitEq> core::fmt::Debug for Wrapper<'_, T> {
+                    fn fmt(&self, f: &mut core::fmt::Formatter) -> core::fmt::Result {
+                        self.0.fmt(f)
+                    }
+                }
+
+                let slice: &[_] = self.as_ref();
+
+                f.debug_list()
+                    .entries(slice.iter().map(|x| Wrapper(x)))
+                    .finish()
+            }
+        }
+        )*
+    };
+}
+
+impl_biteq! {
+    integer impl BitEq for
+        u8, u16, u32, u64, u128, usize,
+        i8, i16, i32, i64, i128, isize,
+        core_simd::mask8, core_simd::mask16, core_simd::mask32, core_simd::mask64, core_simd::mask128, core_simd::masksize,
+}
+
+impl_biteq! {
+    float impl BitEq for f32, f64,
+}
+
+impl_biteq! {
+    vector impl BitEq for
+        core_simd::u8x8,    core_simd::u8x16,   core_simd::u8x32,   core_simd::u8x64,
+        core_simd::i8x8,    core_simd::i8x16,   core_simd::i8x32,   core_simd::i8x64,
+        core_simd::u16x4,   core_simd::u16x8,   core_simd::u16x16,  core_simd::u16x32,
+        core_simd::i16x4,   core_simd::i16x8,   core_simd::i16x16,  core_simd::i16x32,
+        core_simd::u32x2,   core_simd::u32x4,   core_simd::u32x8,   core_simd::u32x16,
+        core_simd::i32x2,   core_simd::i32x4,   core_simd::i32x8,   core_simd::i32x16,
+        core_simd::u64x2,   core_simd::u64x4,   core_simd::u64x8,
+        core_simd::i64x2,   core_simd::i64x4,   core_simd::i64x8,
+        core_simd::u128x2,  core_simd::u128x4,
+        core_simd::i128x2,  core_simd::i128x4,
+        core_simd::usizex2, core_simd::usizex4, core_simd::usizex8,
+        core_simd::isizex2, core_simd::isizex4, core_simd::isizex8,
+        core_simd::f32x2, core_simd::f32x4, core_simd::f32x8, core_simd::f32x16,
+        core_simd::f64x2, core_simd::f64x4, core_simd::f64x8,
+        core_simd::mask8x8,    core_simd::mask8x16,   core_simd::mask8x32,   core_simd::mask8x64,
+        core_simd::mask16x4,   core_simd::mask16x8,   core_simd::mask16x16,  core_simd::mask16x32,
+        core_simd::mask32x2,   core_simd::mask32x4,   core_simd::mask32x8,   core_simd::mask32x16,
+        core_simd::mask64x2,   core_simd::mask64x4,   core_simd::mask64x8,
+        core_simd::mask128x2,  core_simd::mask128x4,
+        core_simd::masksizex2, core_simd::masksizex4, core_simd::masksizex8,
+}
+
+pub(crate) struct BitEqWrapper<'a, T>(pub(crate) &'a T);
+
+impl<T: BitEq> PartialEq for BitEqWrapper<'_, T> {
+    fn eq(&self, other: &Self) -> bool {
+        self.0.biteq(other.0)
+    }
+}
+
+impl<T: BitEq> core::fmt::Debug for BitEqWrapper<'_, T> {
+    fn fmt(&self, f: &mut core::fmt::Formatter) -> core::fmt::Result {
+        self.0.fmt(f)
+    }
+}
+
+macro_rules! assert_biteq {
+    { $a:expr, $b:expr } => {
+        {
+            use helpers::biteq::BitEqWrapper;
+            let a = $a;
+            let b = $b;
+            assert_eq!(BitEqWrapper(&a), BitEqWrapper(&b));
+        }
+    }
+}
diff --git a/crates/core_simd/tests/helpers/lanewise.rs b/crates/core_simd/tests/helpers/lanewise.rs
new file mode 100644
index 000000000000..6ab7803a9678
--- /dev/null
+++ b/crates/core_simd/tests/helpers/lanewise.rs
@@ -0,0 +1,49 @@
+pub fn apply_unary_lanewise<T: Copy, V: AsMut<[T]> + Default>(mut x: V, f: impl Fn(T) -> T) -> V {
+    for lane in x.as_mut() {
+        *lane = f(*lane)
+    }
+    x
+}
+
+pub fn apply_binary_lanewise<T: Copy, V: AsRef<[T]> + AsMut<[T]> + Default>(
+    a: V,
+    b: V,
+    f: impl Fn(T, T) -> T,
+) -> V {
+    let mut out = V::default();
+    let out_slice = out.as_mut();
+    let a_slice = a.as_ref();
+    let b_slice = b.as_ref();
+    for (o, (a, b)) in out_slice.iter_mut().zip(a_slice.iter().zip(b_slice.iter())) {
+        *o = f(*a, *b);
+    }
+    out
+}
+
+pub fn apply_binary_scalar_rhs_lanewise<T: Copy, V: AsRef<[T]> + AsMut<[T]> + Default>(
+    a: V,
+    b: T,
+    f: impl Fn(T, T) -> T,
+) -> V {
+    let mut out = V::default();
+    let out_slice = out.as_mut();
+    let a_slice = a.as_ref();
+    for (o, a) in out_slice.iter_mut().zip(a_slice.iter()) {
+        *o = f(*a, b);
+    }
+    out
+}
+
+pub fn apply_binary_scalar_lhs_lanewise<T: Copy, V: AsRef<[T]> + AsMut<[T]> + Default>(
+    a: T,
+    b: V,
+    f: impl Fn(T, T) -> T,
+) -> V {
+    let mut out = V::default();
+    let out_slice = out.as_mut();
+    let b_slice = b.as_ref();
+    for (o, b) in out_slice.iter_mut().zip(b_slice.iter()) {
+        *o = f(a, *b);
+    }
+    out
+}
diff --git a/crates/core_simd/tests/helpers/mod.rs b/crates/core_simd/tests/helpers/mod.rs
new file mode 100644
index 000000000000..b128f8251ca3
--- /dev/null
+++ b/crates/core_simd/tests/helpers/mod.rs
@@ -0,0 +1,4 @@
+#[macro_use]
+pub mod biteq;
+
+pub mod lanewise;
diff --git a/crates/core_simd/tests/ops.rs b/crates/core_simd/tests/ops.rs
new file mode 100644
index 000000000000..60aff06a76a6
--- /dev/null
+++ b/crates/core_simd/tests/ops.rs
@@ -0,0 +1 @@
+mod ops_impl;
diff --git a/crates/core_simd/tests/ops_impl/f32.rs b/crates/core_simd/tests/ops_impl/f32.rs
new file mode 100644
index 000000000000..f87909b68cd3
--- /dev/null
+++ b/crates/core_simd/tests/ops_impl/f32.rs
@@ -0,0 +1,6 @@
+use super::helpers;
+
+float_tests! { f32x2, f32 }
+float_tests! { f32x4, f32 }
+float_tests! { f32x8, f32 }
+float_tests! { f32x16, f32 }
diff --git a/crates/core_simd/tests/ops_impl/f64.rs b/crates/core_simd/tests/ops_impl/f64.rs
new file mode 100644
index 000000000000..19ae476bd0e1
--- /dev/null
+++ b/crates/core_simd/tests/ops_impl/f64.rs
@@ -0,0 +1,5 @@
+use super::helpers;
+
+float_tests! { f64x2, f64 }
+float_tests! { f64x4, f64 }
+float_tests! { f64x8, f64 }
diff --git a/crates/core_simd/tests/ops_impl/float_macros.rs b/crates/core_simd/tests/ops_impl/float_macros.rs
new file mode 100644
index 000000000000..ddf3bbbe9369
--- /dev/null
+++ b/crates/core_simd/tests/ops_impl/float_macros.rs
@@ -0,0 +1,237 @@
+macro_rules! float_tests {
+    { $vector:ident, $scalar:ident } => {
+        #[cfg(test)]
+        mod $vector {
+            use super::*;
+            use helpers::lanewise::*;
+
+            // TODO impl this as an associated fn on vectors
+            fn from_slice(slice: &[$scalar]) -> core_simd::$vector {
+                let mut value = core_simd::$vector::default();
+                let value_slice: &mut [_] = value.as_mut();
+                value_slice.copy_from_slice(&slice[0..value_slice.len()]);
+                value
+            }
+
+            const A: [$scalar; 16] = [0.,   1.,  2.,  3.,  4.,  5.,  6.,  7.,  8.,  9., 10., 11., 12., 13., 14., 15.];
+            const B: [$scalar; 16] = [16., 17., 18., 19., 20., 21., 22., 23., 24., 25., 26., 27., 28., 29., 30., 31.];
+
+            #[test]
+            fn add() {
+                let a = from_slice(&A);
+                let b = from_slice(&B);
+                let expected = apply_binary_lanewise(a, b, core::ops::Add::add);
+                assert_biteq!(a + b, expected);
+            }
+
+            #[test]
+            fn add_assign() {
+                let mut a = from_slice(&A);
+                let b = from_slice(&B);
+                let expected = apply_binary_lanewise(a, b, core::ops::Add::add);
+                a += b;
+                assert_biteq!(a, expected);
+            }
+
+            #[test]
+            fn add_scalar_rhs() {
+                let a = from_slice(&A);
+                let b = 5.;
+                let expected = apply_binary_scalar_rhs_lanewise(a, b, core::ops::Add::add);
+                assert_biteq!(a + b, expected);
+            }
+
+            #[test]
+            fn add_scalar_lhs() {
+                let a = 5.;
+                let b = from_slice(&B);
+                let expected = apply_binary_scalar_lhs_lanewise(a, b, core::ops::Add::add);
+                assert_biteq!(a + b, expected);
+            }
+
+            #[test]
+            fn add_assign_scalar() {
+                let mut a = from_slice(&A);
+                let b = 5.;
+                let expected = apply_binary_scalar_rhs_lanewise(a, b, core::ops::Add::add);
+                a += b;
+                assert_biteq!(a, expected);
+            }
+
+            #[test]
+            fn sub() {
+                let a = from_slice(&A);
+                let b = from_slice(&B);
+                let expected = apply_binary_lanewise(a, b, core::ops::Sub::sub);
+                assert_biteq!(a - b, expected);
+            }
+
+            #[test]
+            fn sub_assign() {
+                let mut a = from_slice(&A);
+                let b = from_slice(&B);
+                let expected = apply_binary_lanewise(a, b, core::ops::Sub::sub);
+                a -= b;
+                assert_biteq!(a, expected);
+            }
+
+            #[test]
+            fn sub_scalar_rhs() {
+                let a = from_slice(&A);
+                let b = 5.;
+                let expected = apply_binary_scalar_rhs_lanewise(a, b, core::ops::Sub::sub);
+                assert_biteq!(a - b, expected);
+            }
+
+            #[test]
+            fn sub_scalar_lhs() {
+                let a = 5.;
+                let b = from_slice(&B);
+                let expected = apply_binary_scalar_lhs_lanewise(a, b, core::ops::Sub::sub);
+                assert_biteq!(a - b, expected);
+            }
+
+            #[test]
+            fn sub_assign_scalar() {
+                let mut a = from_slice(&A);
+                let b = 5.;
+                let expected = apply_binary_scalar_rhs_lanewise(a, b, core::ops::Sub::sub);
+                a -= b;
+                assert_biteq!(a, expected);
+            }
+
+            #[test]
+            fn mul() {
+                let a = from_slice(&A);
+                let b = from_slice(&B);
+                let expected = apply_binary_lanewise(a, b, core::ops::Mul::mul);
+                assert_biteq!(a * b, expected);
+            }
+
+            #[test]
+            fn mul_assign() {
+                let mut a = from_slice(&A);
+                let b = from_slice(&B);
+                let expected = apply_binary_lanewise(a, b, core::ops::Mul::mul);
+                a *= b;
+                assert_biteq!(a, expected);
+            }
+
+            #[test]
+            fn mul_scalar_rhs() {
+                let a = from_slice(&A);
+                let b = 5.;
+                let expected = apply_binary_scalar_rhs_lanewise(a, b, core::ops::Mul::mul);
+                assert_biteq!(a * b, expected);
+            }
+
+            #[test]
+            fn mul_scalar_lhs() {
+                let a = 5.;
+                let b = from_slice(&B);
+                let expected = apply_binary_scalar_lhs_lanewise(a, b, core::ops::Mul::mul);
+                assert_biteq!(a * b, expected);
+            }
+
+            #[test]
+            fn mul_assign_scalar() {
+                let mut a = from_slice(&A);
+                let b = 5.;
+                let expected = apply_binary_scalar_rhs_lanewise(a, b, core::ops::Mul::mul);
+                a *= b;
+                assert_biteq!(a, expected);
+            }
+
+            #[test]
+            fn div() {
+                let a = from_slice(&A);
+                let b = from_slice(&B);
+                let expected = apply_binary_lanewise(a, b, core::ops::Div::div);
+                assert_biteq!(a / b, expected);
+            }
+
+            #[test]
+            fn div_assign() {
+                let mut a = from_slice(&A);
+                let b = from_slice(&B);
+                let expected = apply_binary_lanewise(a, b, core::ops::Div::div);
+                a /= b;
+                assert_biteq!(a, expected);
+            }
+
+            #[test]
+            fn div_scalar_rhs() {
+                let a = from_slice(&A);
+                let b = 5.;
+                let expected = apply_binary_scalar_rhs_lanewise(a, b, core::ops::Div::div);
+                assert_biteq!(a / b, expected);
+            }
+
+            #[test]
+            fn div_scalar_lhs() {
+                let a = 5.;
+                let b = from_slice(&B);
+                let expected = apply_binary_scalar_lhs_lanewise(a, b, core::ops::Div::div);
+                assert_biteq!(a / b, expected);
+            }
+
+            #[test]
+            fn div_assign_scalar() {
+                let mut a = from_slice(&A);
+                let b = 5.;
+                let expected = apply_binary_scalar_rhs_lanewise(a, b, core::ops::Div::div);
+                a /= b;
+                assert_biteq!(a, expected);
+            }
+
+            #[test]
+            fn rem() {
+                let a = from_slice(&A);
+                let b = from_slice(&B);
+                let expected = apply_binary_lanewise(a, b, core::ops::Rem::rem);
+                assert_biteq!(a % b, expected);
+            }
+
+            #[test]
+            fn rem_assign() {
+                let mut a = from_slice(&A);
+                let b = from_slice(&B);
+                let expected = apply_binary_lanewise(a, b, core::ops::Rem::rem);
+                a %= b;
+                assert_biteq!(a, expected);
+            }
+
+            #[test]
+            fn rem_scalar_rhs() {
+                let a = from_slice(&A);
+                let b = 5.;
+                let expected = apply_binary_scalar_rhs_lanewise(a, b, core::ops::Rem::rem);
+                assert_biteq!(a % b, expected);
+            }
+
+            #[test]
+            fn rem_scalar_lhs() {
+                let a = 5.;
+                let b = from_slice(&B);
+                let expected = apply_binary_scalar_lhs_lanewise(a, b, core::ops::Rem::rem);
+                assert_biteq!(a % b, expected);
+            }
+
+            #[test]
+            fn rem_assign_scalar() {
+                let mut a = from_slice(&A);
+                let b = 5.;
+                let expected = apply_binary_scalar_rhs_lanewise(a, b, core::ops::Rem::rem);
+                a %= b;
+                assert_biteq!(a, expected);
+            }
+
+            #[test]
+            fn neg() {
+                let v = from_slice(&A);
+                let expected = apply_unary_lanewise(v, core::ops::Neg::neg);
+                assert_biteq!(-v, expected);
+            }
+        }
+    }
+}
diff --git a/crates/core_simd/tests/ops_impl/i128.rs b/crates/core_simd/tests/ops_impl/i128.rs
new file mode 100644
index 000000000000..8a0a279b8dce
--- /dev/null
+++ b/crates/core_simd/tests/ops_impl/i128.rs
@@ -0,0 +1,4 @@
+use super::helpers;
+
+int_tests! { i128x2, i128 }
+int_tests! { i128x4, i128 }
diff --git a/crates/core_simd/tests/ops_impl/i16.rs b/crates/core_simd/tests/ops_impl/i16.rs
new file mode 100644
index 000000000000..445436b77a89
--- /dev/null
+++ b/crates/core_simd/tests/ops_impl/i16.rs
@@ -0,0 +1,6 @@
+use super::helpers;
+
+int_tests! { i16x4, i16 }
+int_tests! { i16x8, i16 }
+int_tests! { i16x16, i16 }
+int_tests! { i16x32, i16 }
diff --git a/crates/core_simd/tests/ops_impl/i32.rs b/crates/core_simd/tests/ops_impl/i32.rs
new file mode 100644
index 000000000000..f13ab833a335
--- /dev/null
+++ b/crates/core_simd/tests/ops_impl/i32.rs
@@ -0,0 +1,6 @@
+use super::helpers;
+
+int_tests! { i32x2, i32 }
+int_tests! { i32x4, i32 }
+int_tests! { i32x8, i32 }
+int_tests! { i32x16, i32 }
diff --git a/crates/core_simd/tests/ops_impl/i64.rs b/crates/core_simd/tests/ops_impl/i64.rs
new file mode 100644
index 000000000000..08479c4b994b
--- /dev/null
+++ b/crates/core_simd/tests/ops_impl/i64.rs
@@ -0,0 +1,5 @@
+use super::helpers;
+
+int_tests! { i64x2, i64 }
+int_tests! { i64x4, i64 }
+int_tests! { i64x8, i64 }
diff --git a/crates/core_simd/tests/ops_impl/i8.rs b/crates/core_simd/tests/ops_impl/i8.rs
new file mode 100644
index 000000000000..2a7db7906ac1
--- /dev/null
+++ b/crates/core_simd/tests/ops_impl/i8.rs
@@ -0,0 +1,6 @@
+use super::helpers;
+
+int_tests! { i8x8, i8 }
+int_tests! { i8x16, i8 }
+int_tests! { i8x32, i8 }
+int_tests! { i8x64, i8 }
diff --git a/crates/core_simd/tests/ops_impl/int_macros.rs b/crates/core_simd/tests/ops_impl/int_macros.rs
new file mode 100644
index 000000000000..4175541e892d
--- /dev/null
+++ b/crates/core_simd/tests/ops_impl/int_macros.rs
@@ -0,0 +1,388 @@
+macro_rules! int_tests {
+    { $vector:ident, $scalar:ident } => {
+        #[cfg(test)]
+        mod $vector {
+            use super::*;
+            use helpers::lanewise::*;
+
+            // TODO impl this as an associated fn on vectors
+            fn from_slice(slice: &[$scalar]) -> core_simd::$vector {
+                let mut value = core_simd::$vector::default();
+                let value_slice: &mut [_] = value.as_mut();
+                value_slice.copy_from_slice(&slice[0..value_slice.len()]);
+                value
+            }
+
+            const A: [$scalar; 64] = [
+                7, 7, 7, 7, -7, -7, -7, -7,
+                6, 6, 6, 6, -6, -6, -6, -6,
+                5, 5, 5, 5, -5, -5, -5, -5,
+                4, 4, 4, 4, -4, -4, -4, -4,
+                3, 3, 3, 3, -3, -3, -3, -3,
+                2, 2, 2, 2, -2, -2, -2, -2,
+                1, 1, 1, 1, -1, -1, -1, -1,
+                0, 0, 0, 0,  0,  0,  0,  0,
+            ];
+            const B: [$scalar; 64] = [
+                 1,  2,  3,  4,  5,  6,  7,  8,
+                 1,  2,  3,  4,  5,  6,  7,  8,
+                 1,  2,  3,  4,  5,  6,  7,  8,
+                 1,  2,  3,  4,  5,  6,  7,  8,
+                 -1,  -2,  -3,  -4,  -5,  -6,  -7,  -8,
+                 -1,  -2,  -3,  -4,  -5,  -6,  -7,  -8,
+                 -1,  -2,  -3,  -4,  -5,  -6,  -7,  -8,
+                 -1,  -2,  -3,  -4,  -5,  -6,  -7,  -8,
+            ];
+
+            #[test]
+            fn add() {
+                let a = from_slice(&A);
+                let b = from_slice(&B);
+                let expected = apply_binary_lanewise(a, b, core::ops::Add::add);
+                assert_biteq!(a + b, expected);
+            }
+
+            #[test]
+            fn add_assign() {
+                let mut a = from_slice(&A);
+                let b = from_slice(&B);
+                let expected = apply_binary_lanewise(a, b, core::ops::Add::add);
+                a += b;
+                assert_biteq!(a, expected);
+            }
+
+            #[test]
+            fn add_scalar_rhs() {
+                let a = from_slice(&A);
+                let b = 5;
+                let expected = apply_binary_scalar_rhs_lanewise(a, b, core::ops::Add::add);
+                assert_biteq!(a + b, expected);
+            }
+
+            #[test]
+            fn add_scalar_lhs() {
+                let a = 5;
+                let b = from_slice(&B);
+                let expected = apply_binary_scalar_lhs_lanewise(a, b, core::ops::Add::add);
+                assert_biteq!(a + b, expected);
+            }
+
+            #[test]
+            fn add_assign_scalar() {
+                let mut a = from_slice(&A);
+                let b = 5;
+                let expected = apply_binary_scalar_rhs_lanewise(a, b, core::ops::Add::add);
+                a += b;
+                assert_biteq!(a, expected);
+            }
+
+            #[test]
+            fn sub() {
+                let a = from_slice(&A);
+                let b = from_slice(&B);
+                let expected = apply_binary_lanewise(a, b, core::ops::Sub::sub);
+                assert_biteq!(a - b, expected);
+            }
+
+            #[test]
+            fn sub_assign() {
+                let mut a = from_slice(&A);
+                let b = from_slice(&B);
+                let expected = apply_binary_lanewise(a, b, core::ops::Sub::sub);
+                a -= b;
+                assert_biteq!(a, expected);
+            }
+
+            #[test]
+            fn sub_scalar_rhs() {
+                let a = from_slice(&A);
+                let b = 5;
+                let expected = apply_binary_scalar_rhs_lanewise(a, b, core::ops::Sub::sub);
+                assert_biteq!(a - b, expected);
+            }
+
+            #[test]
+            fn sub_scalar_lhs() {
+                let a = 5;
+                let b = from_slice(&B);
+                let expected = apply_binary_scalar_lhs_lanewise(a, b, core::ops::Sub::sub);
+                assert_biteq!(a - b, expected);
+            }
+
+            #[test]
+            fn sub_assign_scalar() {
+                let mut a = from_slice(&A);
+                let b = 5;
+                let expected = apply_binary_scalar_rhs_lanewise(a, b, core::ops::Sub::sub);
+                a -= b;
+                assert_biteq!(a, expected);
+            }
+
+            #[test]
+            fn mul() {
+                let a = from_slice(&A);
+                let b = from_slice(&B);
+                let expected = apply_binary_lanewise(a, b, core::ops::Mul::mul);
+                assert_biteq!(a * b, expected);
+            }
+
+            #[test]
+            fn mul_assign() {
+                let mut a = from_slice(&A);
+                let b = from_slice(&B);
+                let expected = apply_binary_lanewise(a, b, core::ops::Mul::mul);
+                a *= b;
+                assert_biteq!(a, expected);
+            }
+
+            #[test]
+            fn mul_scalar_rhs() {
+                let a = from_slice(&A);
+                let b = 5;
+                let expected = apply_binary_scalar_rhs_lanewise(a, b, core::ops::Mul::mul);
+                assert_biteq!(a * b, expected);
+            }
+
+            #[test]
+            fn mul_scalar_lhs() {
+                let a = 5;
+                let b = from_slice(&B);
+                let expected = apply_binary_scalar_lhs_lanewise(a, b, core::ops::Mul::mul);
+                assert_biteq!(a * b, expected);
+            }
+
+            #[test]
+            fn mul_assign_scalar() {
+                let mut a = from_slice(&A);
+                let b = 5;
+                let expected = apply_binary_scalar_rhs_lanewise(a, b, core::ops::Mul::mul);
+                a *= b;
+                assert_biteq!(a, expected);
+            }
+
+            #[test]
+            fn div() {
+                let a = from_slice(&A);
+                let b = from_slice(&B);
+                let expected = apply_binary_lanewise(a, b, core::ops::Div::div);
+                assert_biteq!(a / b, expected);
+            }
+
+            #[test]
+            fn div_assign() {
+                let mut a = from_slice(&A);
+                let b = from_slice(&B);
+                let expected = apply_binary_lanewise(a, b, core::ops::Div::div);
+                a /= b;
+                assert_biteq!(a, expected);
+            }
+
+            #[test]
+            fn div_scalar_rhs() {
+                let a = from_slice(&A);
+                let b = 5;
+                let expected = apply_binary_scalar_rhs_lanewise(a, b, core::ops::Div::div);
+                assert_biteq!(a / b, expected);
+            }
+
+            #[test]
+            fn div_scalar_lhs() {
+                let a = 5;
+                let b = from_slice(&B);
+                let expected = apply_binary_scalar_lhs_lanewise(a, b, core::ops::Div::div);
+                assert_biteq!(a / b, expected);
+            }
+
+            #[test]
+            fn div_assign_scalar() {
+                let mut a = from_slice(&A);
+                let b = 5;
+                let expected = apply_binary_scalar_rhs_lanewise(a, b, core::ops::Div::div);
+                a /= b;
+                assert_biteq!(a, expected);
+            }
+
+            #[test]
+            fn rem() {
+                let a = from_slice(&A);
+                let b = from_slice(&B);
+                let expected = apply_binary_lanewise(a, b, core::ops::Rem::rem);
+                assert_biteq!(a % b, expected);
+            }
+
+            #[test]
+            fn rem_assign() {
+                let mut a = from_slice(&A);
+                let b = from_slice(&B);
+                let expected = apply_binary_lanewise(a, b, core::ops::Rem::rem);
+                a %= b;
+                assert_biteq!(a, expected);
+            }
+
+            #[test]
+            fn rem_scalar_rhs() {
+                let a = from_slice(&A);
+                let b = 5;
+                let expected = apply_binary_scalar_rhs_lanewise(a, b, core::ops::Rem::rem);
+                assert_biteq!(a % b, expected);
+            }
+
+            #[test]
+            fn rem_scalar_lhs() {
+                let a = 5;
+                let b = from_slice(&B);
+                let expected = apply_binary_scalar_lhs_lanewise(a, b, core::ops::Rem::rem);
+                assert_biteq!(a % b, expected);
+            }
+
+            #[test]
+            fn rem_assign_scalar() {
+                let mut a = from_slice(&A);
+                let b = 5;
+                let expected = apply_binary_scalar_rhs_lanewise(a, b, core::ops::Rem::rem);
+                a %= b;
+                assert_biteq!(a, expected);
+            }
+
+            #[test]
+            fn bitand() {
+                let a = from_slice(&A);
+                let b = from_slice(&B);
+                let expected = apply_binary_lanewise(a, b, core::ops::BitAnd::bitand);
+                assert_biteq!(a & b, expected);
+            }
+
+            #[test]
+            fn bitand_assign() {
+                let mut a = from_slice(&A);
+                let b = from_slice(&B);
+                let expected = apply_binary_lanewise(a, b, core::ops::BitAnd::bitand);
+                a &= b;
+                assert_biteq!(a, expected);
+            }
+
+            #[test]
+            fn bitand_scalar_rhs() {
+                let a = from_slice(&A);
+                let b = 5;
+                let expected = apply_binary_scalar_rhs_lanewise(a, b, core::ops::BitAnd::bitand);
+                assert_biteq!(a & b, expected);
+            }
+
+            #[test]
+            fn bitand_scalar_lhs() {
+                let a = 5;
+                let b = from_slice(&B);
+                let expected = apply_binary_scalar_lhs_lanewise(a, b, core::ops::BitAnd::bitand);
+                assert_biteq!(a & b, expected);
+            }
+
+            #[test]
+            fn bitand_assign_scalar() {
+                let mut a = from_slice(&A);
+                let b = 5;
+                let expected = apply_binary_scalar_rhs_lanewise(a, b, core::ops::BitAnd::bitand);
+                a &= b;
+                assert_biteq!(a, expected);
+            }
+
+            #[test]
+            fn bitor() {
+                let a = from_slice(&A);
+                let b = from_slice(&B);
+                let expected = apply_binary_lanewise(a, b, core::ops::BitOr::bitor);
+                assert_biteq!(a | b, expected);
+            }
+
+            #[test]
+            fn bitor_assign() {
+                let mut a = from_slice(&A);
+                let b = from_slice(&B);
+                let expected = apply_binary_lanewise(a, b, core::ops::BitOr::bitor);
+                a |= b;
+                assert_biteq!(a, expected);
+            }
+
+            #[test]
+            fn bitor_scalar_rhs() {
+                let a = from_slice(&A);
+                let b = 5;
+                let expected = apply_binary_scalar_rhs_lanewise(a, b, core::ops::BitOr::bitor);
+                assert_biteq!(a | b, expected);
+            }
+
+            #[test]
+            fn bitor_scalar_lhs() {
+                let a = 5;
+                let b = from_slice(&B);
+                let expected = apply_binary_scalar_lhs_lanewise(a, b, core::ops::BitOr::bitor);
+                assert_biteq!(a | b, expected);
+            }
+
+            #[test]
+            fn bitor_assign_scalar() {
+                let mut a = from_slice(&A);
+                let b = 5;
+                let expected = apply_binary_scalar_rhs_lanewise(a, b, core::ops::BitOr::bitor);
+                a |= b;
+                assert_biteq!(a, expected);
+            }
+
+            #[test]
+            fn bitxor() {
+                let a = from_slice(&A);
+                let b = from_slice(&B);
+                let expected = apply_binary_lanewise(a, b, core::ops::BitXor::bitxor);
+                assert_biteq!(a ^ b, expected);
+            }
+
+            #[test]
+            fn bitxor_assign() {
+                let mut a = from_slice(&A);
+                let b = from_slice(&B);
+                let expected = apply_binary_lanewise(a, b, core::ops::BitXor::bitxor);
+                a ^= b;
+                assert_biteq!(a, expected);
+            }
+
+            #[test]
+            fn bitxor_scalar_rhs() {
+                let a = from_slice(&A);
+                let b = 5;
+                let expected = apply_binary_scalar_rhs_lanewise(a, b, core::ops::BitXor::bitxor);
+                assert_biteq!(a ^ b, expected);
+            }
+
+            #[test]
+            fn bitxor_scalar_lhs() {
+                let a = 5;
+                let b = from_slice(&B);
+                let expected = apply_binary_scalar_lhs_lanewise(a, b, core::ops::BitXor::bitxor);
+                assert_biteq!(a ^ b, expected);
+            }
+
+            #[test]
+            fn bitxor_assign_scalar() {
+                let mut a = from_slice(&A);
+                let b = 5;
+                let expected = apply_binary_scalar_rhs_lanewise(a, b, core::ops::BitXor::bitxor);
+                a ^= b;
+                assert_biteq!(a, expected);
+            }
+
+            #[test]
+            fn neg() {
+                let v = from_slice(&A);
+                let expected = apply_unary_lanewise(v, core::ops::Neg::neg);
+                assert_biteq!(-v, expected);
+            }
+
+            #[test]
+            fn not() {
+                let v = from_slice(&A);
+                let expected = apply_unary_lanewise(v, core::ops::Not::not);
+                assert_biteq!(!v, expected);
+            }
+        }
+    }
+}
diff --git a/crates/core_simd/tests/ops_impl/isize.rs b/crates/core_simd/tests/ops_impl/isize.rs
new file mode 100644
index 000000000000..9943e9c357a4
--- /dev/null
+++ b/crates/core_simd/tests/ops_impl/isize.rs
@@ -0,0 +1,5 @@
+use super::helpers;
+
+int_tests! { isizex2, isize }
+int_tests! { isizex4, isize }
+int_tests! { isizex8, isize }
diff --git a/crates/core_simd/tests/ops_impl/mask128.rs b/crates/core_simd/tests/ops_impl/mask128.rs
new file mode 100644
index 000000000000..f0bcdb4d4df9
--- /dev/null
+++ b/crates/core_simd/tests/ops_impl/mask128.rs
@@ -0,0 +1,4 @@
+use super::helpers;
+
+mask_tests! { mask128x2, mask128 }
+mask_tests! { mask128x4, mask128 }
diff --git a/crates/core_simd/tests/ops_impl/mask16.rs b/crates/core_simd/tests/ops_impl/mask16.rs
new file mode 100644
index 000000000000..6f3f8e0ee02e
--- /dev/null
+++ b/crates/core_simd/tests/ops_impl/mask16.rs
@@ -0,0 +1,6 @@
+use super::helpers;
+
+mask_tests! { mask16x4, mask16 }
+mask_tests! { mask16x8, mask16 }
+mask_tests! { mask16x16, mask16 }
+mask_tests! { mask16x32, mask16 }
diff --git a/crates/core_simd/tests/ops_impl/mask32.rs b/crates/core_simd/tests/ops_impl/mask32.rs
new file mode 100644
index 000000000000..5c35885a2f5b
--- /dev/null
+++ b/crates/core_simd/tests/ops_impl/mask32.rs
@@ -0,0 +1,6 @@
+use super::helpers;
+
+mask_tests! { mask32x2, mask32 }
+mask_tests! { mask32x4, mask32 }
+mask_tests! { mask32x8, mask32 }
+mask_tests! { mask32x16, mask32 }
diff --git a/crates/core_simd/tests/ops_impl/mask64.rs b/crates/core_simd/tests/ops_impl/mask64.rs
new file mode 100644
index 000000000000..88d3211465c5
--- /dev/null
+++ b/crates/core_simd/tests/ops_impl/mask64.rs
@@ -0,0 +1,5 @@
+use super::helpers;
+
+mask_tests! { mask64x2, mask64 }
+mask_tests! { mask64x4, mask64 }
+mask_tests! { mask64x8, mask64 }
diff --git a/crates/core_simd/tests/ops_impl/mask8.rs b/crates/core_simd/tests/ops_impl/mask8.rs
new file mode 100644
index 000000000000..fa4bcf09f367
--- /dev/null
+++ b/crates/core_simd/tests/ops_impl/mask8.rs
@@ -0,0 +1,6 @@
+use super::helpers;
+
+mask_tests! { mask8x8, mask8 }
+mask_tests! { mask8x16, mask8 }
+mask_tests! { mask8x32, mask8 }
+mask_tests! { mask8x64, mask8 }
diff --git a/crates/core_simd/tests/ops_impl/mask_macros.rs b/crates/core_simd/tests/ops_impl/mask_macros.rs
new file mode 100644
index 000000000000..9d6bc0cd692f
--- /dev/null
+++ b/crates/core_simd/tests/ops_impl/mask_macros.rs
@@ -0,0 +1,179 @@
+macro_rules! mask_tests {
+    { $vector:ident, $scalar:ident } => {
+        #[cfg(test)]
+        mod $vector {
+            use super::*;
+            use helpers::lanewise::*;
+
+            fn from_slice(slice: &[bool]) -> core_simd::$vector {
+                let mut value = core_simd::$vector::default();
+                let value_slice: &mut [_] = value.as_mut();
+                for (m, b) in value_slice.iter_mut().zip(slice.iter()) {
+                    *m = (*b).into();
+                }
+                value
+            }
+
+            const A: [bool; 64] = [
+                false, true, false, true, false, false, true, true,
+                false, true, false, true, false, false, true, true,
+                false, true, false, true, false, false, true, true,
+                false, true, false, true, false, false, true, true,
+                false, true, false, true, false, false, true, true,
+                false, true, false, true, false, false, true, true,
+                false, true, false, true, false, false, true, true,
+                false, true, false, true, false, false, true, true,
+            ];
+            const B: [bool; 64] = [
+                false, false, true, true, false, true, false, true,
+                false, false, true, true, false, true, false, true,
+                false, false, true, true, false, true, false, true,
+                false, false, true, true, false, true, false, true,
+                false, false, true, true, false, true, false, true,
+                false, false, true, true, false, true, false, true,
+                false, false, true, true, false, true, false, true,
+                false, false, true, true, false, true, false, true,
+            ];
+
+            const SET_SCALAR: core_simd::$scalar = core_simd::$scalar::new(true);
+            const UNSET_SCALAR: core_simd::$scalar = core_simd::$scalar::new(false);
+            const SET_VECTOR: core_simd::$vector = core_simd::$vector::splat(SET_SCALAR);
+            const UNSET_VECTOR: core_simd::$vector = core_simd::$vector::splat(UNSET_SCALAR);
+
+            #[test]
+            fn bitand() {
+                let a = from_slice(&A);
+                let b = from_slice(&B);
+                let expected = apply_binary_lanewise(a, b, core::ops::BitAnd::bitand);
+                assert_biteq!(a & b, expected);
+            }
+
+            #[test]
+            fn bitand_assign() {
+                let mut a = from_slice(&A);
+                let b = from_slice(&B);
+                let expected = apply_binary_lanewise(a, b, core::ops::BitAnd::bitand);
+                a &= b;
+                assert_biteq!(a, expected);
+            }
+
+            #[test]
+            fn bitand_scalar_rhs() {
+                let a = from_slice(&A);
+                let expected = a;
+                assert_biteq!(a & SET_SCALAR, expected);
+                assert_biteq!(a & UNSET_SCALAR, UNSET_VECTOR);
+            }
+
+            #[test]
+            fn bitand_scalar_lhs() {
+                let a = from_slice(&A);
+                let expected = a;
+                assert_biteq!(SET_SCALAR & a, expected);
+                assert_biteq!(UNSET_SCALAR & a, UNSET_VECTOR);
+            }
+
+            #[test]
+            fn bitand_assign_scalar() {
+                let mut a = from_slice(&A);
+                let expected = a;
+                a &= SET_SCALAR;
+                assert_biteq!(a, expected);
+                a &= UNSET_SCALAR;
+                assert_biteq!(a, UNSET_VECTOR);
+            }
+
+            #[test]
+            fn bitor() {
+                let a = from_slice(&A);
+                let b = from_slice(&B);
+                let expected = apply_binary_lanewise(a, b, core::ops::BitOr::bitor);
+                assert_biteq!(a | b, expected);
+            }
+
+            #[test]
+            fn bitor_assign() {
+                let mut a = from_slice(&A);
+                let b = from_slice(&B);
+                let expected = apply_binary_lanewise(a, b, core::ops::BitOr::bitor);
+                a |= b;
+                assert_biteq!(a, expected);
+            }
+
+            #[test]
+            fn bitor_scalar_rhs() {
+                let a = from_slice(&A);
+                assert_biteq!(a | UNSET_SCALAR, a);
+                assert_biteq!(a | SET_SCALAR, SET_VECTOR);
+            }
+
+            #[test]
+            fn bitor_scalar_lhs() {
+                let a = from_slice(&A);
+                assert_biteq!(UNSET_SCALAR | a, a);
+                assert_biteq!(SET_SCALAR | a, SET_VECTOR);
+            }
+
+            #[test]
+            fn bitor_assign_scalar() {
+                let mut a = from_slice(&A);
+                let expected = a;
+                a |= UNSET_SCALAR;
+                assert_biteq!(a, expected);
+                a |= SET_SCALAR;
+                assert_biteq!(a, SET_VECTOR);
+            }
+
+            #[test]
+            fn bitxor() {
+                let a = from_slice(&A);
+                let b = from_slice(&B);
+                let expected = apply_binary_lanewise(a, b, core::ops::BitXor::bitxor);
+                assert_biteq!(a ^ b, expected);
+            }
+
+            #[test]
+            fn bitxor_assign() {
+                let mut a = from_slice(&A);
+                let b = from_slice(&B);
+                let expected = apply_binary_lanewise(a, b, core::ops::BitXor::bitxor);
+                a ^= b;
+                assert_biteq!(a, expected);
+            }
+
+            #[test]
+            fn bitxor_scalar_rhs() {
+                let a = from_slice(&A);
+                let expected = apply_binary_scalar_rhs_lanewise(a, SET_SCALAR, core::ops::BitXor::bitxor);
+                assert_biteq!(a ^ UNSET_SCALAR, a);
+                assert_biteq!(a ^ SET_SCALAR, expected);
+            }
+
+            #[test]
+            fn bitxor_scalar_lhs() {
+                let a = from_slice(&A);
+                let expected = apply_binary_scalar_lhs_lanewise(SET_SCALAR, a, core::ops::BitXor::bitxor);
+                assert_biteq!(UNSET_SCALAR ^ a, a);
+                assert_biteq!(SET_SCALAR ^ a, expected);
+            }
+
+            #[test]
+            fn bitxor_assign_scalar() {
+                let mut a = from_slice(&A);
+                let expected_unset = a;
+                let expected_set = apply_binary_scalar_rhs_lanewise(a, SET_SCALAR, core::ops::BitXor::bitxor);
+                a ^= UNSET_SCALAR;
+                assert_biteq!(a, expected_unset);
+                a ^= SET_SCALAR;
+                assert_biteq!(a, expected_set);
+            }
+
+            #[test]
+            fn not() {
+                let v = from_slice(&A);
+                let expected = apply_unary_lanewise(v, core::ops::Not::not);
+                assert_biteq!(!v, expected);
+            }
+        }
+    }
+}
diff --git a/crates/core_simd/tests/ops_impl/masksize.rs b/crates/core_simd/tests/ops_impl/masksize.rs
new file mode 100644
index 000000000000..76e333f3c154
--- /dev/null
+++ b/crates/core_simd/tests/ops_impl/masksize.rs
@@ -0,0 +1,5 @@
+use super::helpers;
+
+mask_tests! { masksizex2, masksize }
+mask_tests! { masksizex4, masksize }
+mask_tests! { masksizex8, masksize }
diff --git a/crates/core_simd/tests/ops_impl/mod.rs b/crates/core_simd/tests/ops_impl/mod.rs
new file mode 100644
index 000000000000..814f2d04b59c
--- /dev/null
+++ b/crates/core_simd/tests/ops_impl/mod.rs
@@ -0,0 +1,39 @@
+#[macro_use]
+#[path = "../helpers/mod.rs"]
+mod helpers;
+
+#[macro_use]
+mod float_macros;
+
+mod r#f32;
+mod r#f64;
+
+#[macro_use]
+mod int_macros;
+
+mod r#i8;
+mod r#i16;
+mod r#i32;
+mod r#i64;
+mod r#i128;
+mod r#isize;
+
+#[macro_use]
+mod uint_macros;
+
+mod r#u8;
+mod r#u16;
+mod r#u32;
+mod r#u64;
+mod r#u128;
+mod r#usize;
+
+#[macro_use]
+mod mask_macros;
+
+mod mask8;
+mod mask16;
+mod mask32;
+mod mask64;
+mod mask128;
+mod masksize;
diff --git a/crates/core_simd/tests/ops_impl/u128.rs b/crates/core_simd/tests/ops_impl/u128.rs
new file mode 100644
index 000000000000..cfd849640ffe
--- /dev/null
+++ b/crates/core_simd/tests/ops_impl/u128.rs
@@ -0,0 +1,4 @@
+use super::helpers;
+
+uint_tests! { u128x2, u128 }
+uint_tests! { u128x4, u128 }
diff --git a/crates/core_simd/tests/ops_impl/u16.rs b/crates/core_simd/tests/ops_impl/u16.rs
new file mode 100644
index 000000000000..50af4dd48b38
--- /dev/null
+++ b/crates/core_simd/tests/ops_impl/u16.rs
@@ -0,0 +1,6 @@
+use super::helpers;
+
+uint_tests! { u16x4, u16 }
+uint_tests! { u16x8, u16 }
+uint_tests! { u16x16, u16 }
+uint_tests! { u16x32, u16 }
diff --git a/crates/core_simd/tests/ops_impl/u32.rs b/crates/core_simd/tests/ops_impl/u32.rs
new file mode 100644
index 000000000000..8e7faa9d7403
--- /dev/null
+++ b/crates/core_simd/tests/ops_impl/u32.rs
@@ -0,0 +1,6 @@
+use super::helpers;
+
+uint_tests! { u32x2, u32 }
+uint_tests! { u32x4, u32 }
+uint_tests! { u32x8, u32 }
+uint_tests! { u32x16, u32 }
diff --git a/crates/core_simd/tests/ops_impl/u64.rs b/crates/core_simd/tests/ops_impl/u64.rs
new file mode 100644
index 000000000000..1a6385d37bae
--- /dev/null
+++ b/crates/core_simd/tests/ops_impl/u64.rs
@@ -0,0 +1,5 @@
+use super::helpers;
+
+uint_tests! { u64x2, u64 }
+uint_tests! { u64x4, u64 }
+uint_tests! { u64x8, u64 }
diff --git a/crates/core_simd/tests/ops_impl/u8.rs b/crates/core_simd/tests/ops_impl/u8.rs
new file mode 100644
index 000000000000..31568b1eacbe
--- /dev/null
+++ b/crates/core_simd/tests/ops_impl/u8.rs
@@ -0,0 +1,6 @@
+use super::helpers;
+
+uint_tests! { u8x8, u8 }
+uint_tests! { u8x16, u8 }
+uint_tests! { u8x32, u8 }
+uint_tests! { u8x64, u8 }
diff --git a/crates/core_simd/tests/ops_impl/uint_macros.rs b/crates/core_simd/tests/ops_impl/uint_macros.rs
new file mode 100644
index 000000000000..eb9ac34d7efd
--- /dev/null
+++ b/crates/core_simd/tests/ops_impl/uint_macros.rs
@@ -0,0 +1,381 @@
+macro_rules! uint_tests {
+    { $vector:ident, $scalar:ident } => {
+        #[cfg(test)]
+        mod $vector {
+            use super::*;
+            use helpers::lanewise::*;
+
+            // TODO impl this as an associated fn on vectors
+            fn from_slice(slice: &[$scalar]) -> core_simd::$vector {
+                let mut value = core_simd::$vector::default();
+                let value_slice: &mut [_] = value.as_mut();
+                value_slice.copy_from_slice(&slice[0..value_slice.len()]);
+                value
+            }
+
+            const A: [$scalar; 64] = [
+                16, 16, 16, 16, 16, 16, 16, 16,
+                14, 14, 14, 14, 14, 14, 14, 14,
+                12, 12, 12, 12, 12, 12, 12, 12,
+                10, 10, 10, 10, 10, 10, 10, 10,
+                 8,  8,  8,  8,  8,  8,  8,  8,
+                 6,  6,  6,  6,  6,  6,  7,  8,
+                 4,  4,  4,  4,  5,  6,  7,  8,
+                 1,  2,  3,  4,  5,  6,  7,  8,
+            ];
+            const B: [$scalar; 64] = [
+                1, 2, 3, 4, 1, 2, 3, 4,
+                1, 2, 3, 4, 5, 6, 7, 8,
+                1, 2, 3, 4, 5, 6, 7, 8,
+                1, 2, 3, 4, 5, 6, 7, 8,
+                1, 2, 3, 4, 5, 6, 7, 8,
+                1, 2, 3, 4, 5, 6, 7, 8,
+                1, 2, 3, 4, 5, 6, 7, 8,
+                1, 2, 3, 4, 5, 6, 7, 8,
+            ];
+
+            #[test]
+            fn add() {
+                let a = from_slice(&A);
+                let b = from_slice(&B);
+                let expected = apply_binary_lanewise(a, b, core::ops::Add::add);
+                assert_biteq!(a + b, expected);
+            }
+
+            #[test]
+            fn add_assign() {
+                let mut a = from_slice(&A);
+                let b = from_slice(&B);
+                let expected = apply_binary_lanewise(a, b, core::ops::Add::add);
+                a += b;
+                assert_biteq!(a, expected);
+            }
+
+            #[test]
+            fn add_scalar_rhs() {
+                let a = from_slice(&A);
+                let b = 5;
+                let expected = apply_binary_scalar_rhs_lanewise(a, b, core::ops::Add::add);
+                assert_biteq!(a + b, expected);
+            }
+
+            #[test]
+            fn add_scalar_lhs() {
+                let a = 5;
+                let b = from_slice(&B);
+                let expected = apply_binary_scalar_lhs_lanewise(a, b, core::ops::Add::add);
+                assert_biteq!(a + b, expected);
+            }
+
+            #[test]
+            fn add_assign_scalar() {
+                let mut a = from_slice(&A);
+                let b = 5;
+                let expected = apply_binary_scalar_rhs_lanewise(a, b, core::ops::Add::add);
+                a += b;
+                assert_biteq!(a, expected);
+            }
+
+            #[test]
+            fn sub() {
+                let a = from_slice(&A);
+                let b = from_slice(&B);
+                let expected = apply_binary_lanewise(a, b, core::ops::Sub::sub);
+                assert_biteq!(a - b, expected);
+            }
+
+            #[test]
+            fn sub_assign() {
+                let mut a = from_slice(&A);
+                let b = from_slice(&B);
+                let expected = apply_binary_lanewise(a, b, core::ops::Sub::sub);
+                a -= b;
+                assert_biteq!(a, expected);
+            }
+
+            #[test]
+            fn sub_scalar_rhs() {
+                let a = from_slice(&A);
+                let b = 1;
+                let expected = apply_binary_scalar_rhs_lanewise(a, b, core::ops::Sub::sub);
+                assert_biteq!(a - b, expected);
+            }
+
+            #[test]
+            fn sub_scalar_lhs() {
+                let a = 40;
+                let b = from_slice(&B);
+                let expected = apply_binary_scalar_lhs_lanewise(a, b, core::ops::Sub::sub);
+                assert_biteq!(a - b, expected);
+            }
+
+            #[test]
+            fn sub_assign_scalar() {
+                let mut a = from_slice(&A);
+                let b = 1;
+                let expected = apply_binary_scalar_rhs_lanewise(a, b, core::ops::Sub::sub);
+                a -= b;
+                assert_biteq!(a, expected);
+            }
+
+            #[test]
+            fn mul() {
+                let a = from_slice(&A);
+                let b = from_slice(&B);
+                let expected = apply_binary_lanewise(a, b, core::ops::Mul::mul);
+                assert_biteq!(a * b, expected);
+            }
+
+            #[test]
+            fn mul_assign() {
+                let mut a = from_slice(&A);
+                let b = from_slice(&B);
+                let expected = apply_binary_lanewise(a, b, core::ops::Mul::mul);
+                a *= b;
+                assert_biteq!(a, expected);
+            }
+
+            #[test]
+            fn mul_scalar_rhs() {
+                let a = from_slice(&A);
+                let b = 5;
+                let expected = apply_binary_scalar_rhs_lanewise(a, b, core::ops::Mul::mul);
+                assert_biteq!(a * b, expected);
+            }
+
+            #[test]
+            fn mul_scalar_lhs() {
+                let a = 5;
+                let b = from_slice(&B);
+                let expected = apply_binary_scalar_lhs_lanewise(a, b, core::ops::Mul::mul);
+                assert_biteq!(a * b, expected);
+            }
+
+            #[test]
+            fn mul_assign_scalar() {
+                let mut a = from_slice(&A);
+                let b = 5;
+                let expected = apply_binary_scalar_rhs_lanewise(a, b, core::ops::Mul::mul);
+                a *= b;
+                assert_biteq!(a, expected);
+            }
+
+            #[test]
+            fn div() {
+                let a = from_slice(&A);
+                let b = from_slice(&B);
+                let expected = apply_binary_lanewise(a, b, core::ops::Div::div);
+                assert_biteq!(a / b, expected);
+            }
+
+            #[test]
+            fn div_assign() {
+                let mut a = from_slice(&A);
+                let b = from_slice(&B);
+                let expected = apply_binary_lanewise(a, b, core::ops::Div::div);
+                a /= b;
+                assert_biteq!(a, expected);
+            }
+
+            #[test]
+            fn div_scalar_rhs() {
+                let a = from_slice(&A);
+                let b = 5;
+                let expected = apply_binary_scalar_rhs_lanewise(a, b, core::ops::Div::div);
+                assert_biteq!(a / b, expected);
+            }
+
+            #[test]
+            fn div_scalar_lhs() {
+                let a = 5;
+                let b = from_slice(&B);
+                let expected = apply_binary_scalar_lhs_lanewise(a, b, core::ops::Div::div);
+                assert_biteq!(a / b, expected);
+            }
+
+            #[test]
+            fn div_assign_scalar() {
+                let mut a = from_slice(&A);
+                let b = 5;
+                let expected = apply_binary_scalar_rhs_lanewise(a, b, core::ops::Div::div);
+                a /= b;
+                assert_biteq!(a, expected);
+            }
+
+            #[test]
+            fn rem() {
+                let a = from_slice(&A);
+                let b = from_slice(&B);
+                let expected = apply_binary_lanewise(a, b, core::ops::Rem::rem);
+                assert_biteq!(a % b, expected);
+            }
+
+            #[test]
+            fn rem_assign() {
+                let mut a = from_slice(&A);
+                let b = from_slice(&B);
+                let expected = apply_binary_lanewise(a, b, core::ops::Rem::rem);
+                a %= b;
+                assert_biteq!(a, expected);
+            }
+
+            #[test]
+            fn rem_scalar_rhs() {
+                let a = from_slice(&A);
+                let b = 5;
+                let expected = apply_binary_scalar_rhs_lanewise(a, b, core::ops::Rem::rem);
+                assert_biteq!(a % b, expected);
+            }
+
+            #[test]
+            fn rem_scalar_lhs() {
+                let a = 5;
+                let b = from_slice(&B);
+                let expected = apply_binary_scalar_lhs_lanewise(a, b, core::ops::Rem::rem);
+                assert_biteq!(a % b, expected);
+            }
+
+            #[test]
+            fn rem_assign_scalar() {
+                let mut a = from_slice(&A);
+                let b = 5;
+                let expected = apply_binary_scalar_rhs_lanewise(a, b, core::ops::Rem::rem);
+                a %= b;
+                assert_biteq!(a, expected);
+            }
+
+            #[test]
+            fn bitand() {
+                let a = from_slice(&A);
+                let b = from_slice(&B);
+                let expected = apply_binary_lanewise(a, b, core::ops::BitAnd::bitand);
+                assert_biteq!(a & b, expected);
+            }
+
+            #[test]
+            fn bitand_assign() {
+                let mut a = from_slice(&A);
+                let b = from_slice(&B);
+                let expected = apply_binary_lanewise(a, b, core::ops::BitAnd::bitand);
+                a &= b;
+                assert_biteq!(a, expected);
+            }
+
+            #[test]
+            fn bitand_scalar_rhs() {
+                let a = from_slice(&A);
+                let b = 5;
+                let expected = apply_binary_scalar_rhs_lanewise(a, b, core::ops::BitAnd::bitand);
+                assert_biteq!(a & b, expected);
+            }
+
+            #[test]
+            fn bitand_scalar_lhs() {
+                let a = 5;
+                let b = from_slice(&B);
+                let expected = apply_binary_scalar_lhs_lanewise(a, b, core::ops::BitAnd::bitand);
+                assert_biteq!(a & b, expected);
+            }
+
+            #[test]
+            fn bitand_assign_scalar() {
+                let mut a = from_slice(&A);
+                let b = 5;
+                let expected = apply_binary_scalar_rhs_lanewise(a, b, core::ops::BitAnd::bitand);
+                a &= b;
+                assert_biteq!(a, expected);
+            }
+
+            #[test]
+            fn bitor() {
+                let a = from_slice(&A);
+                let b = from_slice(&B);
+                let expected = apply_binary_lanewise(a, b, core::ops::BitOr::bitor);
+                assert_biteq!(a | b, expected);
+            }
+
+            #[test]
+            fn bitor_assign() {
+                let mut a = from_slice(&A);
+                let b = from_slice(&B);
+                let expected = apply_binary_lanewise(a, b, core::ops::BitOr::bitor);
+                a |= b;
+                assert_biteq!(a, expected);
+            }
+
+            #[test]
+            fn bitor_scalar_rhs() {
+                let a = from_slice(&A);
+                let b = 5;
+                let expected = apply_binary_scalar_rhs_lanewise(a, b, core::ops::BitOr::bitor);
+                assert_biteq!(a | b, expected);
+            }
+
+            #[test]
+            fn bitor_scalar_lhs() {
+                let a = 5;
+                let b = from_slice(&B);
+                let expected = apply_binary_scalar_lhs_lanewise(a, b, core::ops::BitOr::bitor);
+                assert_biteq!(a | b, expected);
+            }
+
+            #[test]
+            fn bitor_assign_scalar() {
+                let mut a = from_slice(&A);
+                let b = 5;
+                let expected = apply_binary_scalar_rhs_lanewise(a, b, core::ops::BitOr::bitor);
+                a |= b;
+                assert_biteq!(a, expected);
+            }
+
+            #[test]
+            fn bitxor() {
+                let a = from_slice(&A);
+                let b = from_slice(&B);
+                let expected = apply_binary_lanewise(a, b, core::ops::BitXor::bitxor);
+                assert_biteq!(a ^ b, expected);
+            }
+
+            #[test]
+            fn bitxor_assign() {
+                let mut a = from_slice(&A);
+                let b = from_slice(&B);
+                let expected = apply_binary_lanewise(a, b, core::ops::BitXor::bitxor);
+                a ^= b;
+                assert_biteq!(a, expected);
+            }
+
+            #[test]
+            fn bitxor_scalar_rhs() {
+                let a = from_slice(&A);
+                let b = 5;
+                let expected = apply_binary_scalar_rhs_lanewise(a, b, core::ops::BitXor::bitxor);
+                assert_biteq!(a ^ b, expected);
+            }
+
+            #[test]
+            fn bitxor_scalar_lhs() {
+                let a = 5;
+                let b = from_slice(&B);
+                let expected = apply_binary_scalar_lhs_lanewise(a, b, core::ops::BitXor::bitxor);
+                assert_biteq!(a ^ b, expected);
+            }
+
+            #[test]
+            fn bitxor_assign_scalar() {
+                let mut a = from_slice(&A);
+                let b = 5;
+                let expected = apply_binary_scalar_rhs_lanewise(a, b, core::ops::BitXor::bitxor);
+                a ^= b;
+                assert_biteq!(a, expected);
+            }
+
+            #[test]
+            fn not() {
+                let v = from_slice(&A);
+                let expected = apply_unary_lanewise(v, core::ops::Not::not);
+                assert_biteq!(!v, expected);
+            }
+        }
+    }
+}
diff --git a/crates/core_simd/tests/ops_impl/usize.rs b/crates/core_simd/tests/ops_impl/usize.rs
new file mode 100644
index 000000000000..13da57f15869
--- /dev/null
+++ b/crates/core_simd/tests/ops_impl/usize.rs
@@ -0,0 +1,5 @@
+use super::helpers;
+
+uint_tests! { usizex2, usize }
+uint_tests! { usizex4, usize }
+uint_tests! { usizex8, usize }

From 9028cdde147b3649888b1c11bd4e0d267f85cbef Mon Sep 17 00:00:00 2001
From: Jubilee Young <workingjubilee@gmail.com>
Date: Thu, 1 Oct 2020 18:09:59 -0700
Subject: [PATCH 030/249] Offer a checklist for math PRs

---
 .github/PULL_REQUEST_TEMPLATE.md | 18 ++++++++++++++++++
 1 file changed, 18 insertions(+)
 create mode 100644 .github/PULL_REQUEST_TEMPLATE.md

diff --git a/.github/PULL_REQUEST_TEMPLATE.md b/.github/PULL_REQUEST_TEMPLATE.md
new file mode 100644
index 000000000000..31422b793450
--- /dev/null
+++ b/.github/PULL_REQUEST_TEMPLATE.md
@@ -0,0 +1,18 @@
+Hello, welcome to `std::simd`!
+
+It seems this pull request template checklist was created while a lot of vector math ops were being implemented, and only really applies to ops. Feel free to delete everything here if it's not applicable, or ask for help if you're not sure what it means!
+
+For a given vector math operation on TxN, please add tests for interactions with:
+  - [ ] `T::MAX`
+  - [ ] `T::MIN`
+  - [ ] -1
+  - [ ] 1
+  - [ ] 0
+
+
+For a given vector math operation on TxN where T is a float, please add tests for test interactions with:
+  - [ ] a really large number, larger than the mantissa
+  - [ ] a really small "subnormal" number
+  - [ ] NaN
+  - [ ] Infinity
+  - [ ] Negative Infinity

From e7f133ead05e0556c8a877fde8179798df482014 Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Thu, 1 Oct 2020 23:42:10 -0400
Subject: [PATCH 031/249] Add licenses and update Cargo.toml

---
 LICENSE-APACHE                  | 202 ++++++++++++++++++++++++++++++++
 LICENSE-MIT                     |  19 +++
 crates/core_simd/Cargo.toml     |   5 +
 crates/core_simd/LICENSE-APACHE | 202 ++++++++++++++++++++++++++++++++
 crates/core_simd/LICENSE-MIT    |  19 +++
 5 files changed, 447 insertions(+)
 create mode 100644 LICENSE-APACHE
 create mode 100644 LICENSE-MIT
 create mode 100644 crates/core_simd/LICENSE-APACHE
 create mode 100644 crates/core_simd/LICENSE-MIT

diff --git a/LICENSE-APACHE b/LICENSE-APACHE
new file mode 100644
index 000000000000..d64569567334
--- /dev/null
+++ b/LICENSE-APACHE
@@ -0,0 +1,202 @@
+
+                                 Apache License
+                           Version 2.0, January 2004
+                        http://www.apache.org/licenses/
+
+   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+   1. Definitions.
+
+      "License" shall mean the terms and conditions for use, reproduction,
+      and distribution as defined by Sections 1 through 9 of this document.
+
+      "Licensor" shall mean the copyright owner or entity authorized by
+      the copyright owner that is granting the License.
+
+      "Legal Entity" shall mean the union of the acting entity and all
+      other entities that control, are controlled by, or are under common
+      control with that entity. For the purposes of this definition,
+      "control" means (i) the power, direct or indirect, to cause the
+      direction or management of such entity, whether by contract or
+      otherwise, or (ii) ownership of fifty percent (50%) or more of the
+      outstanding shares, or (iii) beneficial ownership of such entity.
+
+      "You" (or "Your") shall mean an individual or Legal Entity
+      exercising permissions granted by this License.
+
+      "Source" form shall mean the preferred form for making modifications,
+      including but not limited to software source code, documentation
+      source, and configuration files.
+
+      "Object" form shall mean any form resulting from mechanical
+      transformation or translation of a Source form, including but
+      not limited to compiled object code, generated documentation,
+      and conversions to other media types.
+
+      "Work" shall mean the work of authorship, whether in Source or
+      Object form, made available under the License, as indicated by a
+      copyright notice that is included in or attached to the work
+      (an example is provided in the Appendix below).
+
+      "Derivative Works" shall mean any work, whether in Source or Object
+      form, that is based on (or derived from) the Work and for which the
+      editorial revisions, annotations, elaborations, or other modifications
+      represent, as a whole, an original work of authorship. For the purposes
+      of this License, Derivative Works shall not include works that remain
+      separable from, or merely link (or bind by name) to the interfaces of,
+      the Work and Derivative Works thereof.
+
+      "Contribution" shall mean any work of authorship, including
+      the original version of the Work and any modifications or additions
+      to that Work or Derivative Works thereof, that is intentionally
+      submitted to Licensor for inclusion in the Work by the copyright owner
+      or by an individual or Legal Entity authorized to submit on behalf of
+      the copyright owner. For the purposes of this definition, "submitted"
+      means any form of electronic, verbal, or written communication sent
+      to the Licensor or its representatives, including but not limited to
+      communication on electronic mailing lists, source code control systems,
+      and issue tracking systems that are managed by, or on behalf of, the
+      Licensor for the purpose of discussing and improving the Work, but
+      excluding communication that is conspicuously marked or otherwise
+      designated in writing by the copyright owner as "Not a Contribution."
+
+      "Contributor" shall mean Licensor and any individual or Legal Entity
+      on behalf of whom a Contribution has been received by Licensor and
+      subsequently incorporated within the Work.
+
+   2. Grant of Copyright License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      copyright license to reproduce, prepare Derivative Works of,
+      publicly display, publicly perform, sublicense, and distribute the
+      Work and such Derivative Works in Source or Object form.
+
+   3. Grant of Patent License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      (except as stated in this section) patent license to make, have made,
+      use, offer to sell, sell, import, and otherwise transfer the Work,
+      where such license applies only to those patent claims licensable
+      by such Contributor that are necessarily infringed by their
+      Contribution(s) alone or by combination of their Contribution(s)
+      with the Work to which such Contribution(s) was submitted. If You
+      institute patent litigation against any entity (including a
+      cross-claim or counterclaim in a lawsuit) alleging that the Work
+      or a Contribution incorporated within the Work constitutes direct
+      or contributory patent infringement, then any patent licenses
+      granted to You under this License for that Work shall terminate
+      as of the date such litigation is filed.
+
+   4. Redistribution. You may reproduce and distribute copies of the
+      Work or Derivative Works thereof in any medium, with or without
+      modifications, and in Source or Object form, provided that You
+      meet the following conditions:
+
+      (a) You must give any other recipients of the Work or
+          Derivative Works a copy of this License; and
+
+      (b) You must cause any modified files to carry prominent notices
+          stating that You changed the files; and
+
+      (c) You must retain, in the Source form of any Derivative Works
+          that You distribute, all copyright, patent, trademark, and
+          attribution notices from the Source form of the Work,
+          excluding those notices that do not pertain to any part of
+          the Derivative Works; and
+
+      (d) If the Work includes a "NOTICE" text file as part of its
+          distribution, then any Derivative Works that You distribute must
+          include a readable copy of the attribution notices contained
+          within such NOTICE file, excluding those notices that do not
+          pertain to any part of the Derivative Works, in at least one
+          of the following places: within a NOTICE text file distributed
+          as part of the Derivative Works; within the Source form or
+          documentation, if provided along with the Derivative Works; or,
+          within a display generated by the Derivative Works, if and
+          wherever such third-party notices normally appear. The contents
+          of the NOTICE file are for informational purposes only and
+          do not modify the License. You may add Your own attribution
+          notices within Derivative Works that You distribute, alongside
+          or as an addendum to the NOTICE text from the Work, provided
+          that such additional attribution notices cannot be construed
+          as modifying the License.
+
+      You may add Your own copyright statement to Your modifications and
+      may provide additional or different license terms and conditions
+      for use, reproduction, or distribution of Your modifications, or
+      for any such Derivative Works as a whole, provided Your use,
+      reproduction, and distribution of the Work otherwise complies with
+      the conditions stated in this License.
+
+   5. Submission of Contributions. Unless You explicitly state otherwise,
+      any Contribution intentionally submitted for inclusion in the Work
+      by You to the Licensor shall be under the terms and conditions of
+      this License, without any additional terms or conditions.
+      Notwithstanding the above, nothing herein shall supersede or modify
+      the terms of any separate license agreement you may have executed
+      with Licensor regarding such Contributions.
+
+   6. Trademarks. This License does not grant permission to use the trade
+      names, trademarks, service marks, or product names of the Licensor,
+      except as required for reasonable and customary use in describing the
+      origin of the Work and reproducing the content of the NOTICE file.
+
+   7. Disclaimer of Warranty. Unless required by applicable law or
+      agreed to in writing, Licensor provides the Work (and each
+      Contributor provides its Contributions) on an "AS IS" BASIS,
+      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+      implied, including, without limitation, any warranties or conditions
+      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+      PARTICULAR PURPOSE. You are solely responsible for determining the
+      appropriateness of using or redistributing the Work and assume any
+      risks associated with Your exercise of permissions under this License.
+
+   8. Limitation of Liability. In no event and under no legal theory,
+      whether in tort (including negligence), contract, or otherwise,
+      unless required by applicable law (such as deliberate and grossly
+      negligent acts) or agreed to in writing, shall any Contributor be
+      liable to You for damages, including any direct, indirect, special,
+      incidental, or consequential damages of any character arising as a
+      result of this License or out of the use or inability to use the
+      Work (including but not limited to damages for loss of goodwill,
+      work stoppage, computer failure or malfunction, or any and all
+      other commercial damages or losses), even if such Contributor
+      has been advised of the possibility of such damages.
+
+   9. Accepting Warranty or Additional Liability. While redistributing
+      the Work or Derivative Works thereof, You may choose to offer,
+      and charge a fee for, acceptance of support, warranty, indemnity,
+      or other liability obligations and/or rights consistent with this
+      License. However, in accepting such obligations, You may act only
+      on Your own behalf and on Your sole responsibility, not on behalf
+      of any other Contributor, and only if You agree to indemnify,
+      defend, and hold each Contributor harmless for any liability
+      incurred by, or claims asserted against, such Contributor by reason
+      of your accepting any such warranty or additional liability.
+
+   END OF TERMS AND CONDITIONS
+
+   APPENDIX: How to apply the Apache License to your work.
+
+      To apply the Apache License to your work, attach the following
+      boilerplate notice, with the fields enclosed by brackets "[]"
+      replaced with your own identifying information. (Don't include
+      the brackets!)  The text should be enclosed in the appropriate
+      comment syntax for the file format. We also recommend that a
+      file or class name and description of purpose be included on the
+      same "printed page" as the copyright notice for easier
+      identification within third-party archives.
+
+   Copyright [yyyy] [name of copyright owner]
+
+   Licensed under the Apache License, Version 2.0 (the "License");
+   you may not use this file except in compliance with the License.
+   You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.
diff --git a/LICENSE-MIT b/LICENSE-MIT
new file mode 100644
index 000000000000..0e9d2f43a060
--- /dev/null
+++ b/LICENSE-MIT
@@ -0,0 +1,19 @@
+Copyright (c) 2020 The Rust Project Developers
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.
diff --git a/crates/core_simd/Cargo.toml b/crates/core_simd/Cargo.toml
index d4aef6f059c5..0bb4ae9caaf6 100644
--- a/crates/core_simd/Cargo.toml
+++ b/crates/core_simd/Cargo.toml
@@ -3,3 +3,8 @@ name = "core_simd"
 version = "0.1.0"
 authors = ["Caleb Zulawski <caleb.zulawski@gmail.com>"]
 edition = "2018"
+homepage = "https://github.com/rust-lang/stdsimd"
+repository = "https://github.com/rust-lang/stdsimd"
+keywords = ["core", "simd", "intrinsics"]
+categories = ["hardware-support", "no-std"]
+license = "MIT OR Apache-2.0"
diff --git a/crates/core_simd/LICENSE-APACHE b/crates/core_simd/LICENSE-APACHE
new file mode 100644
index 000000000000..d64569567334
--- /dev/null
+++ b/crates/core_simd/LICENSE-APACHE
@@ -0,0 +1,202 @@
+
+                                 Apache License
+                           Version 2.0, January 2004
+                        http://www.apache.org/licenses/
+
+   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+   1. Definitions.
+
+      "License" shall mean the terms and conditions for use, reproduction,
+      and distribution as defined by Sections 1 through 9 of this document.
+
+      "Licensor" shall mean the copyright owner or entity authorized by
+      the copyright owner that is granting the License.
+
+      "Legal Entity" shall mean the union of the acting entity and all
+      other entities that control, are controlled by, or are under common
+      control with that entity. For the purposes of this definition,
+      "control" means (i) the power, direct or indirect, to cause the
+      direction or management of such entity, whether by contract or
+      otherwise, or (ii) ownership of fifty percent (50%) or more of the
+      outstanding shares, or (iii) beneficial ownership of such entity.
+
+      "You" (or "Your") shall mean an individual or Legal Entity
+      exercising permissions granted by this License.
+
+      "Source" form shall mean the preferred form for making modifications,
+      including but not limited to software source code, documentation
+      source, and configuration files.
+
+      "Object" form shall mean any form resulting from mechanical
+      transformation or translation of a Source form, including but
+      not limited to compiled object code, generated documentation,
+      and conversions to other media types.
+
+      "Work" shall mean the work of authorship, whether in Source or
+      Object form, made available under the License, as indicated by a
+      copyright notice that is included in or attached to the work
+      (an example is provided in the Appendix below).
+
+      "Derivative Works" shall mean any work, whether in Source or Object
+      form, that is based on (or derived from) the Work and for which the
+      editorial revisions, annotations, elaborations, or other modifications
+      represent, as a whole, an original work of authorship. For the purposes
+      of this License, Derivative Works shall not include works that remain
+      separable from, or merely link (or bind by name) to the interfaces of,
+      the Work and Derivative Works thereof.
+
+      "Contribution" shall mean any work of authorship, including
+      the original version of the Work and any modifications or additions
+      to that Work or Derivative Works thereof, that is intentionally
+      submitted to Licensor for inclusion in the Work by the copyright owner
+      or by an individual or Legal Entity authorized to submit on behalf of
+      the copyright owner. For the purposes of this definition, "submitted"
+      means any form of electronic, verbal, or written communication sent
+      to the Licensor or its representatives, including but not limited to
+      communication on electronic mailing lists, source code control systems,
+      and issue tracking systems that are managed by, or on behalf of, the
+      Licensor for the purpose of discussing and improving the Work, but
+      excluding communication that is conspicuously marked or otherwise
+      designated in writing by the copyright owner as "Not a Contribution."
+
+      "Contributor" shall mean Licensor and any individual or Legal Entity
+      on behalf of whom a Contribution has been received by Licensor and
+      subsequently incorporated within the Work.
+
+   2. Grant of Copyright License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      copyright license to reproduce, prepare Derivative Works of,
+      publicly display, publicly perform, sublicense, and distribute the
+      Work and such Derivative Works in Source or Object form.
+
+   3. Grant of Patent License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      (except as stated in this section) patent license to make, have made,
+      use, offer to sell, sell, import, and otherwise transfer the Work,
+      where such license applies only to those patent claims licensable
+      by such Contributor that are necessarily infringed by their
+      Contribution(s) alone or by combination of their Contribution(s)
+      with the Work to which such Contribution(s) was submitted. If You
+      institute patent litigation against any entity (including a
+      cross-claim or counterclaim in a lawsuit) alleging that the Work
+      or a Contribution incorporated within the Work constitutes direct
+      or contributory patent infringement, then any patent licenses
+      granted to You under this License for that Work shall terminate
+      as of the date such litigation is filed.
+
+   4. Redistribution. You may reproduce and distribute copies of the
+      Work or Derivative Works thereof in any medium, with or without
+      modifications, and in Source or Object form, provided that You
+      meet the following conditions:
+
+      (a) You must give any other recipients of the Work or
+          Derivative Works a copy of this License; and
+
+      (b) You must cause any modified files to carry prominent notices
+          stating that You changed the files; and
+
+      (c) You must retain, in the Source form of any Derivative Works
+          that You distribute, all copyright, patent, trademark, and
+          attribution notices from the Source form of the Work,
+          excluding those notices that do not pertain to any part of
+          the Derivative Works; and
+
+      (d) If the Work includes a "NOTICE" text file as part of its
+          distribution, then any Derivative Works that You distribute must
+          include a readable copy of the attribution notices contained
+          within such NOTICE file, excluding those notices that do not
+          pertain to any part of the Derivative Works, in at least one
+          of the following places: within a NOTICE text file distributed
+          as part of the Derivative Works; within the Source form or
+          documentation, if provided along with the Derivative Works; or,
+          within a display generated by the Derivative Works, if and
+          wherever such third-party notices normally appear. The contents
+          of the NOTICE file are for informational purposes only and
+          do not modify the License. You may add Your own attribution
+          notices within Derivative Works that You distribute, alongside
+          or as an addendum to the NOTICE text from the Work, provided
+          that such additional attribution notices cannot be construed
+          as modifying the License.
+
+      You may add Your own copyright statement to Your modifications and
+      may provide additional or different license terms and conditions
+      for use, reproduction, or distribution of Your modifications, or
+      for any such Derivative Works as a whole, provided Your use,
+      reproduction, and distribution of the Work otherwise complies with
+      the conditions stated in this License.
+
+   5. Submission of Contributions. Unless You explicitly state otherwise,
+      any Contribution intentionally submitted for inclusion in the Work
+      by You to the Licensor shall be under the terms and conditions of
+      this License, without any additional terms or conditions.
+      Notwithstanding the above, nothing herein shall supersede or modify
+      the terms of any separate license agreement you may have executed
+      with Licensor regarding such Contributions.
+
+   6. Trademarks. This License does not grant permission to use the trade
+      names, trademarks, service marks, or product names of the Licensor,
+      except as required for reasonable and customary use in describing the
+      origin of the Work and reproducing the content of the NOTICE file.
+
+   7. Disclaimer of Warranty. Unless required by applicable law or
+      agreed to in writing, Licensor provides the Work (and each
+      Contributor provides its Contributions) on an "AS IS" BASIS,
+      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+      implied, including, without limitation, any warranties or conditions
+      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+      PARTICULAR PURPOSE. You are solely responsible for determining the
+      appropriateness of using or redistributing the Work and assume any
+      risks associated with Your exercise of permissions under this License.
+
+   8. Limitation of Liability. In no event and under no legal theory,
+      whether in tort (including negligence), contract, or otherwise,
+      unless required by applicable law (such as deliberate and grossly
+      negligent acts) or agreed to in writing, shall any Contributor be
+      liable to You for damages, including any direct, indirect, special,
+      incidental, or consequential damages of any character arising as a
+      result of this License or out of the use or inability to use the
+      Work (including but not limited to damages for loss of goodwill,
+      work stoppage, computer failure or malfunction, or any and all
+      other commercial damages or losses), even if such Contributor
+      has been advised of the possibility of such damages.
+
+   9. Accepting Warranty or Additional Liability. While redistributing
+      the Work or Derivative Works thereof, You may choose to offer,
+      and charge a fee for, acceptance of support, warranty, indemnity,
+      or other liability obligations and/or rights consistent with this
+      License. However, in accepting such obligations, You may act only
+      on Your own behalf and on Your sole responsibility, not on behalf
+      of any other Contributor, and only if You agree to indemnify,
+      defend, and hold each Contributor harmless for any liability
+      incurred by, or claims asserted against, such Contributor by reason
+      of your accepting any such warranty or additional liability.
+
+   END OF TERMS AND CONDITIONS
+
+   APPENDIX: How to apply the Apache License to your work.
+
+      To apply the Apache License to your work, attach the following
+      boilerplate notice, with the fields enclosed by brackets "[]"
+      replaced with your own identifying information. (Don't include
+      the brackets!)  The text should be enclosed in the appropriate
+      comment syntax for the file format. We also recommend that a
+      file or class name and description of purpose be included on the
+      same "printed page" as the copyright notice for easier
+      identification within third-party archives.
+
+   Copyright [yyyy] [name of copyright owner]
+
+   Licensed under the Apache License, Version 2.0 (the "License");
+   you may not use this file except in compliance with the License.
+   You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.
diff --git a/crates/core_simd/LICENSE-MIT b/crates/core_simd/LICENSE-MIT
new file mode 100644
index 000000000000..0e9d2f43a060
--- /dev/null
+++ b/crates/core_simd/LICENSE-MIT
@@ -0,0 +1,19 @@
+Copyright (c) 2020 The Rust Project Developers
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

From 0fbf64bae8be57b02c058f8758d1b53159c42dd8 Mon Sep 17 00:00:00 2001
From: Ashley Mannix <kodraus@hey.com>
Date: Sat, 3 Oct 2020 16:36:51 +1000
Subject: [PATCH 032/249] Add CI for wasm targets using wasm-bindgen (#20)

---
 .travis.yml                                   | 24 ++++++++++
 crates/core_simd/Cargo.toml                   |  6 +++
 .../core_simd/tests/ops_impl/float_macros.rs  | 32 +++++++++++++
 crates/core_simd/tests/ops_impl/int_macros.rs | 48 +++++++++++++++++++
 .../core_simd/tests/ops_impl/mask_macros.rs   | 22 +++++++++
 .../core_simd/tests/ops_impl/uint_macros.rs   | 47 ++++++++++++++++++
 crates/core_simd/webdriver.json               |  7 +++
 7 files changed, 186 insertions(+)
 create mode 100644 crates/core_simd/webdriver.json

diff --git a/.travis.yml b/.travis.yml
index 0bd9760ab0ae..b9df36386504 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -230,6 +230,30 @@ matrix:
       arch: amd64
       env:
         - TARGET=x86_64-apple-darwin
+    
+    # WebAssembly (wasm-bindgen)
+
+    - name: "wasm32-unknown-unknown (node, firefox, chrome)"
+      os: linux
+      arch: amd64
+      addons:
+        firefox: latest
+        chrome : stable
+      install:
+        - curl https://rustwasm.github.io/wasm-pack/installer/init.sh -sSf | sh
+      script:
+        - wasm-pack test --node --firefox --chrome --headless crates/core_simd
+    
+    - name: "wasm32-unknown-unknown+simd128 (chrome)"
+      os: linux
+      arch: amd64
+      addons:
+        chrome : stable
+      install:
+        - curl https://rustwasm.github.io/wasm-pack/installer/init.sh -sSf | sh
+      script:
+        - RUSTFLAGS="-C target-feature=+simd128"
+        - wasm-pack test --chrome --headless crates/core_simd
 
 script:
   - rustup target add $TARGET
diff --git a/crates/core_simd/Cargo.toml b/crates/core_simd/Cargo.toml
index 0bb4ae9caaf6..f9e8a62e4857 100644
--- a/crates/core_simd/Cargo.toml
+++ b/crates/core_simd/Cargo.toml
@@ -8,3 +8,9 @@ repository = "https://github.com/rust-lang/stdsimd"
 keywords = ["core", "simd", "intrinsics"]
 categories = ["hardware-support", "no-std"]
 license = "MIT OR Apache-2.0"
+
+[target.'cfg(target_arch = "wasm32")'.dependencies.wasm-bindgen]
+version = "0.2"
+
+[dev-dependencies.wasm-bindgen-test]
+version = "0.3"
diff --git a/crates/core_simd/tests/ops_impl/float_macros.rs b/crates/core_simd/tests/ops_impl/float_macros.rs
index ddf3bbbe9369..9bcb894b2572 100644
--- a/crates/core_simd/tests/ops_impl/float_macros.rs
+++ b/crates/core_simd/tests/ops_impl/float_macros.rs
@@ -5,6 +5,12 @@ macro_rules! float_tests {
             use super::*;
             use helpers::lanewise::*;
 
+            #[cfg(target_arch = "wasm32")]
+            use wasm_bindgen_test::*;
+
+            #[cfg(target_arch = "wasm32")]
+            wasm_bindgen_test_configure!(run_in_browser);
+
             // TODO impl this as an associated fn on vectors
             fn from_slice(slice: &[$scalar]) -> core_simd::$vector {
                 let mut value = core_simd::$vector::default();
@@ -17,6 +23,7 @@ macro_rules! float_tests {
             const B: [$scalar; 16] = [16., 17., 18., 19., 20., 21., 22., 23., 24., 25., 26., 27., 28., 29., 30., 31.];
 
             #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn add() {
                 let a = from_slice(&A);
                 let b = from_slice(&B);
@@ -25,6 +32,7 @@ macro_rules! float_tests {
             }
 
             #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn add_assign() {
                 let mut a = from_slice(&A);
                 let b = from_slice(&B);
@@ -34,6 +42,7 @@ macro_rules! float_tests {
             }
 
             #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn add_scalar_rhs() {
                 let a = from_slice(&A);
                 let b = 5.;
@@ -42,6 +51,7 @@ macro_rules! float_tests {
             }
 
             #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn add_scalar_lhs() {
                 let a = 5.;
                 let b = from_slice(&B);
@@ -50,6 +60,7 @@ macro_rules! float_tests {
             }
 
             #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn add_assign_scalar() {
                 let mut a = from_slice(&A);
                 let b = 5.;
@@ -59,6 +70,7 @@ macro_rules! float_tests {
             }
 
             #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn sub() {
                 let a = from_slice(&A);
                 let b = from_slice(&B);
@@ -67,6 +79,7 @@ macro_rules! float_tests {
             }
 
             #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn sub_assign() {
                 let mut a = from_slice(&A);
                 let b = from_slice(&B);
@@ -76,6 +89,7 @@ macro_rules! float_tests {
             }
 
             #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn sub_scalar_rhs() {
                 let a = from_slice(&A);
                 let b = 5.;
@@ -84,6 +98,7 @@ macro_rules! float_tests {
             }
 
             #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn sub_scalar_lhs() {
                 let a = 5.;
                 let b = from_slice(&B);
@@ -92,6 +107,7 @@ macro_rules! float_tests {
             }
 
             #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn sub_assign_scalar() {
                 let mut a = from_slice(&A);
                 let b = 5.;
@@ -101,6 +117,7 @@ macro_rules! float_tests {
             }
 
             #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn mul() {
                 let a = from_slice(&A);
                 let b = from_slice(&B);
@@ -109,6 +126,7 @@ macro_rules! float_tests {
             }
 
             #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn mul_assign() {
                 let mut a = from_slice(&A);
                 let b = from_slice(&B);
@@ -118,6 +136,7 @@ macro_rules! float_tests {
             }
 
             #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn mul_scalar_rhs() {
                 let a = from_slice(&A);
                 let b = 5.;
@@ -126,6 +145,7 @@ macro_rules! float_tests {
             }
 
             #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn mul_scalar_lhs() {
                 let a = 5.;
                 let b = from_slice(&B);
@@ -134,6 +154,7 @@ macro_rules! float_tests {
             }
 
             #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn mul_assign_scalar() {
                 let mut a = from_slice(&A);
                 let b = 5.;
@@ -143,6 +164,7 @@ macro_rules! float_tests {
             }
 
             #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn div() {
                 let a = from_slice(&A);
                 let b = from_slice(&B);
@@ -151,6 +173,7 @@ macro_rules! float_tests {
             }
 
             #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn div_assign() {
                 let mut a = from_slice(&A);
                 let b = from_slice(&B);
@@ -160,6 +183,7 @@ macro_rules! float_tests {
             }
 
             #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn div_scalar_rhs() {
                 let a = from_slice(&A);
                 let b = 5.;
@@ -168,6 +192,7 @@ macro_rules! float_tests {
             }
 
             #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn div_scalar_lhs() {
                 let a = 5.;
                 let b = from_slice(&B);
@@ -176,6 +201,7 @@ macro_rules! float_tests {
             }
 
             #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn div_assign_scalar() {
                 let mut a = from_slice(&A);
                 let b = 5.;
@@ -185,6 +211,7 @@ macro_rules! float_tests {
             }
 
             #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn rem() {
                 let a = from_slice(&A);
                 let b = from_slice(&B);
@@ -193,6 +220,7 @@ macro_rules! float_tests {
             }
 
             #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn rem_assign() {
                 let mut a = from_slice(&A);
                 let b = from_slice(&B);
@@ -202,6 +230,7 @@ macro_rules! float_tests {
             }
 
             #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn rem_scalar_rhs() {
                 let a = from_slice(&A);
                 let b = 5.;
@@ -210,6 +239,7 @@ macro_rules! float_tests {
             }
 
             #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn rem_scalar_lhs() {
                 let a = 5.;
                 let b = from_slice(&B);
@@ -218,6 +248,7 @@ macro_rules! float_tests {
             }
 
             #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn rem_assign_scalar() {
                 let mut a = from_slice(&A);
                 let b = 5.;
@@ -227,6 +258,7 @@ macro_rules! float_tests {
             }
 
             #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn neg() {
                 let v = from_slice(&A);
                 let expected = apply_unary_lanewise(v, core::ops::Neg::neg);
diff --git a/crates/core_simd/tests/ops_impl/int_macros.rs b/crates/core_simd/tests/ops_impl/int_macros.rs
index 4175541e892d..af956aa3e521 100644
--- a/crates/core_simd/tests/ops_impl/int_macros.rs
+++ b/crates/core_simd/tests/ops_impl/int_macros.rs
@@ -5,6 +5,12 @@ macro_rules! int_tests {
             use super::*;
             use helpers::lanewise::*;
 
+            #[cfg(target_arch = "wasm32")]
+            use wasm_bindgen_test::*;
+
+            #[cfg(target_arch = "wasm32")]
+            wasm_bindgen_test_configure!(run_in_browser);
+
             // TODO impl this as an associated fn on vectors
             fn from_slice(slice: &[$scalar]) -> core_simd::$vector {
                 let mut value = core_simd::$vector::default();
@@ -35,6 +41,7 @@ macro_rules! int_tests {
             ];
 
             #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn add() {
                 let a = from_slice(&A);
                 let b = from_slice(&B);
@@ -43,6 +50,7 @@ macro_rules! int_tests {
             }
 
             #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn add_assign() {
                 let mut a = from_slice(&A);
                 let b = from_slice(&B);
@@ -52,6 +60,7 @@ macro_rules! int_tests {
             }
 
             #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn add_scalar_rhs() {
                 let a = from_slice(&A);
                 let b = 5;
@@ -60,6 +69,7 @@ macro_rules! int_tests {
             }
 
             #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn add_scalar_lhs() {
                 let a = 5;
                 let b = from_slice(&B);
@@ -68,6 +78,7 @@ macro_rules! int_tests {
             }
 
             #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn add_assign_scalar() {
                 let mut a = from_slice(&A);
                 let b = 5;
@@ -77,6 +88,7 @@ macro_rules! int_tests {
             }
 
             #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn sub() {
                 let a = from_slice(&A);
                 let b = from_slice(&B);
@@ -85,6 +97,7 @@ macro_rules! int_tests {
             }
 
             #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn sub_assign() {
                 let mut a = from_slice(&A);
                 let b = from_slice(&B);
@@ -94,6 +107,7 @@ macro_rules! int_tests {
             }
 
             #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn sub_scalar_rhs() {
                 let a = from_slice(&A);
                 let b = 5;
@@ -102,6 +116,7 @@ macro_rules! int_tests {
             }
 
             #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn sub_scalar_lhs() {
                 let a = 5;
                 let b = from_slice(&B);
@@ -110,6 +125,7 @@ macro_rules! int_tests {
             }
 
             #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn sub_assign_scalar() {
                 let mut a = from_slice(&A);
                 let b = 5;
@@ -119,6 +135,7 @@ macro_rules! int_tests {
             }
 
             #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn mul() {
                 let a = from_slice(&A);
                 let b = from_slice(&B);
@@ -127,6 +144,7 @@ macro_rules! int_tests {
             }
 
             #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn mul_assign() {
                 let mut a = from_slice(&A);
                 let b = from_slice(&B);
@@ -136,6 +154,7 @@ macro_rules! int_tests {
             }
 
             #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn mul_scalar_rhs() {
                 let a = from_slice(&A);
                 let b = 5;
@@ -144,6 +163,7 @@ macro_rules! int_tests {
             }
 
             #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn mul_scalar_lhs() {
                 let a = 5;
                 let b = from_slice(&B);
@@ -152,6 +172,7 @@ macro_rules! int_tests {
             }
 
             #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn mul_assign_scalar() {
                 let mut a = from_slice(&A);
                 let b = 5;
@@ -161,6 +182,7 @@ macro_rules! int_tests {
             }
 
             #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn div() {
                 let a = from_slice(&A);
                 let b = from_slice(&B);
@@ -169,6 +191,7 @@ macro_rules! int_tests {
             }
 
             #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn div_assign() {
                 let mut a = from_slice(&A);
                 let b = from_slice(&B);
@@ -178,6 +201,7 @@ macro_rules! int_tests {
             }
 
             #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn div_scalar_rhs() {
                 let a = from_slice(&A);
                 let b = 5;
@@ -186,6 +210,7 @@ macro_rules! int_tests {
             }
 
             #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn div_scalar_lhs() {
                 let a = 5;
                 let b = from_slice(&B);
@@ -194,6 +219,7 @@ macro_rules! int_tests {
             }
 
             #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn div_assign_scalar() {
                 let mut a = from_slice(&A);
                 let b = 5;
@@ -203,6 +229,7 @@ macro_rules! int_tests {
             }
 
             #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn rem() {
                 let a = from_slice(&A);
                 let b = from_slice(&B);
@@ -211,6 +238,7 @@ macro_rules! int_tests {
             }
 
             #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn rem_assign() {
                 let mut a = from_slice(&A);
                 let b = from_slice(&B);
@@ -220,6 +248,7 @@ macro_rules! int_tests {
             }
 
             #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn rem_scalar_rhs() {
                 let a = from_slice(&A);
                 let b = 5;
@@ -228,6 +257,7 @@ macro_rules! int_tests {
             }
 
             #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn rem_scalar_lhs() {
                 let a = 5;
                 let b = from_slice(&B);
@@ -236,6 +266,7 @@ macro_rules! int_tests {
             }
 
             #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn rem_assign_scalar() {
                 let mut a = from_slice(&A);
                 let b = 5;
@@ -245,6 +276,7 @@ macro_rules! int_tests {
             }
 
             #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn bitand() {
                 let a = from_slice(&A);
                 let b = from_slice(&B);
@@ -253,6 +285,7 @@ macro_rules! int_tests {
             }
 
             #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn bitand_assign() {
                 let mut a = from_slice(&A);
                 let b = from_slice(&B);
@@ -262,6 +295,7 @@ macro_rules! int_tests {
             }
 
             #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn bitand_scalar_rhs() {
                 let a = from_slice(&A);
                 let b = 5;
@@ -270,6 +304,7 @@ macro_rules! int_tests {
             }
 
             #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn bitand_scalar_lhs() {
                 let a = 5;
                 let b = from_slice(&B);
@@ -278,6 +313,7 @@ macro_rules! int_tests {
             }
 
             #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn bitand_assign_scalar() {
                 let mut a = from_slice(&A);
                 let b = 5;
@@ -287,6 +323,7 @@ macro_rules! int_tests {
             }
 
             #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn bitor() {
                 let a = from_slice(&A);
                 let b = from_slice(&B);
@@ -295,6 +332,7 @@ macro_rules! int_tests {
             }
 
             #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn bitor_assign() {
                 let mut a = from_slice(&A);
                 let b = from_slice(&B);
@@ -304,6 +342,7 @@ macro_rules! int_tests {
             }
 
             #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn bitor_scalar_rhs() {
                 let a = from_slice(&A);
                 let b = 5;
@@ -312,6 +351,7 @@ macro_rules! int_tests {
             }
 
             #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn bitor_scalar_lhs() {
                 let a = 5;
                 let b = from_slice(&B);
@@ -320,6 +360,7 @@ macro_rules! int_tests {
             }
 
             #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn bitor_assign_scalar() {
                 let mut a = from_slice(&A);
                 let b = 5;
@@ -329,6 +370,7 @@ macro_rules! int_tests {
             }
 
             #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn bitxor() {
                 let a = from_slice(&A);
                 let b = from_slice(&B);
@@ -337,6 +379,7 @@ macro_rules! int_tests {
             }
 
             #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn bitxor_assign() {
                 let mut a = from_slice(&A);
                 let b = from_slice(&B);
@@ -346,6 +389,7 @@ macro_rules! int_tests {
             }
 
             #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn bitxor_scalar_rhs() {
                 let a = from_slice(&A);
                 let b = 5;
@@ -354,6 +398,7 @@ macro_rules! int_tests {
             }
 
             #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn bitxor_scalar_lhs() {
                 let a = 5;
                 let b = from_slice(&B);
@@ -362,6 +407,7 @@ macro_rules! int_tests {
             }
 
             #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn bitxor_assign_scalar() {
                 let mut a = from_slice(&A);
                 let b = 5;
@@ -371,6 +417,7 @@ macro_rules! int_tests {
             }
 
             #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn neg() {
                 let v = from_slice(&A);
                 let expected = apply_unary_lanewise(v, core::ops::Neg::neg);
@@ -378,6 +425,7 @@ macro_rules! int_tests {
             }
 
             #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn not() {
                 let v = from_slice(&A);
                 let expected = apply_unary_lanewise(v, core::ops::Not::not);
diff --git a/crates/core_simd/tests/ops_impl/mask_macros.rs b/crates/core_simd/tests/ops_impl/mask_macros.rs
index 9d6bc0cd692f..e6aee4c1d302 100644
--- a/crates/core_simd/tests/ops_impl/mask_macros.rs
+++ b/crates/core_simd/tests/ops_impl/mask_macros.rs
@@ -5,6 +5,12 @@ macro_rules! mask_tests {
             use super::*;
             use helpers::lanewise::*;
 
+            #[cfg(target_arch = "wasm32")]
+            use wasm_bindgen_test::*;
+
+            #[cfg(target_arch = "wasm32")]
+            wasm_bindgen_test_configure!(run_in_browser);
+
             fn from_slice(slice: &[bool]) -> core_simd::$vector {
                 let mut value = core_simd::$vector::default();
                 let value_slice: &mut [_] = value.as_mut();
@@ -41,6 +47,7 @@ macro_rules! mask_tests {
             const UNSET_VECTOR: core_simd::$vector = core_simd::$vector::splat(UNSET_SCALAR);
 
             #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn bitand() {
                 let a = from_slice(&A);
                 let b = from_slice(&B);
@@ -49,6 +56,7 @@ macro_rules! mask_tests {
             }
 
             #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn bitand_assign() {
                 let mut a = from_slice(&A);
                 let b = from_slice(&B);
@@ -58,6 +66,7 @@ macro_rules! mask_tests {
             }
 
             #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn bitand_scalar_rhs() {
                 let a = from_slice(&A);
                 let expected = a;
@@ -66,6 +75,7 @@ macro_rules! mask_tests {
             }
 
             #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn bitand_scalar_lhs() {
                 let a = from_slice(&A);
                 let expected = a;
@@ -74,6 +84,7 @@ macro_rules! mask_tests {
             }
 
             #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn bitand_assign_scalar() {
                 let mut a = from_slice(&A);
                 let expected = a;
@@ -84,6 +95,7 @@ macro_rules! mask_tests {
             }
 
             #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn bitor() {
                 let a = from_slice(&A);
                 let b = from_slice(&B);
@@ -92,6 +104,7 @@ macro_rules! mask_tests {
             }
 
             #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn bitor_assign() {
                 let mut a = from_slice(&A);
                 let b = from_slice(&B);
@@ -101,6 +114,7 @@ macro_rules! mask_tests {
             }
 
             #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn bitor_scalar_rhs() {
                 let a = from_slice(&A);
                 assert_biteq!(a | UNSET_SCALAR, a);
@@ -108,6 +122,7 @@ macro_rules! mask_tests {
             }
 
             #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn bitor_scalar_lhs() {
                 let a = from_slice(&A);
                 assert_biteq!(UNSET_SCALAR | a, a);
@@ -115,6 +130,7 @@ macro_rules! mask_tests {
             }
 
             #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn bitor_assign_scalar() {
                 let mut a = from_slice(&A);
                 let expected = a;
@@ -125,6 +141,7 @@ macro_rules! mask_tests {
             }
 
             #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn bitxor() {
                 let a = from_slice(&A);
                 let b = from_slice(&B);
@@ -133,6 +150,7 @@ macro_rules! mask_tests {
             }
 
             #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn bitxor_assign() {
                 let mut a = from_slice(&A);
                 let b = from_slice(&B);
@@ -142,6 +160,7 @@ macro_rules! mask_tests {
             }
 
             #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn bitxor_scalar_rhs() {
                 let a = from_slice(&A);
                 let expected = apply_binary_scalar_rhs_lanewise(a, SET_SCALAR, core::ops::BitXor::bitxor);
@@ -150,6 +169,7 @@ macro_rules! mask_tests {
             }
 
             #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn bitxor_scalar_lhs() {
                 let a = from_slice(&A);
                 let expected = apply_binary_scalar_lhs_lanewise(SET_SCALAR, a, core::ops::BitXor::bitxor);
@@ -158,6 +178,7 @@ macro_rules! mask_tests {
             }
 
             #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn bitxor_assign_scalar() {
                 let mut a = from_slice(&A);
                 let expected_unset = a;
@@ -169,6 +190,7 @@ macro_rules! mask_tests {
             }
 
             #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn not() {
                 let v = from_slice(&A);
                 let expected = apply_unary_lanewise(v, core::ops::Not::not);
diff --git a/crates/core_simd/tests/ops_impl/uint_macros.rs b/crates/core_simd/tests/ops_impl/uint_macros.rs
index eb9ac34d7efd..bc8b3be74860 100644
--- a/crates/core_simd/tests/ops_impl/uint_macros.rs
+++ b/crates/core_simd/tests/ops_impl/uint_macros.rs
@@ -5,6 +5,12 @@ macro_rules! uint_tests {
             use super::*;
             use helpers::lanewise::*;
 
+            #[cfg(target_arch = "wasm32")]
+            use wasm_bindgen_test::*;
+
+            #[cfg(target_arch = "wasm32")]
+            wasm_bindgen_test_configure!(run_in_browser);
+
             // TODO impl this as an associated fn on vectors
             fn from_slice(slice: &[$scalar]) -> core_simd::$vector {
                 let mut value = core_simd::$vector::default();
@@ -35,6 +41,7 @@ macro_rules! uint_tests {
             ];
 
             #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn add() {
                 let a = from_slice(&A);
                 let b = from_slice(&B);
@@ -43,6 +50,7 @@ macro_rules! uint_tests {
             }
 
             #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn add_assign() {
                 let mut a = from_slice(&A);
                 let b = from_slice(&B);
@@ -52,6 +60,7 @@ macro_rules! uint_tests {
             }
 
             #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn add_scalar_rhs() {
                 let a = from_slice(&A);
                 let b = 5;
@@ -60,6 +69,7 @@ macro_rules! uint_tests {
             }
 
             #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn add_scalar_lhs() {
                 let a = 5;
                 let b = from_slice(&B);
@@ -68,6 +78,7 @@ macro_rules! uint_tests {
             }
 
             #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn add_assign_scalar() {
                 let mut a = from_slice(&A);
                 let b = 5;
@@ -77,6 +88,7 @@ macro_rules! uint_tests {
             }
 
             #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn sub() {
                 let a = from_slice(&A);
                 let b = from_slice(&B);
@@ -85,6 +97,7 @@ macro_rules! uint_tests {
             }
 
             #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn sub_assign() {
                 let mut a = from_slice(&A);
                 let b = from_slice(&B);
@@ -94,6 +107,7 @@ macro_rules! uint_tests {
             }
 
             #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn sub_scalar_rhs() {
                 let a = from_slice(&A);
                 let b = 1;
@@ -102,6 +116,7 @@ macro_rules! uint_tests {
             }
 
             #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn sub_scalar_lhs() {
                 let a = 40;
                 let b = from_slice(&B);
@@ -110,6 +125,7 @@ macro_rules! uint_tests {
             }
 
             #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn sub_assign_scalar() {
                 let mut a = from_slice(&A);
                 let b = 1;
@@ -119,6 +135,7 @@ macro_rules! uint_tests {
             }
 
             #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn mul() {
                 let a = from_slice(&A);
                 let b = from_slice(&B);
@@ -127,6 +144,7 @@ macro_rules! uint_tests {
             }
 
             #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn mul_assign() {
                 let mut a = from_slice(&A);
                 let b = from_slice(&B);
@@ -136,6 +154,7 @@ macro_rules! uint_tests {
             }
 
             #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn mul_scalar_rhs() {
                 let a = from_slice(&A);
                 let b = 5;
@@ -144,6 +163,7 @@ macro_rules! uint_tests {
             }
 
             #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn mul_scalar_lhs() {
                 let a = 5;
                 let b = from_slice(&B);
@@ -152,6 +172,7 @@ macro_rules! uint_tests {
             }
 
             #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn mul_assign_scalar() {
                 let mut a = from_slice(&A);
                 let b = 5;
@@ -161,6 +182,7 @@ macro_rules! uint_tests {
             }
 
             #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn div() {
                 let a = from_slice(&A);
                 let b = from_slice(&B);
@@ -169,6 +191,7 @@ macro_rules! uint_tests {
             }
 
             #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn div_assign() {
                 let mut a = from_slice(&A);
                 let b = from_slice(&B);
@@ -178,6 +201,7 @@ macro_rules! uint_tests {
             }
 
             #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn div_scalar_rhs() {
                 let a = from_slice(&A);
                 let b = 5;
@@ -186,6 +210,7 @@ macro_rules! uint_tests {
             }
 
             #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn div_scalar_lhs() {
                 let a = 5;
                 let b = from_slice(&B);
@@ -194,6 +219,7 @@ macro_rules! uint_tests {
             }
 
             #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn div_assign_scalar() {
                 let mut a = from_slice(&A);
                 let b = 5;
@@ -203,6 +229,7 @@ macro_rules! uint_tests {
             }
 
             #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn rem() {
                 let a = from_slice(&A);
                 let b = from_slice(&B);
@@ -211,6 +238,7 @@ macro_rules! uint_tests {
             }
 
             #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn rem_assign() {
                 let mut a = from_slice(&A);
                 let b = from_slice(&B);
@@ -220,6 +248,7 @@ macro_rules! uint_tests {
             }
 
             #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn rem_scalar_rhs() {
                 let a = from_slice(&A);
                 let b = 5;
@@ -228,6 +257,7 @@ macro_rules! uint_tests {
             }
 
             #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn rem_scalar_lhs() {
                 let a = 5;
                 let b = from_slice(&B);
@@ -236,6 +266,7 @@ macro_rules! uint_tests {
             }
 
             #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn rem_assign_scalar() {
                 let mut a = from_slice(&A);
                 let b = 5;
@@ -245,6 +276,7 @@ macro_rules! uint_tests {
             }
 
             #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn bitand() {
                 let a = from_slice(&A);
                 let b = from_slice(&B);
@@ -253,6 +285,7 @@ macro_rules! uint_tests {
             }
 
             #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn bitand_assign() {
                 let mut a = from_slice(&A);
                 let b = from_slice(&B);
@@ -262,6 +295,7 @@ macro_rules! uint_tests {
             }
 
             #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn bitand_scalar_rhs() {
                 let a = from_slice(&A);
                 let b = 5;
@@ -270,6 +304,7 @@ macro_rules! uint_tests {
             }
 
             #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn bitand_scalar_lhs() {
                 let a = 5;
                 let b = from_slice(&B);
@@ -278,6 +313,7 @@ macro_rules! uint_tests {
             }
 
             #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn bitand_assign_scalar() {
                 let mut a = from_slice(&A);
                 let b = 5;
@@ -287,6 +323,7 @@ macro_rules! uint_tests {
             }
 
             #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn bitor() {
                 let a = from_slice(&A);
                 let b = from_slice(&B);
@@ -295,6 +332,7 @@ macro_rules! uint_tests {
             }
 
             #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn bitor_assign() {
                 let mut a = from_slice(&A);
                 let b = from_slice(&B);
@@ -304,6 +342,7 @@ macro_rules! uint_tests {
             }
 
             #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn bitor_scalar_rhs() {
                 let a = from_slice(&A);
                 let b = 5;
@@ -312,6 +351,7 @@ macro_rules! uint_tests {
             }
 
             #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn bitor_scalar_lhs() {
                 let a = 5;
                 let b = from_slice(&B);
@@ -320,6 +360,7 @@ macro_rules! uint_tests {
             }
 
             #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn bitor_assign_scalar() {
                 let mut a = from_slice(&A);
                 let b = 5;
@@ -329,6 +370,7 @@ macro_rules! uint_tests {
             }
 
             #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn bitxor() {
                 let a = from_slice(&A);
                 let b = from_slice(&B);
@@ -337,6 +379,7 @@ macro_rules! uint_tests {
             }
 
             #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn bitxor_assign() {
                 let mut a = from_slice(&A);
                 let b = from_slice(&B);
@@ -346,6 +389,7 @@ macro_rules! uint_tests {
             }
 
             #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn bitxor_scalar_rhs() {
                 let a = from_slice(&A);
                 let b = 5;
@@ -354,6 +398,7 @@ macro_rules! uint_tests {
             }
 
             #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn bitxor_scalar_lhs() {
                 let a = 5;
                 let b = from_slice(&B);
@@ -362,6 +407,7 @@ macro_rules! uint_tests {
             }
 
             #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn bitxor_assign_scalar() {
                 let mut a = from_slice(&A);
                 let b = 5;
@@ -371,6 +417,7 @@ macro_rules! uint_tests {
             }
 
             #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn not() {
                 let v = from_slice(&A);
                 let expected = apply_unary_lanewise(v, core::ops::Not::not);
diff --git a/crates/core_simd/webdriver.json b/crates/core_simd/webdriver.json
new file mode 100644
index 000000000000..f1d5734f1ceb
--- /dev/null
+++ b/crates/core_simd/webdriver.json
@@ -0,0 +1,7 @@
+{
+    "goog:chromeOptions": {
+        "args": [
+            "--enable-features=WebAssemblySimd"
+        ]
+    }
+}

From 5aa7cffde1c0d1a8ed5dc8145f3c4c33b8c72f0a Mon Sep 17 00:00:00 2001
From: Ashley Mannix <kodraus@hey.com>
Date: Sat, 3 Oct 2020 16:40:18 +1000
Subject: [PATCH 033/249] add build badge

---
 README.md | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/README.md b/README.md
index 976ec644e415..58dd50ae1247 100644
--- a/README.md
+++ b/README.md
@@ -1,4 +1,6 @@
-# stdsimd
+# stdsimd - Rust's standard library portable SIMD API
+
+[![Build Status](https://travis-ci.com/rust-lang/stdsimd.svg?branch=master)](https://travis-ci.com/rust-lang/stdsimd)
 
 Code repository for the [Portable SIMD Project Group](https://github.com/rust-lang/project-portable-simd).
 

From 866971adf50539f944d9d1e034fcaab9a1be9609 Mon Sep 17 00:00:00 2001
From: Thom Chiovoloni <chiovolonit@gmail.com>
Date: Tue, 6 Oct 2020 11:21:54 -0700
Subject: [PATCH 034/249] Implement abs, to_bits, and from_bits for float
 vectors

---
 crates/core_simd/src/macros.rs                | 46 +++++++++++++++++++
 crates/core_simd/src/vectors_f32.rs           | 14 ++++--
 crates/core_simd/src/vectors_f64.rs           |  9 ++--
 .../core_simd/tests/ops_impl/float_macros.rs  | 44 ++++++++++++++++++
 4 files changed, 106 insertions(+), 7 deletions(-)

diff --git a/crates/core_simd/src/macros.rs b/crates/core_simd/src/macros.rs
index 2f93db190352..214867c5ddce 100644
--- a/crates/core_simd/src/macros.rs
+++ b/crates/core_simd/src/macros.rs
@@ -270,6 +270,52 @@ macro_rules! define_vector {
     }
 }
 
+/// Implements inherent methods for a float vector `$name` containing multiple
+/// `$lanes` of float `$type`, which uses `$bits_ty` as its binary
+/// representation. Called from `define_float_vector!`.
+macro_rules! impl_float_vector {
+    { $name:path => [$type:ty; $lanes:literal]; bits $bits_ty:ty; } => {
+        impl $name {
+            /// Raw transmutation to an unsigned integer vector type with the
+            /// same size and number of lanes.
+            #[inline]
+            pub fn to_bits(self) -> $bits_ty {
+                unsafe { core::mem::transmute(self) }
+            }
+
+            /// Raw transmutation from an unsigned integer vector type with the
+            /// same size and number of lanes.
+            #[inline]
+            pub fn from_bits(bits: $bits_ty) -> Self {
+                unsafe { core::mem::transmute(bits) }
+            }
+
+            /// Produces a vector where every lane has the absolute value of the
+            /// equivalently-indexed lane in `self`.
+            #[inline]
+            pub fn abs(self) -> Self {
+                let no_sign = <$bits_ty>::splat(!0 >> 1);
+                let abs = unsafe { crate::intrinsics::simd_and(self.to_bits(), no_sign) };
+                Self::from_bits(abs)
+            }
+        }
+    };
+}
+
+/// Defines a float vector `$name` containing multiple `$lanes` of float
+/// `$type`, which uses `$bits_ty` as its binary representation.
+macro_rules! define_float_vector {
+    { $(#[$attr:meta])* struct $name:ident([$type:ty; $lanes:tt]); bits $bits_ty:ty; } => {
+        define_vector! {
+            $(#[$attr])*
+            struct $name([$type; $lanes]);
+        }
+
+        impl_float_vector! { $name => [$type; $lanes]; bits $bits_ty; }
+    }
+}
+
+
 /// Defines an integer vector `$name` containing multiple `$lanes` of integer `$type`.
 macro_rules! define_integer_vector {
     { $(#[$attr:meta])* struct $name:ident([$type:ty; $lanes:tt]); } => {
diff --git a/crates/core_simd/src/vectors_f32.rs b/crates/core_simd/src/vectors_f32.rs
index 9fcbd9d53f0f..17b382ee739e 100644
--- a/crates/core_simd/src/vectors_f32.rs
+++ b/crates/core_simd/src/vectors_f32.rs
@@ -1,23 +1,29 @@
-define_vector! {
+define_float_vector! {
     /// Vector of two `f32` values
     struct f32x2([f32; 2]);
+    bits crate::u32x2;
 }
 
-define_vector! {
+define_float_vector! {
     /// Vector of four `f32` values
     struct f32x4([f32; 4]);
+    bits crate::u32x4;
 }
 
-define_vector! {
+define_float_vector! {
     /// Vector of eight `f32` values
     struct f32x8([f32; 8]);
+    bits crate::u32x8;
 }
 
-define_vector! {
+define_float_vector! {
     /// Vector of 16 `f32` values
     struct f32x16([f32; 16]);
+    bits crate::u32x16;
 }
 
 from_transmute_x86! { unsafe f32x4 => __m128 }
 from_transmute_x86! { unsafe f32x8 => __m256 }
 //from_transmute_x86! { unsafe f32x16 => __m512 }
+
+
diff --git a/crates/core_simd/src/vectors_f64.rs b/crates/core_simd/src/vectors_f64.rs
index d741aabe88e0..b41923ca6f10 100644
--- a/crates/core_simd/src/vectors_f64.rs
+++ b/crates/core_simd/src/vectors_f64.rs
@@ -1,16 +1,19 @@
-define_vector! {
+define_float_vector! {
     /// Vector of two `f64` values
     struct f64x2([f64; 2]);
+    bits crate::u64x2;
 }
 
-define_vector! {
+define_float_vector! {
     /// Vector of four `f64` values
     struct f64x4([f64; 4]);
+    bits crate::u64x4;
 }
 
-define_vector! {
+define_float_vector! {
     /// Vector of eight `f64` values
     struct f64x8([f64; 8]);
+    bits crate::u64x8;
 }
 
 from_transmute_x86! { unsafe f64x2 => __m128d }
diff --git a/crates/core_simd/tests/ops_impl/float_macros.rs b/crates/core_simd/tests/ops_impl/float_macros.rs
index 9bcb894b2572..708b4d14327c 100644
--- a/crates/core_simd/tests/ops_impl/float_macros.rs
+++ b/crates/core_simd/tests/ops_impl/float_macros.rs
@@ -21,6 +21,26 @@ macro_rules! float_tests {
 
             const A: [$scalar; 16] = [0.,   1.,  2.,  3.,  4.,  5.,  6.,  7.,  8.,  9., 10., 11., 12., 13., 14., 15.];
             const B: [$scalar; 16] = [16., 17., 18., 19., 20., 21., 22., 23., 24., 25., 26., 27., 28., 29., 30., 31.];
+            const C: [$scalar; 16] = [
+                -0.0,
+                0.0,
+                -1.0,
+                1.0,
+                <$scalar>::MIN,
+                <$scalar>::MAX,
+                <$scalar>::INFINITY,
+                -<$scalar>::INFINITY,
+                <$scalar>::MIN_POSITIVE,
+                -<$scalar>::MIN_POSITIVE,
+                <$scalar>::EPSILON,
+                -<$scalar>::EPSILON,
+                0.0 / 0.0,
+                -0.0 / 0.0,
+                // Still not sure if wasm can have weird nans, or I'd check them
+                // too. Until then
+                1.0 / 3.0,
+                -1.0 / 4.0
+            ];
 
             #[test]
             #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
@@ -264,6 +284,30 @@ macro_rules! float_tests {
                 let expected = apply_unary_lanewise(v, core::ops::Neg::neg);
                 assert_biteq!(-v, expected);
             }
+
+            #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
+            fn abs_negative() {
+                let v = -from_slice(&A);
+                let expected = apply_unary_lanewise(v, <$scalar>::abs);
+                assert_biteq!(v.abs(), expected);
+            }
+
+            #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
+            fn abs_positive() {
+                let v = from_slice(&B);
+                let expected = apply_unary_lanewise(v, <$scalar>::abs);
+                assert_biteq!(v.abs(), expected);
+            }
+
+            #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
+            fn abs_odd_floats() {
+                let v = from_slice(&C);
+                let expected = apply_unary_lanewise(v, <$scalar>::abs);
+                assert_biteq!(v.abs(), expected);
+            }
         }
     }
 }

From 541369c38e6fac164198fd4298e043b34ccbef9f Mon Sep 17 00:00:00 2001
From: Thom Chiovoloni <chiovolonit@gmail.com>
Date: Tue, 6 Oct 2020 12:30:19 -0700
Subject: [PATCH 035/249] use NEG_INFINITY and NAN constants instead computing
 them

---
 crates/core_simd/tests/ops_impl/float_macros.rs | 13 ++++++-------
 1 file changed, 6 insertions(+), 7 deletions(-)

diff --git a/crates/core_simd/tests/ops_impl/float_macros.rs b/crates/core_simd/tests/ops_impl/float_macros.rs
index 708b4d14327c..141cb52207be 100644
--- a/crates/core_simd/tests/ops_impl/float_macros.rs
+++ b/crates/core_simd/tests/ops_impl/float_macros.rs
@@ -29,17 +29,16 @@ macro_rules! float_tests {
                 <$scalar>::MIN,
                 <$scalar>::MAX,
                 <$scalar>::INFINITY,
-                -<$scalar>::INFINITY,
+                <$scalar>::NEG_INFINITY,
                 <$scalar>::MIN_POSITIVE,
                 -<$scalar>::MIN_POSITIVE,
                 <$scalar>::EPSILON,
                 -<$scalar>::EPSILON,
-                0.0 / 0.0,
-                -0.0 / 0.0,
-                // Still not sure if wasm can have weird nans, or I'd check them
-                // too. Until then
-                1.0 / 3.0,
-                -1.0 / 4.0
+                <$scalar>::NAN,
+                -<$scalar>::NAN,
+                // TODO: Would be nice to check sNaN...
+                100.0 / 3.0,
+                -100.0 / 3.0,
             ];
 
             #[test]

From 8d3d616b130e65bcb926645597b69fb136cc03b5 Mon Sep 17 00:00:00 2001
From: Thom Chiovoloni <chiovolonit@gmail.com>
Date: Tue, 6 Oct 2020 13:40:39 -0700
Subject: [PATCH 036/249] Apply review feedback

---
 crates/core_simd/src/macros.rs                  |  3 +--
 crates/core_simd/tests/ops_impl/float_macros.rs | 12 +++++++++---
 2 files changed, 10 insertions(+), 5 deletions(-)

diff --git a/crates/core_simd/src/macros.rs b/crates/core_simd/src/macros.rs
index 214867c5ddce..f37d13c3ca32 100644
--- a/crates/core_simd/src/macros.rs
+++ b/crates/core_simd/src/macros.rs
@@ -295,8 +295,7 @@ macro_rules! impl_float_vector {
             #[inline]
             pub fn abs(self) -> Self {
                 let no_sign = <$bits_ty>::splat(!0 >> 1);
-                let abs = unsafe { crate::intrinsics::simd_and(self.to_bits(), no_sign) };
-                Self::from_bits(abs)
+                Self::from_bits(self.to_bits() & no_sign)
             }
         }
     };
diff --git a/crates/core_simd/tests/ops_impl/float_macros.rs b/crates/core_simd/tests/ops_impl/float_macros.rs
index 141cb52207be..1c969a2e8af3 100644
--- a/crates/core_simd/tests/ops_impl/float_macros.rs
+++ b/crates/core_simd/tests/ops_impl/float_macros.rs
@@ -19,6 +19,11 @@ macro_rules! float_tests {
                 value
             }
 
+            fn slice_chunks(slice: &[$scalar]) -> impl Iterator<Item = core_simd::$vector> + '_ {
+                let lanes = core::mem::size_of::<core_simd::$vector>() / core::mem::size_of::<$scalar>();
+                slice.chunks_exact(lanes).map(from_slice)
+            }
+
             const A: [$scalar; 16] = [0.,   1.,  2.,  3.,  4.,  5.,  6.,  7.,  8.,  9., 10., 11., 12., 13., 14., 15.];
             const B: [$scalar; 16] = [16., 17., 18., 19., 20., 21., 22., 23., 24., 25., 26., 27., 28., 29., 30., 31.];
             const C: [$scalar; 16] = [
@@ -303,9 +308,10 @@ macro_rules! float_tests {
             #[test]
             #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn abs_odd_floats() {
-                let v = from_slice(&C);
-                let expected = apply_unary_lanewise(v, <$scalar>::abs);
-                assert_biteq!(v.abs(), expected);
+                for v in slice_chunks(&C) {
+                    let expected = apply_unary_lanewise(v, <$scalar>::abs);
+                    assert_biteq!(v.abs(), expected);
+                }
             }
         }
     }

From e8a7f474bbdf4002da52f853f2ff6d48c880be5e Mon Sep 17 00:00:00 2001
From: Thom Chiovoloni <chiovolonit@gmail.com>
Date: Wed, 7 Oct 2020 01:39:56 -0700
Subject: [PATCH 037/249] Rewrite (and improve) CI.

---
 .github/workflows/ci.yml | 151 +++++++++++++++++++++++
 .travis.yml              | 257 ++++++---------------------------------
 2 files changed, 188 insertions(+), 220 deletions(-)
 create mode 100644 .github/workflows/ci.yml

diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
new file mode 100644
index 000000000000..2104c74a4d92
--- /dev/null
+++ b/.github/workflows/ci.yml
@@ -0,0 +1,151 @@
+name: CI
+
+on:
+  pull_request:
+  push:
+    branches:
+      - master
+
+env:
+  CARGO_NET_RETRY: 10
+  RUSTUP_MAX_RETRIES: 10
+
+jobs:
+  x86-tests:
+    name: "${{ matrix.target_feature }} on ${{ matrix.target }}"
+    runs-on: ${{ matrix.os }}
+    strategy:
+      fail-fast: false
+      matrix:
+        target: [x86_64-pc-windows-msvc, i686-pc-windows-msvc, i586-pc-windows-msvc, x86_64-unknown-linux-gnu, x86_64-apple-darwin]
+        # `default` means we use the default target config for the target,
+        # `native` means we run with `-Ctarget-cpu=native`, and anything else is
+        # an arg to `-Ctarget-feature`
+        target_feature: [default, native, +sse3, +ssse3, +sse4.1, +sse4.2, +avx, +avx2]
+
+        exclude:
+          # The macos runners seem to only reliably support up to `avx`.
+          - { target: x86_64-apple-darwin, target_feature: +avx2 }
+          # These features are statically known to be present for all 64 bit
+          # macs, and thus are covered by the `default` test
+          - { target: x86_64-apple-darwin, target_feature: +sse3 }
+          - { target: x86_64-apple-darwin, target_feature: +ssse3 }
+          # -Ctarget-cpu=native sounds like bad-news if target != host
+          - { target: i686-pc-windows-msvc, target_feature: native }
+          - { target: i586-pc-windows-msvc, target_feature: native }
+
+        include:
+          # Populate the `matrix.os` field
+          - { target: x86_64-apple-darwin,      os: macos-latest }
+          - { target: x86_64-unknown-linux-gnu, os: ubuntu-latest }
+          - { target: x86_64-pc-windows-msvc,   os: windows-latest }
+          - { target: i686-pc-windows-msvc,     os: windows-latest }
+          - { target: i586-pc-windows-msvc,     os: windows-latest }
+
+          # These are globally available on all the other targets.
+          - { target: i586-pc-windows-msvc, target_feature: +sse, os: windows-latest }
+          - { target: i586-pc-windows-msvc, target_feature: +sse2, os: windows-latest }
+
+          # Annoyingly, the x86_64-unknown-linux-gnu runner *almost* always has
+          # avx512vl, but occasionally doesn't. As a result, we still run that
+          # one under travis.
+
+    steps:
+      - uses: actions/checkout@v2
+      - name: Setup Rust
+        run: |
+          rustup update nightly --no-self-update
+          rustup default nightly
+          rustup target add ${{ matrix.target }}
+
+      - name: Configure RUSTFLAGS
+        shell: bash
+        run: |
+          case "${{ matrix.target_feature }}" in
+            default)
+              ;;
+            native)
+              echo "RUSTFLAGS=-Ctarget-cpu=native" >> $GITHUB_ENV
+              ;;
+            *)
+              echo "RUSTFLAGS=-Ctarget-feature=${{ matrix.target_feature }}" >> $GITHUB_ENV
+              ;;
+          esac
+
+      # Super useful for debugging why a SIGILL occurred.
+      - name: Dump target configuration and support
+        run: |
+          rustc -Vv
+
+          echo "Caveat: not all target features are expected to be logged"
+
+          echo "## Requested target configuration (RUSTFLAGS=$RUSTFLAGS)"
+          rustc --print=cfg --target=${{ matrix.target }} $RUSTFLAGS
+
+          echo "## Supported target configuration for --target=${{ matrix.target }}"
+          rustc --print=cfg --target=${{ matrix.target }} -Ctarget-cpu=native
+
+          echo "## Natively supported target configuration"
+          rustc --print=cfg -Ctarget-cpu=native
+
+      - name: Test (debug)
+        run: cargo test --verbose --target=${{ matrix.target }}
+
+      - name: Test (release)
+        run: cargo test --verbose --target=${{ matrix.target }} --release
+
+  cross-tests:
+    name: "${{ matrix.target }} (via cross)"
+    runs-on: ubuntu-latest
+    strategy:
+      fail-fast: false
+      # TODO: Sadly, we cant configure target-feature in a meaningful way
+      # because `cross` doesn't tell qemu to enable any non-default cpu
+      # features, nor does it give us a way to do so.
+      #
+      # Ultimately, we'd like to do something like [rust-lang/stdarch][stdarch].
+      # This is a lot more complex... but in practice it's likely that we can just
+      # snarf the docker config from around [here][1000-dockerfiles].
+      #
+      # [stdarch]: https://github.com/rust-lang/stdarch/blob/a5db4eaf/.github/workflows/main.yml#L67
+      # [1000-dockerfiles]: https://github.com/rust-lang/stdarch/tree/a5db4eaf/ci/docker
+
+      matrix:
+        target:
+          - i586-unknown-linux-gnu
+          # 32-bit arm has a few idiosyncracies like having subnormal flushing
+          # to zero on by default. Ideally we'd set
+          - armv7-unknown-linux-gnueabihf
+          # Note: The issue above means neither of these mips targets will use
+          # MSA (mips simd) but MIPS uses a nonstandard binary representation
+          # for NaNs which makes it worth testing on despite that.
+          - mips-unknown-linux-gnu
+          - mips64-unknown-linux-gnuabi64
+          - riscv64gc-unknown-linux-gnu
+
+    steps:
+      - uses: actions/checkout@v2
+      - name: Setup Rust
+        run: |
+          rustup update nightly --no-self-update
+          rustup default nightly
+          rustup target add ${{ matrix.target }}
+          rustup component add rust-src
+
+      - name: Install Cross
+        # Equivalent to `cargo install cross`, but downloading a prebuilt
+        # binary. Ideally we wouldn't hardcode a version, but the version number
+        # being part of the tarball means we can't just use the download/latest
+        # URL :(
+        run: |
+          CROSS_URL=https://github.com/rust-embedded/cross/releases/download/v0.2.1/cross-v0.2.1-x86_64-unknown-linux-gnu.tar.gz
+          mkdir -p "$HOME/.bin"
+          curl -sfSL --retry-delay 10 --retry 5 "${CROSS_URL}" | tar zxf - -C "$HOME/.bin"
+          echo "$HOME/.bin" >> $GITHUB_PATH
+
+      - name: Test (debug)
+        run: cross test --verbose --target=${{ matrix.target }}
+
+      - name: Test (release)
+        run: cross test --verbose --target=${{ matrix.target }} --release
+
diff --git a/.travis.yml b/.travis.yml
index b9df36386504..c1fda2e4f170 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -5,257 +5,74 @@ rust:
 matrix:
   fast_finish: true
   include:
-    # Linux (x86_64)
-
-    - name: "x86_64-unknown-linux-gnu"
+    # Linux (aarch64)
+    - name: "aarch64-unknown-linux-gnu (neon)"
       os: linux
-      arch: amd64
-      env:
-        - TARGET=x86_64-unknown-linux-gnu
+      arch: arm64
 
-    - name: "x86_64-unknown-linux-gnu+sse"
+    - name: "aarch64-unknown-linux-gnu (neon, sve)"
       os: linux
-      arch: amd64
-      env:
-        - TARGET=x86_64-unknown-linux-gnu
-        - TARGET_FEATURE=sse
+      arch: arm64
+      env: RUSTFLAGS=-Ctarget-feature=+sve
 
-    - name: "x86_64-unknown-linux-gnu+sse2"
+    - name: "aarch64-unknown-linux-gnu (native, see log for cfg)"
       os: linux
-      arch: amd64
-      env:
-        - TARGET=x86_64-unknown-linux-gnu
-        - TARGET_FEATURE=sse2
+      arch: arm64
+      env: RUSTFLAGS=-Ctarget-cpu=native
 
-    - name: "x86_64-unknown-linux-gnu+sse3"
+    # Linux (powerpc64le)
+    - name: "powerpc64le-unknown-linux-gnu (altivec, vsx, power8-*)"
       os: linux
-      arch: amd64
-      env:
-        - TARGET=x86_64-unknown-linux-gnu
-        - TARGET_FEATURE=sse3
+      arch: ppc64le
 
-    - name: "x86_64-unknown-linux-gnu+sse4.1"
+    - name: "powerpc64le-unknown-linux-gnu (native, see log for cfg)"
       os: linux
-      arch: amd64
-      env:
-        - TARGET=x86_64-unknown-linux-gnu
-        - TARGET_FEATURE=sse4.1
-
-    - name: "x86_64-unknown-linux-gnu+sse4.2"
-      os: linux
-      arch: amd64
-      env:
-        - TARGET=x86_64-unknown-linux-gnu
-        - TARGET_FEATURE=sse4.2
-
-    - name: "x86_64-unknown-linux-gnu+avx"
-      os: linux
-      arch: amd64
-      env:
-        - TARGET=x86_64-unknown-linux-gnu
-        - TARGET_FEATURE=avx
-
-    - name: "x86_64-unknown-linux-gnu+avx2"
-      os: linux
-      arch: amd64
-      env:
-        - TARGET=x86_64-unknown-linux-gnu
-        - TARGET_FEATURE=avx2
+      arch: ppc64le
+      env: RUSTFLAGS=-Ctarget-cpu=native
 
+    # Linux (x86_64) (for AVX512, which sadly seems to only *usually* be present
+    # on the github actions linux runner...)
     - name: "x86_64-unknown-linux-gnu+avx512vl"
       os: linux
       arch: amd64
-      env:
-        - TARGET=x86_64-unknown-linux-gnu
-        - TARGET_FEATURE=avx512vl
+      env: RUSTFLAGS=-Ctarget-feature=+avx512vl
 
-    # Linux (aarch64)
-
-    - name: "aarch64-unknown-linux-gnu"
-      os: linux
-      arch: arm64
-      env:
-        - TARGET=aarch64-unknown-linux-gnu
-
-    - name: "aarch64-unknown-linux-gnu+neon"
-      os: linux
-      arch: arm64
-      env:
-        - TARGET=aarch64-unknown-linux-gnu
-        - TARGET_FEATURE=neon
-
-    - name: "aarch64-unknown-linux-gnu+sve"
-      os: linux
-      arch: arm64
-      env:
-        - TARGET=aarch64-unknown-linux-gnu
-        - TARGET_FEATURE=sve
-
-    # Linux (powerpc64)
-
-    - name: "powerpc64le-unknown-linux-gnu"
-      os: linux
-      arch: ppc64le
-      env:
-        - TARGET=powerpc64le-unknown-linux-gnu
-
-    - name: "powerpc64le-unknown-linux-gnu+vsx"
-      os: linux
-      arch: ppc64le
-      env:
-        - TARGET=powerpc64le-unknown-linux-gnu
-        - TARGET_FEATURE=vsx
-
-    # Windows (x86_64)
-
-    - name: "x86_64-pc-windows-msvc"
-      os: windows
-      arch: amd64
-      env: TARGET=x86_64-pc-windows-msvc
-    
-    # Windows (i686)
-
-    - name: "i686-pc-windows-msvc"
-      os: windows
-      env: TARGET=i686-pc-windows-msvc
-
-    - name: "i686-pc-windows-msvc+sse"
-      os: windows
-      arch: amd64
-      env:
-        - TARGET=i686-pc-windows-msvc
-        - TARGET_FEATURE=sse
-
-    - name: "i686-pc-windows-msvc+sse2"
-      os: windows
-      arch: amd64
-      env:
-        - TARGET=i686-pc-windows-msvc
-        - TARGET_FEATURE=sse2
-
-    - name: "i686-pc-windows-msvc+sse3"
-      os: windows
-      arch: amd64
-      env:
-        - TARGET=i686-pc-windows-msvc
-        - TARGET_FEATURE=sse3
-
-    - name: "i686-pc-windows-msvc+sse4.1"
-      os: windows
-      arch: amd64
-      env:
-        - TARGET=i686-pc-windows-msvc
-        - TARGET_FEATURE=sse4.1
-
-    - name: "i686-pc-windows-msvc+sse4.2"
-      os: windows
-      arch: amd64
-      env:
-        - TARGET=i686-pc-windows-msvc
-        - TARGET_FEATURE=sse4.2
-
-    - name: "i686-pc-windows-msvc+avx"
-      os: windows
-      arch: amd64
-      env:
-        - TARGET=i686-pc-windows-msvc
-        - TARGET_FEATURE=avx
-
-    - name: "i686-pc-windows-msvc+avx2"
-      os: windows
-      arch: amd64
-      env:
-        - TARGET=i686-pc-windows-msvc
-        - TARGET_FEATURE=avx2
-
-    # Windows (i586)
-
-    - name: "i586-pc-windows-msvc"
-      os: windows
-      env: TARGET=i586-pc-windows-msvc
-
-    - name: "i586-pc-windows-msvc+sse"
-      os: windows
-      arch: amd64
-      env:
-        - TARGET=i586-pc-windows-msvc
-        - TARGET_FEATURE=sse
-
-    - name: "i586-pc-windows-msvc+sse2"
-      os: windows
-      arch: amd64
-      env:
-        - TARGET=i586-pc-windows-msvc
-        - TARGET_FEATURE=sse2
-
-    - name: "i586-pc-windows-msvc+sse3"
-      os: windows
-      arch: amd64
-      env:
-        - TARGET=i586-pc-windows-msvc
-        - TARGET_FEATURE=sse3
-
-    - name: "i586-pc-windows-msvc+sse4.1"
-      os: windows
-      arch: amd64
-      env:
-        - TARGET=i586-pc-windows-msvc
-        - TARGET_FEATURE=sse4.1
-
-    - name: "i586-pc-windows-msvc+sse4.2"
-      os: windows
-      arch: amd64
-      env:
-        - TARGET=i586-pc-windows-msvc
-        - TARGET_FEATURE=sse4.2
-
-    - name: "i586-pc-windows-msvc+avx"
-      os: windows
-      arch: amd64
-      env:
-        - TARGET=i586-pc-windows-msvc
-        - TARGET_FEATURE=avx
-
-    - name: "i586-pc-windows-msvc+avx2"
-      os: windows
-      arch: amd64
-      env:
-        - TARGET=i586-pc-windows-msvc
-        - TARGET_FEATURE=avx2
-
-    # OSX (x86_64)
-
-    - name: "x86_64-apple-darwin"
-      os: osx
-      arch: amd64
-      env:
-        - TARGET=x86_64-apple-darwin
-    
     # WebAssembly (wasm-bindgen)
-
     - name: "wasm32-unknown-unknown (node, firefox, chrome)"
       os: linux
       arch: amd64
       addons:
         firefox: latest
-        chrome : stable
+        chrome: stable
       install:
         - curl https://rustwasm.github.io/wasm-pack/installer/init.sh -sSf | sh
       script:
         - wasm-pack test --node --firefox --chrome --headless crates/core_simd
-    
+        - wasm-pack test --node --firefox --chrome --headless crates/core_simd --release
+
     - name: "wasm32-unknown-unknown+simd128 (chrome)"
       os: linux
       arch: amd64
       addons:
-        chrome : stable
+        chrome: stable
       install:
         - curl https://rustwasm.github.io/wasm-pack/installer/init.sh -sSf | sh
       script:
-        - RUSTFLAGS="-C target-feature=+simd128"
+        - export RUSTFLAGS="-C target-feature=+simd128"
         - wasm-pack test --chrome --headless crates/core_simd
+        - wasm-pack test --chrome --headless crates/core_simd --release
 
 script:
-  - rustup target add $TARGET
-  - if [ -n "$TARGET_FEATURE" ]; then RUSTFLAGS="-C target-feature=+$TARGET_FEATURE"; fi
-  - cargo test -v --target $TARGET
+  - echo "## Requested target configuration (RUSTFLAGS=$RUSTFLAGS)"
+  - rustc --print=cfg $RUSTFLAGS
+
+  - echo "## Supported target configuration"
+  - rustc --print=cfg -Ctarget-cpu=native
+
+  - echo "\n---\n"
+
+  - echo "## Running tests (debug)"
+  - cargo test -v
+
+  - echo "## Running tests (release)"
+  - cargo test -v --release

From 873639d6fd11c734cee6ae566e6e1e377f208fe7 Mon Sep 17 00:00:00 2001
From: Thom Chiovoloni <chiovolonit@gmail.com>
Date: Wed, 7 Oct 2020 11:51:54 -0700
Subject: [PATCH 038/249] Use bitxor to implement Neg for floats

---
 crates/core_simd/src/ops.rs                     | 15 +++++++++++++--
 crates/core_simd/tests/ops_impl/float_macros.rs |  9 +++++++++
 2 files changed, 22 insertions(+), 2 deletions(-)

diff --git a/crates/core_simd/src/ops.rs b/crates/core_simd/src/ops.rs
index 5af10a4e1886..9e0bc3f42a66 100644
--- a/crates/core_simd/src/ops.rs
+++ b/crates/core_simd/src/ops.rs
@@ -212,7 +212,18 @@ macro_rules! impl_op {
             impl core::ops::Neg for $type {
                 type Output = Self;
                 fn neg(self) -> Self::Output {
-                    <$type>::splat(-<$scalar>::default()) - self
+                    <$type>::splat(0) - self
+                }
+            }
+        }
+    };
+
+    { impl Neg for $type:ty, $scalar:ty, @float } => {
+        impl_ref_ops! {
+            impl core::ops::Neg for $type {
+                type Output = Self;
+                fn neg(self) -> Self::Output {
+                    Self::from_bits(<$type>::splat(-0.0).to_bits() ^ self.to_bits())
                 }
             }
         }
@@ -310,7 +321,7 @@ macro_rules! impl_float_ops {
                 impl_op! { impl Mul for $vector, $scalar }
                 impl_op! { impl Div for $vector, $scalar }
                 impl_op! { impl Rem for $vector, $scalar }
-                impl_op! { impl Neg for $vector, $scalar }
+                impl_op! { impl Neg for $vector, $scalar, @float }
                 impl_op! { impl Index for $vector, $scalar }
             )*
         )*
diff --git a/crates/core_simd/tests/ops_impl/float_macros.rs b/crates/core_simd/tests/ops_impl/float_macros.rs
index 1c969a2e8af3..7df30ec39f6c 100644
--- a/crates/core_simd/tests/ops_impl/float_macros.rs
+++ b/crates/core_simd/tests/ops_impl/float_macros.rs
@@ -289,6 +289,15 @@ macro_rules! float_tests {
                 assert_biteq!(-v, expected);
             }
 
+            #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
+            fn neg_odd_floats() {
+                for v in slice_chunks(&C) {
+                    let expected = apply_unary_lanewise(v, core::ops::Neg::neg);
+                    assert_biteq!(-v, expected);
+                }
+            }
+
             #[test]
             #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn abs_negative() {

From ffd562f2181f5969d56c4a6c9399be27058c8a74 Mon Sep 17 00:00:00 2001
From: Thom Chiovoloni <chiovolonit@gmail.com>
Date: Wed, 7 Oct 2020 13:24:21 -0700
Subject: [PATCH 039/249] Add comment about fneg to the bitxor in float neg

---
 crates/core_simd/src/ops.rs | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/crates/core_simd/src/ops.rs b/crates/core_simd/src/ops.rs
index 9e0bc3f42a66..5a186649821b 100644
--- a/crates/core_simd/src/ops.rs
+++ b/crates/core_simd/src/ops.rs
@@ -223,6 +223,8 @@ macro_rules! impl_op {
             impl core::ops::Neg for $type {
                 type Output = Self;
                 fn neg(self) -> Self::Output {
+                    // FIXME: Replace this with fneg intrinsic once available.
+                    // https://github.com/rust-lang/stdsimd/issues/32
                     Self::from_bits(<$type>::splat(-0.0).to_bits() ^ self.to_bits())
                 }
             }

From ffedbe5b1501c365f2e8f5fc46d260d2af17aa63 Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Sat, 10 Oct 2020 00:16:58 -0400
Subject: [PATCH 040/249] Add rounding functions

---
 crates/core_simd/src/lib.rs   |   4 +-
 crates/core_simd/src/round.rs | 124 ++++++++++++++++++++++++++++++++++
 2 files changed, 127 insertions(+), 1 deletion(-)
 create mode 100644 crates/core_simd/src/round.rs

diff --git a/crates/core_simd/src/lib.rs b/crates/core_simd/src/lib.rs
index d08ef400f9bb..cea39e6f3f3d 100644
--- a/crates/core_simd/src/lib.rs
+++ b/crates/core_simd/src/lib.rs
@@ -1,5 +1,5 @@
 #![no_std]
-#![feature(repr_simd, platform_intrinsics)]
+#![feature(repr_simd, platform_intrinsics, link_llvm_intrinsics, simd_ffi)]
 #![warn(missing_docs)]
 //! Portable SIMD module.
 
@@ -56,3 +56,5 @@ mod vectors_mask128;
 pub use vectors_mask128::*;
 mod vectors_masksize;
 pub use vectors_masksize::*;
+
+mod round;
diff --git a/crates/core_simd/src/round.rs b/crates/core_simd/src/round.rs
new file mode 100644
index 000000000000..43285cdd3f7f
--- /dev/null
+++ b/crates/core_simd/src/round.rs
@@ -0,0 +1,124 @@
+macro_rules! implement {
+    {
+        impl $type:ident {
+            floor = $floor_intrinsic:literal,
+            ceil = $ceil_intrinsic:literal,
+            round = $round_intrinsic:literal,
+            trunc = $trunc_intrinsic:literal,
+        }
+    } => {
+        mod $type {
+            #[allow(improper_ctypes)]
+            extern "C" {
+                #[link_name = $floor_intrinsic]
+                fn floor_intrinsic(x: crate::$type) -> crate::$type;
+                #[link_name = $ceil_intrinsic]
+                fn ceil_intrinsic(x: crate::$type) -> crate::$type;
+                #[link_name = $round_intrinsic]
+                fn round_intrinsic(x: crate::$type) -> crate::$type;
+                #[link_name = $trunc_intrinsic]
+                fn trunc_intrinsic(x: crate::$type) -> crate::$type;
+            }
+
+            impl crate::$type {
+                /// Returns the largest integer less than or equal to each lane.
+                #[must_use = "method returns a new vector and does not mutate the original value"]
+                #[inline]
+                pub fn floor(self) -> Self {
+                    unsafe { floor_intrinsic(self) }
+                }
+
+                /// Returns the smallest integer greater than or equal to each lane.
+                #[must_use = "method returns a new vector and does not mutate the original value"]
+                #[inline]
+                pub fn ceil(self) -> Self {
+                    unsafe { ceil_intrinsic(self) }
+                }
+
+                /// Returns the nearest integer to each lane. Round half-way cases away from 0.0.
+                #[must_use = "method returns a new vector and does not mutate the original value"]
+                #[inline]
+                pub fn round(self) -> Self {
+                    unsafe { round_intrinsic(self) }
+                }
+
+                /// Returns the integer part of each lane.
+                #[must_use = "method returns a new vector and does not mutate the original value"]
+                #[inline]
+                pub fn trunc(self) -> Self {
+                    unsafe { trunc_intrinsic(self) }
+                }
+
+                /// Returns the fractional part of each lane.
+                #[must_use = "method returns a new vector and does not mutate the original value"]
+                #[inline]
+                pub fn fract(self) -> Self {
+                    self - self.trunc()
+                }
+            }
+        }
+    }
+}
+
+implement! {
+    impl f32x2 {
+        floor = "llvm.floor.v2f32",
+        ceil = "llvm.ceil.v2f32",
+        round = "llvm.round.v2f32",
+        trunc = "llvm.trunc.v2f32",
+    }
+}
+
+implement! {
+    impl f32x4 {
+        floor = "llvm.floor.v4f32",
+        ceil = "llvm.ceil.v4f32",
+        round = "llvm.round.v4f32",
+        trunc = "llvm.trunc.v4f32",
+    }
+}
+
+implement! {
+    impl f32x8 {
+        floor = "llvm.floor.v8f32",
+        ceil = "llvm.ceil.v8f32",
+        round = "llvm.round.v8f32",
+        trunc = "llvm.trunc.v8f32",
+    }
+}
+
+implement! {
+    impl f32x16 {
+        floor = "llvm.floor.v16f32",
+        ceil = "llvm.ceil.v16f32",
+        round = "llvm.round.v16f32",
+        trunc = "llvm.trunc.v16f32",
+    }
+}
+
+implement! {
+    impl f64x2 {
+        floor = "llvm.floor.v2f64",
+        ceil = "llvm.ceil.v2f64",
+        round = "llvm.round.v2f64",
+        trunc = "llvm.trunc.v2f64",
+    }
+}
+
+implement! {
+    impl f64x4 {
+        floor = "llvm.floor.v4f64",
+        ceil = "llvm.ceil.v4f64",
+        round = "llvm.round.v4f64",
+        trunc = "llvm.trunc.v4f64",
+    }
+}
+
+implement! {
+    impl f64x8 {
+        floor = "llvm.floor.v8f64",
+        ceil = "llvm.ceil.v8f64",
+        round = "llvm.round.v8f64",
+        trunc = "llvm.trunc.v8f64",
+    }
+}

From e4332915a721fa56fdb251dcf7894f6a3abcc2c7 Mon Sep 17 00:00:00 2001
From: Jubilee <46493976+workingjubilee@users.noreply.github.com>
Date: Fri, 9 Oct 2020 23:49:43 -0700
Subject: [PATCH 041/249] Draft a CONTRIBUTING.md (#33)

---
 CONTRIBUTING.md | 32 ++++++++++++++++++++++++++++++++
 1 file changed, 32 insertions(+)
 create mode 100644 CONTRIBUTING.md

diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
new file mode 100644
index 000000000000..f9ba12d3a1b3
--- /dev/null
+++ b/CONTRIBUTING.md
@@ -0,0 +1,32 @@
+# Contributing to `std::simd`
+
+Simple version:
+1. Fork it and `git clone` it
+2. Create your feature branch: `git checkout -b my-branch`
+3. Write your changes.
+4. Test it: `cargo test`. Remember to enable whatever SIMD features you intend to test by setting `RUSTFLAGS`.
+5. Commit your changes: `git commit add ./path/to/changes && git commit -m 'Fix some bug'`
+6. Push the branch: `git push --set-upstream origin my-branch`
+7. Submit a pull request!
+
+## Taking on an Issue
+
+SIMD can be quite complex, and even a "simple" issue can be huge. If an issue is organized like a tracking issue, with an itemized list of items that don't necessarily have to be done in a specific order, please take the issue one item at a time. This will help by letting work proceed apace on the rest of the issue. If it's a (relatively) small issue, feel free to announce your intention to solve it on the issue tracker and take it in one go!
+
+## CI
+
+We currently have 2 CI matrices through Travis CI and GitHub Actions that will automatically build and test your change in order to verify that `std::simd`'s portable API is, in fact, portable. If your change builds locally, but does not build on either, this is likely due to a platform-specific concern that your code has not addressed. Please consult the build logs and address the error, or ask for help if you need it.
+
+## Beyond stdsimd
+
+A large amount of the core SIMD implementation is found in the rustc_codegen_* crates in the [main rustc repo](https://github.com/rust-lang/rust). In addition, actual platform-specific functions are implemented in [stdarch]. Not all changes to `std::simd` require interacting with either of these, but if you're wondering where something is and it doesn't seem to be in this repository, those might be where to start looking.
+
+## Questions? Concerns? Need Help?
+
+Please feel free to ask in the [#project-portable-simd][zulip-portable-simd] stream on the [rust-lang Zulip][zulip] for help with making changes to `std::simd`!
+If your changes include directly modifying the compiler, it might also be useful to ask in [#t-compiler/help][zulip-compiler-help].
+
+[zulip-portable-simd]: https://rust-lang.zulipchat.com/#narrow/stream/257879-project-portable-simd
+[zulip-compiler-help]: https://rust-lang.zulipchat.com/#narrow/stream/182449-t-compiler.2Fhelp
+[zulip]: https://rust-lang.zulipchat.com
+[stdarch]: https://github.com/rust-lang/stdarch

From 75fdacde1ce6da3a93058a9fca248bfaf2611cd3 Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Sun, 11 Oct 2020 13:22:26 -0400
Subject: [PATCH 042/249] Add rounding to integers

---
 crates/core_simd/src/round.rs | 25 +++++++++++++++++++++++++
 1 file changed, 25 insertions(+)

diff --git a/crates/core_simd/src/round.rs b/crates/core_simd/src/round.rs
index 43285cdd3f7f..6a03f7f81768 100644
--- a/crates/core_simd/src/round.rs
+++ b/crates/core_simd/src/round.rs
@@ -1,10 +1,12 @@
 macro_rules! implement {
     {
         impl $type:ident {
+            int_type = $int_type:ident,
             floor = $floor_intrinsic:literal,
             ceil = $ceil_intrinsic:literal,
             round = $round_intrinsic:literal,
             trunc = $trunc_intrinsic:literal,
+            round_to_int = $round_to_int_intrinsic:literal,
         }
     } => {
         mod $type {
@@ -18,6 +20,8 @@ macro_rules! implement {
                 fn round_intrinsic(x: crate::$type) -> crate::$type;
                 #[link_name = $trunc_intrinsic]
                 fn trunc_intrinsic(x: crate::$type) -> crate::$type;
+                #[link_name = $round_to_int_intrinsic]
+                fn round_to_int_intrinsic(x: crate::$type) -> crate::$int_type;
             }
 
             impl crate::$type {
@@ -55,6 +59,13 @@ macro_rules! implement {
                 pub fn fract(self) -> Self {
                     self - self.trunc()
                 }
+
+                /// Returns the nearest integer to each lane. Round half-way cases away from 0.0.
+                #[must_use = "method returns a new vector and does not mutate the original value"]
+                #[inline]
+                pub fn round_to_int(self) -> crate::$int_type {
+                    unsafe { round_to_int_intrinsic(self) }
+                }
             }
         }
     }
@@ -62,63 +73,77 @@ macro_rules! implement {
 
 implement! {
     impl f32x2 {
+        int_type = i32x2,
         floor = "llvm.floor.v2f32",
         ceil = "llvm.ceil.v2f32",
         round = "llvm.round.v2f32",
         trunc = "llvm.trunc.v2f32",
+        round_to_int = "llvm.lround.i32.v2f32",
     }
 }
 
 implement! {
     impl f32x4 {
+        int_type = i32x4,
         floor = "llvm.floor.v4f32",
         ceil = "llvm.ceil.v4f32",
         round = "llvm.round.v4f32",
         trunc = "llvm.trunc.v4f32",
+        round_to_int = "llvm.lround.i32.v4f32",
     }
 }
 
 implement! {
     impl f32x8 {
+        int_type = i32x8,
         floor = "llvm.floor.v8f32",
         ceil = "llvm.ceil.v8f32",
         round = "llvm.round.v8f32",
         trunc = "llvm.trunc.v8f32",
+        round_to_int = "llvm.lround.i32.v8f32",
     }
 }
 
 implement! {
     impl f32x16 {
+        int_type = i32x16,
         floor = "llvm.floor.v16f32",
         ceil = "llvm.ceil.v16f32",
         round = "llvm.round.v16f32",
         trunc = "llvm.trunc.v16f32",
+        round_to_int = "llvm.lround.i32.v16f32",
     }
 }
 
 implement! {
     impl f64x2 {
+        int_type = i64x2,
         floor = "llvm.floor.v2f64",
         ceil = "llvm.ceil.v2f64",
         round = "llvm.round.v2f64",
         trunc = "llvm.trunc.v2f64",
+        round_to_int = "llvm.lround.i64.v2f64",
     }
 }
 
 implement! {
     impl f64x4 {
+        int_type = i64x4,
         floor = "llvm.floor.v4f64",
         ceil = "llvm.ceil.v4f64",
         round = "llvm.round.v4f64",
         trunc = "llvm.trunc.v4f64",
+        round_to_int = "llvm.lround.i64.v4f64",
     }
 }
 
 implement! {
     impl f64x8 {
+        int_type = i64x8,
         floor = "llvm.floor.v8f64",
         ceil = "llvm.ceil.v8f64",
         round = "llvm.round.v8f64",
         trunc = "llvm.trunc.v8f64",
+        round_to_int = "llvm.lround.i64.v8f64",
     }
 }

From 3d8721b053ce0a7118b037fdc52a804c58f94822 Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Sun, 11 Oct 2020 14:32:46 -0400
Subject: [PATCH 043/249] Fix casts, add tests

---
 crates/core_simd/src/intrinsics.rs            |   3 +
 crates/core_simd/src/round.rs                 |  31 ++---
 crates/core_simd/tests/helpers/lanewise.rs    |  13 +-
 crates/core_simd/tests/ops_impl/f32.rs        |   8 +-
 crates/core_simd/tests/ops_impl/f64.rs        |   6 +-
 .../core_simd/tests/ops_impl/float_macros.rs  | 115 +++++++++++++++++-
 6 files changed, 150 insertions(+), 26 deletions(-)

diff --git a/crates/core_simd/src/intrinsics.rs b/crates/core_simd/src/intrinsics.rs
index c2cef778560f..acf85983a983 100644
--- a/crates/core_simd/src/intrinsics.rs
+++ b/crates/core_simd/src/intrinsics.rs
@@ -36,4 +36,7 @@ extern "platform-intrinsic" {
 
     /// xor
     pub(crate) fn simd_xor<T>(x: T, y: T) -> T;
+
+    /// fptoui/fptosi/uitofp/sitofp
+    pub(crate) fn simd_cast<T, U>(x: T) -> U;
 }
diff --git a/crates/core_simd/src/round.rs b/crates/core_simd/src/round.rs
index 6a03f7f81768..aadade1fd066 100644
--- a/crates/core_simd/src/round.rs
+++ b/crates/core_simd/src/round.rs
@@ -6,7 +6,6 @@ macro_rules! implement {
             ceil = $ceil_intrinsic:literal,
             round = $round_intrinsic:literal,
             trunc = $trunc_intrinsic:literal,
-            round_to_int = $round_to_int_intrinsic:literal,
         }
     } => {
         mod $type {
@@ -20,8 +19,6 @@ macro_rules! implement {
                 fn round_intrinsic(x: crate::$type) -> crate::$type;
                 #[link_name = $trunc_intrinsic]
                 fn trunc_intrinsic(x: crate::$type) -> crate::$type;
-                #[link_name = $round_to_int_intrinsic]
-                fn round_to_int_intrinsic(x: crate::$type) -> crate::$int_type;
             }
 
             impl crate::$type {
@@ -60,11 +57,24 @@ macro_rules! implement {
                     self - self.trunc()
                 }
 
-                /// Returns the nearest integer to each lane. Round half-way cases away from 0.0.
-                #[must_use = "method returns a new vector and does not mutate the original value"]
+                /// Rounds toward zero and converts to the same-width integer type, assuming that
+                /// the value is finite and fits in that type.
+                ///
+                /// # Safety
+                /// The value must:
+                ///
+                /// * Not be NaN
+                /// * Not be infinite
+                /// * Be representable in the return type, after truncating off its fractional part
                 #[inline]
-                pub fn round_to_int(self) -> crate::$int_type {
-                    unsafe { round_to_int_intrinsic(self) }
+                pub unsafe fn to_int_unchecked(self) -> crate::$int_type {
+                    crate::intrinsics::simd_cast(self)
+                }
+
+                /// Returns the nearest integer to each lane. Round half-way cases away from 0.0.
+                #[inline]
+                pub fn round_from_int(value: crate::$int_type) -> Self {
+                    unsafe { crate::intrinsics::simd_cast(value) }
                 }
             }
         }
@@ -78,7 +88,6 @@ implement! {
         ceil = "llvm.ceil.v2f32",
         round = "llvm.round.v2f32",
         trunc = "llvm.trunc.v2f32",
-        round_to_int = "llvm.lround.i32.v2f32",
     }
 }
 
@@ -89,7 +98,6 @@ implement! {
         ceil = "llvm.ceil.v4f32",
         round = "llvm.round.v4f32",
         trunc = "llvm.trunc.v4f32",
-        round_to_int = "llvm.lround.i32.v4f32",
     }
 }
 
@@ -100,7 +108,6 @@ implement! {
         ceil = "llvm.ceil.v8f32",
         round = "llvm.round.v8f32",
         trunc = "llvm.trunc.v8f32",
-        round_to_int = "llvm.lround.i32.v8f32",
     }
 }
 
@@ -111,7 +118,6 @@ implement! {
         ceil = "llvm.ceil.v16f32",
         round = "llvm.round.v16f32",
         trunc = "llvm.trunc.v16f32",
-        round_to_int = "llvm.lround.i32.v16f32",
     }
 }
 
@@ -122,7 +128,6 @@ implement! {
         ceil = "llvm.ceil.v2f64",
         round = "llvm.round.v2f64",
         trunc = "llvm.trunc.v2f64",
-        round_to_int = "llvm.lround.i64.v2f64",
     }
 }
 
@@ -133,7 +138,6 @@ implement! {
         ceil = "llvm.ceil.v4f64",
         round = "llvm.round.v4f64",
         trunc = "llvm.trunc.v4f64",
-        round_to_int = "llvm.lround.i64.v4f64",
     }
 }
 
@@ -144,6 +148,5 @@ implement! {
         ceil = "llvm.ceil.v8f64",
         round = "llvm.round.v8f64",
         trunc = "llvm.trunc.v8f64",
-        round_to_int = "llvm.lround.i64.v8f64",
     }
 }
diff --git a/crates/core_simd/tests/helpers/lanewise.rs b/crates/core_simd/tests/helpers/lanewise.rs
index 6ab7803a9678..15f1a88ffd5c 100644
--- a/crates/core_simd/tests/helpers/lanewise.rs
+++ b/crates/core_simd/tests/helpers/lanewise.rs
@@ -1,8 +1,13 @@
-pub fn apply_unary_lanewise<T: Copy, V: AsMut<[T]> + Default>(mut x: V, f: impl Fn(T) -> T) -> V {
-    for lane in x.as_mut() {
-        *lane = f(*lane)
+pub fn apply_unary_lanewise<T1: Copy, T2: Copy, V1: AsRef<[T1]>, V2: AsMut<[T2]> + Default>(
+    x: V1,
+    f: impl Fn(T1) -> T2,
+) -> V2 {
+    let mut y = V2::default();
+    assert_eq!(x.as_ref().len(), y.as_mut().len());
+    for (x, y) in x.as_ref().iter().zip(y.as_mut().iter_mut()) {
+        *y = f(*x);
     }
-    x
+    y
 }
 
 pub fn apply_binary_lanewise<T: Copy, V: AsRef<[T]> + AsMut<[T]> + Default>(
diff --git a/crates/core_simd/tests/ops_impl/f32.rs b/crates/core_simd/tests/ops_impl/f32.rs
index f87909b68cd3..1472822fe1fe 100644
--- a/crates/core_simd/tests/ops_impl/f32.rs
+++ b/crates/core_simd/tests/ops_impl/f32.rs
@@ -1,6 +1,6 @@
 use super::helpers;
 
-float_tests! { f32x2, f32 }
-float_tests! { f32x4, f32 }
-float_tests! { f32x8, f32 }
-float_tests! { f32x16, f32 }
+float_tests! { f32x2, f32, i32x2, i32 }
+float_tests! { f32x4, f32, i32x4, i32 }
+float_tests! { f32x8, f32, i32x8, i32 }
+float_tests! { f32x16, f32, i32x16, i32 }
diff --git a/crates/core_simd/tests/ops_impl/f64.rs b/crates/core_simd/tests/ops_impl/f64.rs
index 19ae476bd0e1..8f573baa1ad2 100644
--- a/crates/core_simd/tests/ops_impl/f64.rs
+++ b/crates/core_simd/tests/ops_impl/f64.rs
@@ -1,5 +1,5 @@
 use super::helpers;
 
-float_tests! { f64x2, f64 }
-float_tests! { f64x4, f64 }
-float_tests! { f64x8, f64 }
+float_tests! { f64x2, f64, i64x2, i64 }
+float_tests! { f64x4, f64, i64x4, i64 }
+float_tests! { f64x8, f64, i64x8, i64 }
diff --git a/crates/core_simd/tests/ops_impl/float_macros.rs b/crates/core_simd/tests/ops_impl/float_macros.rs
index 7df30ec39f6c..6f6f6385d976 100644
--- a/crates/core_simd/tests/ops_impl/float_macros.rs
+++ b/crates/core_simd/tests/ops_impl/float_macros.rs
@@ -1,5 +1,5 @@
 macro_rules! float_tests {
-    { $vector:ident, $scalar:ident } => {
+    { $vector:ident, $scalar:ident, $int_vector:ident, $int_scalar:ident } => {
         #[cfg(test)]
         mod $vector {
             use super::*;
@@ -24,6 +24,18 @@ macro_rules! float_tests {
                 slice.chunks_exact(lanes).map(from_slice)
             }
 
+            fn from_slice_int(slice: &[$int_scalar]) -> core_simd::$int_vector {
+                let mut value = core_simd::$int_vector::default();
+                let value_slice: &mut [_] = value.as_mut();
+                value_slice.copy_from_slice(&slice[0..value_slice.len()]);
+                value
+            }
+
+            fn slice_chunks_int(slice: &[$int_scalar]) -> impl Iterator<Item = core_simd::$int_vector> + '_ {
+                let lanes = core::mem::size_of::<core_simd::$int_vector>() / core::mem::size_of::<$int_scalar>();
+                slice.chunks_exact(lanes).map(from_slice_int)
+            }
+
             const A: [$scalar; 16] = [0.,   1.,  2.,  3.,  4.,  5.,  6.,  7.,  8.,  9., 10., 11., 12., 13., 14., 15.];
             const B: [$scalar; 16] = [16., 17., 18., 19., 20., 21., 22., 23., 24., 25., 26., 27., 28., 29., 30., 31.];
             const C: [$scalar; 16] = [
@@ -322,6 +334,107 @@ macro_rules! float_tests {
                     assert_biteq!(v.abs(), expected);
                 }
             }
+
+            #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
+            fn ceil_odd_floats() {
+                for v in slice_chunks(&C) {
+                    let expected = apply_unary_lanewise(v, <$scalar>::ceil);
+                    assert_biteq!(v.ceil(), expected);
+                }
+            }
+
+            #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
+            fn floor_odd_floats() {
+                for v in slice_chunks(&C) {
+                    let expected = apply_unary_lanewise(v, <$scalar>::floor);
+                    assert_biteq!(v.floor(), expected);
+                }
+            }
+
+            #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
+            fn round_odd_floats() {
+                for v in slice_chunks(&C) {
+                    let expected = apply_unary_lanewise(v, <$scalar>::round);
+                    assert_biteq!(v.round(), expected);
+                }
+            }
+
+            #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
+            fn trunc_odd_floats() {
+                for v in slice_chunks(&C) {
+                    let expected = apply_unary_lanewise(v, <$scalar>::trunc);
+                    assert_biteq!(v.trunc(), expected);
+                }
+            }
+
+            #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
+            fn fract_odd_floats() {
+                for v in slice_chunks(&C) {
+                    let expected = apply_unary_lanewise(v, <$scalar>::fract);
+                    assert_biteq!(v.fract(), expected);
+                }
+            }
+
+            #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
+            fn to_int_unchecked() {
+                const VALUES: [$scalar; 16] = [
+                    -0.0,
+                    0.0,
+                    -1.0,
+                    1.0,
+                    <$scalar>::MIN,
+                    -<$scalar>::MIN,
+                    <$scalar>::MIN_POSITIVE,
+                    -<$scalar>::MIN_POSITIVE,
+                    <$scalar>::EPSILON,
+                    -<$scalar>::EPSILON,
+                    core::$scalar::consts::PI,
+                    -core::$scalar::consts::PI,
+                    core::$scalar::consts::TAU,
+                    -core::$scalar::consts::TAU,
+                    100.0 / 3.0,
+                    -100.0 / 3.0,
+                ];
+
+                for v in slice_chunks(&VALUES) {
+                    let expected = apply_unary_lanewise(v, |x| unsafe { x.to_int_unchecked() });
+                    assert_biteq!(unsafe { v.to_int_unchecked() }, expected);
+                }
+            }
+
+            #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
+            fn round_from_int() {
+                const VALUES: [$int_scalar; 16] = [
+                    0,
+                    0,
+                    1,
+                    -1,
+                    100,
+                    -100,
+                    200,
+                    -200,
+                    413,
+                    -413,
+                    1017,
+                    -1017,
+                    1234567,
+                    -1234567,
+                    <$int_scalar>::MAX,
+                    <$int_scalar>::MIN,
+                ];
+
+                for v in slice_chunks_int(&VALUES) {
+                    let expected = apply_unary_lanewise(v, |x| x as $scalar);
+                    assert_biteq!(core_simd::$vector::round_from_int(v), expected);
+                }
+            }
         }
     }
 }

From 5805c7a05150688242ec260205c91bd3721db7a7 Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Sun, 11 Oct 2020 14:34:57 -0400
Subject: [PATCH 044/249] Fix comment

---
 crates/core_simd/src/round.rs | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/crates/core_simd/src/round.rs b/crates/core_simd/src/round.rs
index aadade1fd066..021e5af88737 100644
--- a/crates/core_simd/src/round.rs
+++ b/crates/core_simd/src/round.rs
@@ -71,7 +71,8 @@ macro_rules! implement {
                     crate::intrinsics::simd_cast(self)
                 }
 
-                /// Returns the nearest integer to each lane. Round half-way cases away from 0.0.
+                /// Creates a floating-point vector from an integer vector.  Rounds values that are
+                /// not exactly representable.
                 #[inline]
                 pub fn round_from_int(value: crate::$int_type) -> Self {
                     unsafe { crate::intrinsics::simd_cast(value) }

From 6e07982c4c3a1667b546722bbb63dda27cd14b56 Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Sun, 11 Oct 2020 19:08:25 -0400
Subject: [PATCH 045/249] Fix UB in test

---
 crates/core_simd/tests/ops_impl/float_macros.rs | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/crates/core_simd/tests/ops_impl/float_macros.rs b/crates/core_simd/tests/ops_impl/float_macros.rs
index 6f6f6385d976..36854d231e6f 100644
--- a/crates/core_simd/tests/ops_impl/float_macros.rs
+++ b/crates/core_simd/tests/ops_impl/float_macros.rs
@@ -388,8 +388,8 @@ macro_rules! float_tests {
                     0.0,
                     -1.0,
                     1.0,
-                    <$scalar>::MIN,
-                    -<$scalar>::MIN,
+                    <$int_scalar>::MAX as $scalar,
+                    <$int_scalar>::MIN as $scalar,
                     <$scalar>::MIN_POSITIVE,
                     -<$scalar>::MIN_POSITIVE,
                     <$scalar>::EPSILON,

From c27c76182b6ced0ed9e91755e31279492b3aa36b Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Sun, 11 Oct 2020 21:28:50 -0400
Subject: [PATCH 046/249] Fix UB in test (really this time)

---
 crates/core_simd/tests/ops_impl/float_macros.rs | 17 +++++++++++------
 1 file changed, 11 insertions(+), 6 deletions(-)

diff --git a/crates/core_simd/tests/ops_impl/float_macros.rs b/crates/core_simd/tests/ops_impl/float_macros.rs
index 36854d231e6f..c1e0072950f8 100644
--- a/crates/core_simd/tests/ops_impl/float_macros.rs
+++ b/crates/core_simd/tests/ops_impl/float_macros.rs
@@ -383,21 +383,26 @@ macro_rules! float_tests {
             #[test]
             #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn to_int_unchecked() {
+                // The maximum integer that can be represented by the equivalently sized float has
+                // all of the mantissa digits set to 1, pushed up to the MSB.
+                const ALL_MANTISSA_BITS: $int_scalar = ((1 << <$scalar>::MANTISSA_DIGITS) - 1);
+                const MAX_REPRESENTABLE_VALUE: $int_scalar =
+                    ALL_MANTISSA_BITS << (core::mem::size_of::<$scalar>() * 8 as usize - <$scalar>::MANTISSA_DIGITS as usize);
                 const VALUES: [$scalar; 16] = [
                     -0.0,
                     0.0,
                     -1.0,
                     1.0,
-                    <$int_scalar>::MAX as $scalar,
-                    <$int_scalar>::MIN as $scalar,
+                    ALL_MANTISSA_BITS as $scalar,
+                    -ALL_MANTISSA_BITS as $scalar,
+                    MAX_REPRESENTABLE_VALUE as $scalar,
+                    -MAX_REPRESENTABLE_VALUE as $scalar,
+                    (MAX_REPRESENTABLE_VALUE / 2) as $scalar,
+                    (-MAX_REPRESENTABLE_VALUE / 2) as $scalar,
                     <$scalar>::MIN_POSITIVE,
                     -<$scalar>::MIN_POSITIVE,
                     <$scalar>::EPSILON,
                     -<$scalar>::EPSILON,
-                    core::$scalar::consts::PI,
-                    -core::$scalar::consts::PI,
-                    core::$scalar::consts::TAU,
-                    -core::$scalar::consts::TAU,
                     100.0 / 3.0,
                     -100.0 / 3.0,
                 ];

From dc85c13ff3670a5a2e1bdcc4b50d536eff101b0c Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Sun, 11 Oct 2020 21:41:26 -0400
Subject: [PATCH 047/249] Account for sign bit

---
 crates/core_simd/tests/ops_impl/float_macros.rs | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/crates/core_simd/tests/ops_impl/float_macros.rs b/crates/core_simd/tests/ops_impl/float_macros.rs
index c1e0072950f8..e63b689a1755 100644
--- a/crates/core_simd/tests/ops_impl/float_macros.rs
+++ b/crates/core_simd/tests/ops_impl/float_macros.rs
@@ -387,7 +387,7 @@ macro_rules! float_tests {
                 // all of the mantissa digits set to 1, pushed up to the MSB.
                 const ALL_MANTISSA_BITS: $int_scalar = ((1 << <$scalar>::MANTISSA_DIGITS) - 1);
                 const MAX_REPRESENTABLE_VALUE: $int_scalar =
-                    ALL_MANTISSA_BITS << (core::mem::size_of::<$scalar>() * 8 as usize - <$scalar>::MANTISSA_DIGITS as usize);
+                    ALL_MANTISSA_BITS << (core::mem::size_of::<$scalar>() * 8 - <$scalar>::MANTISSA_DIGITS as usize - 1);
                 const VALUES: [$scalar; 16] = [
                     -0.0,
                     0.0,

From 3ad356d90268e5dd3236f460422df6136e9d0e11 Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Mon, 12 Oct 2020 15:36:14 -0400
Subject: [PATCH 048/249] Disable riscv64gc

---
 .github/workflows/ci.yml | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
index 2104c74a4d92..8d3312576927 100644
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -121,7 +121,8 @@ jobs:
           # for NaNs which makes it worth testing on despite that.
           - mips-unknown-linux-gnu
           - mips64-unknown-linux-gnuabi64
-          - riscv64gc-unknown-linux-gnu
+          # TODO: reenable pending https://github.com/rust-lang/rust/issues/77866
+          # - riscv64gc-unknown-linux-gnu
 
     steps:
       - uses: actions/checkout@v2

From db7d98675bf734d923ec0386e5ca98324aa756b8 Mon Sep 17 00:00:00 2001
From: Jubilee <46493976+workingjubilee@users.noreply.github.com>
Date: Mon, 12 Oct 2020 14:31:55 -0700
Subject: [PATCH 049/249] Add cross-links to Zulip, etc.

---
 README.md | 11 ++++++++++-
 1 file changed, 10 insertions(+), 1 deletion(-)

diff --git a/README.md b/README.md
index 58dd50ae1247..049c8b097e7e 100644
--- a/README.md
+++ b/README.md
@@ -1,8 +1,13 @@
 # stdsimd - Rust's standard library portable SIMD API
-
 [![Build Status](https://travis-ci.com/rust-lang/stdsimd.svg?branch=master)](https://travis-ci.com/rust-lang/stdsimd)
 
 Code repository for the [Portable SIMD Project Group](https://github.com/rust-lang/project-portable-simd).
+Please refer to [CONTRIBUTING.md](./CONTRIBUTING.md) for our contributing guidelines.
+
+If you have questions about SIMD, we have begun writing a [guide][simd-guide].
+We can also be found on [Zulip][zulip-project-portable-simd].
+
+If you are interested in support for a specific architecture, you may want [stdarch] instead.
 
 ## Code Organization
 
@@ -20,3 +25,7 @@ The supported element types are as follows:
 
 Floating point, signed integers, and unsigned integers are the [primitive types](https://doc.rust-lang.org/core/primitive/index.html) you're already used to.
 The `mask` types are "truthy" values, but they use the number of bits in their name instead of just 1 bit like a normal `bool` uses.
+
+[simd-guide]: ./beginners-guide.md
+[zulip-project-portable-simd]: https://rust-lang.zulipchat.com/#narrow/stream/257879-project-portable-simd
+[stdarch]: https://github.com/rust-lang/stdarch

From 387063382853cc4f20900eb666c1e97dc05cb998 Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Mon, 12 Oct 2020 20:48:05 -0400
Subject: [PATCH 050/249] Add rounding mode test

---
 crates/core_simd/tests/ops_impl/float_macros.rs | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/crates/core_simd/tests/ops_impl/float_macros.rs b/crates/core_simd/tests/ops_impl/float_macros.rs
index e63b689a1755..1f49aef9f12c 100644
--- a/crates/core_simd/tests/ops_impl/float_macros.rs
+++ b/crates/core_simd/tests/ops_impl/float_macros.rs
@@ -362,6 +362,15 @@ macro_rules! float_tests {
                 }
             }
 
+            #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
+            fn round_mode() {
+                assert_biteq!(core_simd::$vector::splat(1.5).round(), core_simd::$vector::splat(2.0));
+                assert_biteq!(core_simd::$vector::splat(2.5).round(), core_simd::$vector::splat(3.0));
+                assert_biteq!(core_simd::$vector::splat(-1.5).round(), core_simd::$vector::splat(-2.0));
+                assert_biteq!(core_simd::$vector::splat(-2.5).round(), core_simd::$vector::splat(-3.0));
+            }
+
             #[test]
             #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn trunc_odd_floats() {

From a5bdb8b1ff7ec99e25d8dd195b142b1ab7058e30 Mon Sep 17 00:00:00 2001
From: Jubilee Young <workingjubilee@gmail.com>
Date: Fri, 16 Oct 2020 16:37:50 -0700
Subject: [PATCH 051/249] Document size/align of SIMD types

---
 beginners-guide.md | 11 +++++++++++
 1 file changed, 11 insertions(+)

diff --git a/beginners-guide.md b/beginners-guide.md
index a39243170fc5..998dabfed1fb 100644
--- a/beginners-guide.md
+++ b/beginners-guide.md
@@ -65,3 +65,14 @@ This is no bug in Rust, or soundness hole in the type system. You just plain can
 This is why the various Rust targets *don't* enable many CPU feature flags by default: requiring a more advanced CPU makes the final binary *less* portable.
 
 So please select an appropriate CPU feature level when building your programs.
+
+## Size, Alignment, and Unsafe Code
+
+Most of the portable SIMD API is designed to allow the user to gloss over the details of different architectures and avoid using unsafe code. However, there are plenty of reasons to want to use unsafe code with these SIMD types, such as using an intrinsic function from `core::arch` to further accelerate particularly specialized SIMD operations on a given platform, while still using the portable API elsewhere. For these cases, there are some rules to keep in mind.
+
+Fortunately, most SIMD types have a fairly predictable size. `i32x4` is bit-equivalent to `[i32; 4]` and so can be bitcast to it, e.g. using [`mem::transmute`], though the API usually offers a safe cast you can use instead.
+
+However, this is not the same as alignment. Computer architectures generally prefer aligned accesses, especially when moving data between memory and vector registers, and while some support specialized operations that can bend the rules to help with this, unaligned access is still typically slow, or even undefined behavior. In addition, different architectures can require different alignments when interacting with their native SIMD types. For this reason, any `#[repr(simd)]` type has a non-portable alignment. If it is necessary to directly interact with the alignment of these types, it should be via [`mem::align_of`].
+
+[`mem::transmute`]: https://doc.rust-lang.org/core/mem/fn.transmute.html
+[`mem::align_of`]: https://doc.rust-lang.org/core/mem/fn.align_of.html
\ No newline at end of file

From 7538ff810aa683a0c3694f20e749dee4a6469f39 Mon Sep 17 00:00:00 2001
From: Jubilee Young <workingjubilee@gmail.com>
Date: Mon, 30 Nov 2020 12:39:26 -0800
Subject: [PATCH 052/249] Revert "Disable riscv64gc"

This reverts commit 3ad356d90268e5dd3236f460422df6136e9d0e11.
---
 .github/workflows/ci.yml | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
index 8d3312576927..2104c74a4d92 100644
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -121,8 +121,7 @@ jobs:
           # for NaNs which makes it worth testing on despite that.
           - mips-unknown-linux-gnu
           - mips64-unknown-linux-gnuabi64
-          # TODO: reenable pending https://github.com/rust-lang/rust/issues/77866
-          # - riscv64gc-unknown-linux-gnu
+          - riscv64gc-unknown-linux-gnu
 
     steps:
       - uses: actions/checkout@v2

From e9cc3066a82941a15ceb60de7a55e0c163c25b2c Mon Sep 17 00:00:00 2001
From: Jubilee Young <workingjubilee@gmail.com>
Date: Mon, 30 Nov 2020 12:49:37 -0800
Subject: [PATCH 053/249] Remove round, trunc tests

There are no platform intrinsics in the rustc for these functions yet,
so this removes them as a distinct commit for later reversion.
---
 .../core_simd/tests/ops_impl/float_macros.rs  | 36 -------------------
 1 file changed, 36 deletions(-)

diff --git a/crates/core_simd/tests/ops_impl/float_macros.rs b/crates/core_simd/tests/ops_impl/float_macros.rs
index 1f49aef9f12c..fe347a5362da 100644
--- a/crates/core_simd/tests/ops_impl/float_macros.rs
+++ b/crates/core_simd/tests/ops_impl/float_macros.rs
@@ -353,42 +353,6 @@ macro_rules! float_tests {
                 }
             }
 
-            #[test]
-            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
-            fn round_odd_floats() {
-                for v in slice_chunks(&C) {
-                    let expected = apply_unary_lanewise(v, <$scalar>::round);
-                    assert_biteq!(v.round(), expected);
-                }
-            }
-
-            #[test]
-            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
-            fn round_mode() {
-                assert_biteq!(core_simd::$vector::splat(1.5).round(), core_simd::$vector::splat(2.0));
-                assert_biteq!(core_simd::$vector::splat(2.5).round(), core_simd::$vector::splat(3.0));
-                assert_biteq!(core_simd::$vector::splat(-1.5).round(), core_simd::$vector::splat(-2.0));
-                assert_biteq!(core_simd::$vector::splat(-2.5).round(), core_simd::$vector::splat(-3.0));
-            }
-
-            #[test]
-            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
-            fn trunc_odd_floats() {
-                for v in slice_chunks(&C) {
-                    let expected = apply_unary_lanewise(v, <$scalar>::trunc);
-                    assert_biteq!(v.trunc(), expected);
-                }
-            }
-
-            #[test]
-            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
-            fn fract_odd_floats() {
-                for v in slice_chunks(&C) {
-                    let expected = apply_unary_lanewise(v, <$scalar>::fract);
-                    assert_biteq!(v.fract(), expected);
-                }
-            }
-
             #[test]
             #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn to_int_unchecked() {

From 3d9bbf9b86f9f41d9ea5fe649343f3d763003866 Mon Sep 17 00:00:00 2001
From: Jubilee Young <workingjubilee@gmail.com>
Date: Mon, 30 Nov 2020 12:52:30 -0800
Subject: [PATCH 054/249] Use simd_{floor,ceil} intrinsics for round.rs

---
 crates/core_simd/src/intrinsics.rs |  6 +++
 crates/core_simd/src/round.rs      | 85 ++++--------------------------
 2 files changed, 16 insertions(+), 75 deletions(-)

diff --git a/crates/core_simd/src/intrinsics.rs b/crates/core_simd/src/intrinsics.rs
index acf85983a983..b2edc3747efc 100644
--- a/crates/core_simd/src/intrinsics.rs
+++ b/crates/core_simd/src/intrinsics.rs
@@ -39,4 +39,10 @@ extern "platform-intrinsic" {
 
     /// fptoui/fptosi/uitofp/sitofp
     pub(crate) fn simd_cast<T, U>(x: T) -> U;
+
+    // floor
+    pub(crate) fn simd_floor<T>(x: T) -> T;
+
+    // ceil
+    pub(crate) fn simd_ceil<T>(x: T) -> T;
 }
diff --git a/crates/core_simd/src/round.rs b/crates/core_simd/src/round.rs
index 021e5af88737..0529bbe0080c 100644
--- a/crates/core_simd/src/round.rs
+++ b/crates/core_simd/src/round.rs
@@ -1,60 +1,23 @@
 macro_rules! implement {
     {
         impl $type:ident {
-            int_type = $int_type:ident,
-            floor = $floor_intrinsic:literal,
-            ceil = $ceil_intrinsic:literal,
-            round = $round_intrinsic:literal,
-            trunc = $trunc_intrinsic:literal,
+            int_type = $int_type:ident
         }
     } => {
         mod $type {
-            #[allow(improper_ctypes)]
-            extern "C" {
-                #[link_name = $floor_intrinsic]
-                fn floor_intrinsic(x: crate::$type) -> crate::$type;
-                #[link_name = $ceil_intrinsic]
-                fn ceil_intrinsic(x: crate::$type) -> crate::$type;
-                #[link_name = $round_intrinsic]
-                fn round_intrinsic(x: crate::$type) -> crate::$type;
-                #[link_name = $trunc_intrinsic]
-                fn trunc_intrinsic(x: crate::$type) -> crate::$type;
-            }
-
             impl crate::$type {
                 /// Returns the largest integer less than or equal to each lane.
                 #[must_use = "method returns a new vector and does not mutate the original value"]
                 #[inline]
                 pub fn floor(self) -> Self {
-                    unsafe { floor_intrinsic(self) }
+                    unsafe { crate::intrinsics::simd_floor(self) }
                 }
 
                 /// Returns the smallest integer greater than or equal to each lane.
                 #[must_use = "method returns a new vector and does not mutate the original value"]
                 #[inline]
                 pub fn ceil(self) -> Self {
-                    unsafe { ceil_intrinsic(self) }
-                }
-
-                /// Returns the nearest integer to each lane. Round half-way cases away from 0.0.
-                #[must_use = "method returns a new vector and does not mutate the original value"]
-                #[inline]
-                pub fn round(self) -> Self {
-                    unsafe { round_intrinsic(self) }
-                }
-
-                /// Returns the integer part of each lane.
-                #[must_use = "method returns a new vector and does not mutate the original value"]
-                #[inline]
-                pub fn trunc(self) -> Self {
-                    unsafe { trunc_intrinsic(self) }
-                }
-
-                /// Returns the fractional part of each lane.
-                #[must_use = "method returns a new vector and does not mutate the original value"]
-                #[inline]
-                pub fn fract(self) -> Self {
-                    self - self.trunc()
+                    unsafe { crate::intrinsics::simd_ceil(self) }
                 }
 
                 /// Rounds toward zero and converts to the same-width integer type, assuming that
@@ -84,70 +47,42 @@ macro_rules! implement {
 
 implement! {
     impl f32x2 {
-        int_type = i32x2,
-        floor = "llvm.floor.v2f32",
-        ceil = "llvm.ceil.v2f32",
-        round = "llvm.round.v2f32",
-        trunc = "llvm.trunc.v2f32",
+        int_type = i32x2
     }
 }
 
 implement! {
     impl f32x4 {
-        int_type = i32x4,
-        floor = "llvm.floor.v4f32",
-        ceil = "llvm.ceil.v4f32",
-        round = "llvm.round.v4f32",
-        trunc = "llvm.trunc.v4f32",
+        int_type = i32x4
     }
 }
 
 implement! {
     impl f32x8 {
-        int_type = i32x8,
-        floor = "llvm.floor.v8f32",
-        ceil = "llvm.ceil.v8f32",
-        round = "llvm.round.v8f32",
-        trunc = "llvm.trunc.v8f32",
+        int_type = i32x8
     }
 }
 
 implement! {
     impl f32x16 {
-        int_type = i32x16,
-        floor = "llvm.floor.v16f32",
-        ceil = "llvm.ceil.v16f32",
-        round = "llvm.round.v16f32",
-        trunc = "llvm.trunc.v16f32",
+        int_type = i32x16
     }
 }
 
 implement! {
     impl f64x2 {
-        int_type = i64x2,
-        floor = "llvm.floor.v2f64",
-        ceil = "llvm.ceil.v2f64",
-        round = "llvm.round.v2f64",
-        trunc = "llvm.trunc.v2f64",
+        int_type = i64x2
     }
 }
 
 implement! {
     impl f64x4 {
-        int_type = i64x4,
-        floor = "llvm.floor.v4f64",
-        ceil = "llvm.ceil.v4f64",
-        round = "llvm.round.v4f64",
-        trunc = "llvm.trunc.v4f64",
+        int_type = i64x4
     }
 }
 
 implement! {
     impl f64x8 {
-        int_type = i64x8,
-        floor = "llvm.floor.v8f64",
-        ceil = "llvm.ceil.v8f64",
-        round = "llvm.round.v8f64",
-        trunc = "llvm.trunc.v8f64",
+        int_type = i64x8
     }
 }

From cebc2ca7072b8b33a397adf4f3380099ea25fb19 Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Tue, 13 Oct 2020 01:28:03 -0400
Subject: [PATCH 055/249] Add opaque masks

---
 crates/core_simd/src/fmt.rs                   |  20 +-
 crates/core_simd/src/lib.rs                   |  16 +-
 crates/core_simd/src/macros.rs                |  11 +-
 crates/core_simd/src/masks/mod.rs             | 215 ++++++++++++++++++
 .../src/{masks.rs => masks/wide/mod.rs}       |  27 ++-
 .../core_simd/src/masks/wide/vectors_m128.rs  |  11 +
 .../core_simd/src/masks/wide/vectors_m16.rs   |  21 ++
 .../core_simd/src/masks/wide/vectors_m32.rs   |  21 ++
 .../core_simd/src/masks/wide/vectors_m64.rs   |  16 ++
 crates/core_simd/src/masks/wide/vectors_m8.rs |  21 ++
 .../core_simd/src/masks/wide/vectors_msize.rs |  16 ++
 crates/core_simd/src/ops.rs                   |  24 +-
 crates/core_simd/src/vectors_mask128.rs       |  11 -
 crates/core_simd/src/vectors_mask16.rs        |  21 --
 crates/core_simd/src/vectors_mask32.rs        |  21 --
 crates/core_simd/src/vectors_mask64.rs        |  16 --
 crates/core_simd/src/vectors_mask8.rs         |  21 --
 crates/core_simd/src/vectors_masksize.rs      |  16 --
 18 files changed, 379 insertions(+), 146 deletions(-)
 create mode 100644 crates/core_simd/src/masks/mod.rs
 rename crates/core_simd/src/{masks.rs => masks/wide/mod.rs} (87%)
 create mode 100644 crates/core_simd/src/masks/wide/vectors_m128.rs
 create mode 100644 crates/core_simd/src/masks/wide/vectors_m16.rs
 create mode 100644 crates/core_simd/src/masks/wide/vectors_m32.rs
 create mode 100644 crates/core_simd/src/masks/wide/vectors_m64.rs
 create mode 100644 crates/core_simd/src/masks/wide/vectors_m8.rs
 create mode 100644 crates/core_simd/src/masks/wide/vectors_msize.rs
 delete mode 100644 crates/core_simd/src/vectors_mask128.rs
 delete mode 100644 crates/core_simd/src/vectors_mask16.rs
 delete mode 100644 crates/core_simd/src/vectors_mask32.rs
 delete mode 100644 crates/core_simd/src/vectors_mask64.rs
 delete mode 100644 crates/core_simd/src/vectors_mask8.rs
 delete mode 100644 crates/core_simd/src/vectors_masksize.rs

diff --git a/crates/core_simd/src/fmt.rs b/crates/core_simd/src/fmt.rs
index c634e0546bc1..07332c1ccc8f 100644
--- a/crates/core_simd/src/fmt.rs
+++ b/crates/core_simd/src/fmt.rs
@@ -74,10 +74,10 @@ macro_rules! impl_fmt_trait {
 
 impl_fmt_trait! {
     integers:
-        crate::u8x8,    crate::u8x16,    crate::u8x32,    crate::u8x64,
-        crate::i8x8,    crate::i8x16,    crate::i8x32,    crate::i8x64,
-        crate::u16x4,   crate::u16x8,   crate::u16x16,   crate::u16x32,
-        crate::i16x4,   crate::i16x8,   crate::i16x16,   crate::i16x32,
+        crate::u8x8,    crate::u8x16,   crate::u8x32,   crate::u8x64,
+        crate::i8x8,    crate::i8x16,   crate::i8x32,   crate::i8x64,
+        crate::u16x4,   crate::u16x8,   crate::u16x16,  crate::u16x32,
+        crate::i16x4,   crate::i16x8,   crate::i16x16,  crate::i16x32,
         crate::u32x2,   crate::u32x4,   crate::u32x8,   crate::u32x16,
         crate::i32x2,   crate::i32x4,   crate::i32x8,   crate::i32x16,
         crate::u64x2,   crate::u64x4,   crate::u64x8,
@@ -96,10 +96,10 @@ impl_fmt_trait! {
 
 impl_fmt_trait! {
     masks:
-        crate::mask8x8,    crate::mask8x16,    crate::mask8x32,    crate::mask8x64,
-        crate::mask16x4,   crate::mask16x8,   crate::mask16x16,   crate::mask16x32,
-        crate::mask32x2,   crate::mask32x4,   crate::mask32x8,   crate::mask32x16,
-        crate::mask64x2,   crate::mask64x4,   crate::mask64x8,
-        crate::mask128x2,  crate::mask128x4,
-        crate::masksizex2, crate::masksizex4, crate::masksizex8,
+        crate::masks::wide::m8x8,    crate::masks::wide::m8x16,   crate::masks::wide::m8x32,   crate::masks::wide::m8x64,
+        crate::masks::wide::m16x4,   crate::masks::wide::m16x8,   crate::masks::wide::m16x16,  crate::masks::wide::m16x32,
+        crate::masks::wide::m32x2,   crate::masks::wide::m32x4,   crate::masks::wide::m32x8,   crate::masks::wide::m32x16,
+        crate::masks::wide::m64x2,   crate::masks::wide::m64x4,   crate::masks::wide::m64x8,
+        crate::masks::wide::m128x2,  crate::masks::wide::m128x4,
+        crate::masks::wide::msizex2, crate::masks::wide::msizex4, crate::masks::wide::msizex8,
 }
diff --git a/crates/core_simd/src/lib.rs b/crates/core_simd/src/lib.rs
index cea39e6f3f3d..fd4f9dd16fd5 100644
--- a/crates/core_simd/src/lib.rs
+++ b/crates/core_simd/src/lib.rs
@@ -10,8 +10,7 @@ mod fmt;
 mod intrinsics;
 mod ops;
 
-mod masks;
-pub use masks::*;
+pub mod masks;
 
 mod vectors_u8;
 pub use vectors_u8::*;
@@ -44,17 +43,4 @@ pub use vectors_f32::*;
 mod vectors_f64;
 pub use vectors_f64::*;
 
-mod vectors_mask8;
-pub use vectors_mask8::*;
-mod vectors_mask16;
-pub use vectors_mask16::*;
-mod vectors_mask32;
-pub use vectors_mask32::*;
-mod vectors_mask64;
-pub use vectors_mask64::*;
-mod vectors_mask128;
-pub use vectors_mask128::*;
-mod vectors_masksize;
-pub use vectors_masksize::*;
-
 mod round;
diff --git a/crates/core_simd/src/macros.rs b/crates/core_simd/src/macros.rs
index f37d13c3ca32..b8324ffdb929 100644
--- a/crates/core_simd/src/macros.rs
+++ b/crates/core_simd/src/macros.rs
@@ -314,7 +314,6 @@ macro_rules! define_float_vector {
     }
 }
 
-
 /// Defines an integer vector `$name` containing multiple `$lanes` of integer `$type`.
 macro_rules! define_integer_vector {
     { $(#[$attr:meta])* struct $name:ident([$type:ty; $lanes:tt]); } => {
@@ -336,6 +335,7 @@ macro_rules! define_mask_vector {
         impl $name {
             call_repeat! { $lanes => define_mask_vector [$impl_type] splat $type | }
             call_counting_args! { $lanes => define_mask_vector => new $type | }
+            call_counting_args! { $lanes => define_mask_vector => new_from_bool $type | }
         }
 
         base_vector_traits! { $name => [$type; $lanes] }
@@ -361,5 +361,14 @@ macro_rules! define_mask_vector {
         pub const fn new($($var: $type),*) -> Self {
             Self($($var.0),*)
         }
+    };
+    { new_from_bool $type:ty | $($var:ident)* } => {
+        /// Used internally (since we can't use the Into trait in `const fn`s)
+        #[allow(clippy::too_many_arguments)]
+        #[allow(unused)]
+        #[inline]
+        pub(crate) const fn new_from_bool($($var: bool),*) -> Self {
+            Self($(<$type>::new($var).0),*)
+        }
     }
 }
diff --git a/crates/core_simd/src/masks/mod.rs b/crates/core_simd/src/masks/mod.rs
new file mode 100644
index 000000000000..e138a1b4dd80
--- /dev/null
+++ b/crates/core_simd/src/masks/mod.rs
@@ -0,0 +1,215 @@
+//! Types and traits associated with masking lanes of vectors.
+
+pub mod wide;
+
+trait MaskImpl {
+    type Mask;
+}
+
+impl MaskImpl for [u8; 8] {
+    type Mask = wide::m8x8;
+}
+
+impl MaskImpl for [u8; 16] {
+    type Mask = wide::m8x16;
+}
+
+impl MaskImpl for [u8; 32] {
+    type Mask = wide::m8x32;
+}
+
+impl MaskImpl for [u8; 64] {
+    type Mask = wide::m8x64;
+}
+
+impl MaskImpl for [u16; 4] {
+    type Mask = wide::m16x4;
+}
+
+impl MaskImpl for [u16; 8] {
+    type Mask = wide::m16x8;
+}
+
+impl MaskImpl for [u16; 16] {
+    type Mask = wide::m16x16;
+}
+
+impl MaskImpl for [u16; 32] {
+    type Mask = wide::m16x32;
+}
+
+impl MaskImpl for [u32; 2] {
+    type Mask = wide::m32x2;
+}
+
+impl MaskImpl for [u32; 4] {
+    type Mask = wide::m32x4;
+}
+
+impl MaskImpl for [u32; 8] {
+    type Mask = wide::m32x8;
+}
+
+impl MaskImpl for [u32; 16] {
+    type Mask = wide::m32x16;
+}
+
+impl MaskImpl for [u64; 2] {
+    type Mask = wide::m64x2;
+}
+
+impl MaskImpl for [u64; 4] {
+    type Mask = wide::m64x4;
+}
+
+impl MaskImpl for [u64; 8] {
+    type Mask = wide::m64x8;
+}
+
+impl MaskImpl for [u128; 2] {
+    type Mask = wide::m128x2;
+}
+
+impl MaskImpl for [u128; 4] {
+    type Mask = wide::m128x4;
+}
+
+impl MaskImpl for [usize; 2] {
+    type Mask = wide::msizex2;
+}
+
+impl MaskImpl for [usize; 4] {
+    type Mask = wide::msizex4;
+}
+
+impl MaskImpl for [usize; 8] {
+    type Mask = wide::msizex8;
+}
+
+macro_rules! define_opaque_mask {
+    {
+        $(#[$attr:meta])*
+        struct $name:ident([$width:ty; $lanes:tt]);
+    } => {
+        $(#[$attr])*
+        #[allow(non_camel_case_types)]
+        pub struct $name(<[$width; $lanes] as MaskImpl>::Mask);
+
+        impl $name {
+            /// Construct a mask by setting all lanes to the given value.
+            pub fn splat(value: bool) -> Self {
+                Self(<[$width; $lanes] as MaskImpl>::Mask::splat(value.into()))
+            }
+
+            call_counting_args! { $lanes => define_opaque_mask => new [$width; $lanes] }
+        }
+    };
+    { new [$width:ty; $lanes:tt] $($var:ident)* } => {
+        /// Construct a vector by setting each lane to the given values.
+        #[allow(clippy::too_many_arguments)]
+        #[inline]
+        pub const fn new($($var: bool),*) -> Self {
+            Self(<[$width; $lanes] as MaskImpl>::Mask::new_from_bool($($var),*))
+        }
+    }
+}
+
+define_opaque_mask! {
+    /// Mask for 8 8-bit lanes
+    struct mask8x8([u8; 8]);
+}
+
+define_opaque_mask! {
+    /// Mask for 16 8-bit lanes
+    struct mask8x16([u8; 16]);
+}
+
+define_opaque_mask! {
+    /// Mask for 32 8-bit lanes
+    struct mask8x32([u8; 32]);
+}
+
+define_opaque_mask! {
+    /// Mask for 64 8-bit lanes
+    struct mask8x64([u8; 64]);
+}
+
+define_opaque_mask! {
+    /// Mask for 4 16-bit lanes
+    struct mask16x4([u16; 4]);
+}
+
+define_opaque_mask! {
+    /// Mask for 8 16-bit lanes
+    struct mask16x8([u16; 8]);
+}
+
+define_opaque_mask! {
+    /// Mask for 16 16-bit lanes
+    struct mask16x16([u16; 16]);
+}
+
+define_opaque_mask! {
+    /// Mask for 32 16-bit lanes
+    struct mask16x32([u16; 32]);
+}
+
+define_opaque_mask! {
+    /// Mask for 2 32-bit lanes
+    struct mask32x2([u32; 2]);
+}
+
+define_opaque_mask! {
+    /// Mask for 4 32-bit lanes
+    struct mask32x4([u32; 4]);
+}
+
+define_opaque_mask! {
+    /// Mask for 8 32-bit lanes
+    struct mask32x8([u32; 8]);
+}
+
+define_opaque_mask! {
+    /// Mask for 16 32-bit lanes
+    struct mask32x16([u32; 16]);
+}
+
+define_opaque_mask! {
+    /// Mask for 2 64-bit lanes
+    struct mask64x2([u64; 2]);
+}
+
+define_opaque_mask! {
+    /// Mask for 4 64-bit lanes
+    struct mask64x4([u64; 4]);
+}
+
+define_opaque_mask! {
+    /// Mask for 8 64-bit lanes
+    struct mask64x8([u64; 8]);
+}
+
+define_opaque_mask! {
+    /// Mask for 2 128-bit lanes
+    struct mask128x2([u128; 2]);
+}
+
+define_opaque_mask! {
+    /// Mask for 4 128-bit lanes
+    struct mask128x4([u128; 4]);
+}
+
+define_opaque_mask! {
+    /// Mask for 2 `isize`-wide lanes
+    struct masksizex2([usize; 2]);
+}
+
+define_opaque_mask! {
+    /// Mask for 4 `isize`-wide lanes
+    struct masksizex4([usize; 4]);
+}
+
+define_opaque_mask! {
+    /// Mask for 8 `isize`-wide lanes
+    struct masksizex8([usize; 8]);
+}
diff --git a/crates/core_simd/src/masks.rs b/crates/core_simd/src/masks/wide/mod.rs
similarity index 87%
rename from crates/core_simd/src/masks.rs
rename to crates/core_simd/src/masks/wide/mod.rs
index cba76b6a2a35..1462992197dc 100644
--- a/crates/core_simd/src/masks.rs
+++ b/crates/core_simd/src/masks/wide/mod.rs
@@ -1,3 +1,18 @@
+//! Masks that take up full vector registers.
+
+mod vectors_m8;
+pub use vectors_m8::*;
+mod vectors_m16;
+pub use vectors_m16::*;
+mod vectors_m32;
+pub use vectors_m32::*;
+mod vectors_m64;
+pub use vectors_m64::*;
+mod vectors_m128;
+pub use vectors_m128::*;
+mod vectors_msize;
+pub use vectors_msize::*;
+
 /// The error type returned when converting an integer to a mask fails.
 #[derive(Debug, Copy, Clone, PartialEq, Eq)]
 pub struct TryFromMaskError(());
@@ -95,30 +110,30 @@ macro_rules! define_mask {
 
 define_mask! {
     /// 8-bit mask
-    struct mask8(i8);
+    struct m8(i8);
 }
 
 define_mask! {
     /// 16-bit mask
-    struct mask16(i16);
+    struct m16(i16);
 }
 
 define_mask! {
     /// 32-bit mask
-    struct mask32(i32);
+    struct m32(i32);
 }
 
 define_mask! {
     /// 64-bit mask
-    struct mask64(i64);
+    struct m64(i64);
 }
 
 define_mask! {
     /// 128-bit mask
-    struct mask128(i128);
+    struct m128(i128);
 }
 
 define_mask! {
     /// `isize`-wide mask
-    struct masksize(isize);
+    struct msize(isize);
 }
diff --git a/crates/core_simd/src/masks/wide/vectors_m128.rs b/crates/core_simd/src/masks/wide/vectors_m128.rs
new file mode 100644
index 000000000000..fddddac5fc4f
--- /dev/null
+++ b/crates/core_simd/src/masks/wide/vectors_m128.rs
@@ -0,0 +1,11 @@
+use super::m128;
+
+define_mask_vector! {
+    /// Vector of two `m128` values
+    struct m128x2([i128 as m128; 2]);
+}
+
+define_mask_vector! {
+    /// Vector of four `m128` values
+    struct m128x4([i128 as m128; 4]);
+}
diff --git a/crates/core_simd/src/masks/wide/vectors_m16.rs b/crates/core_simd/src/masks/wide/vectors_m16.rs
new file mode 100644
index 000000000000..3b05e83f673d
--- /dev/null
+++ b/crates/core_simd/src/masks/wide/vectors_m16.rs
@@ -0,0 +1,21 @@
+use super::m16;
+
+define_mask_vector! {
+    /// Vector of four `m16` values
+    struct m16x4([i16 as m16; 4]);
+}
+
+define_mask_vector! {
+    /// Vector of eight `m16` values
+    struct m16x8([i16 as m16; 8]);
+}
+
+define_mask_vector! {
+    /// Vector of 16 `m16` values
+    struct m16x16([i16 as m16; 16]);
+}
+
+define_mask_vector! {
+    /// Vector of 32 `m16` values
+    struct m16x32([i16 as m16; 32]);
+}
diff --git a/crates/core_simd/src/masks/wide/vectors_m32.rs b/crates/core_simd/src/masks/wide/vectors_m32.rs
new file mode 100644
index 000000000000..de5745fb2833
--- /dev/null
+++ b/crates/core_simd/src/masks/wide/vectors_m32.rs
@@ -0,0 +1,21 @@
+use super::m32;
+
+define_mask_vector! {
+    /// Vector of two `m32` values
+    struct m32x2([i32 as m32; 2]);
+}
+
+define_mask_vector! {
+    /// Vector of four `m32` values
+    struct m32x4([i32 as m32; 4]);
+}
+
+define_mask_vector! {
+    /// Vector of eight `m32` values
+    struct m32x8([i32 as m32; 8]);
+}
+
+define_mask_vector! {
+    /// Vector of 16 `m32` values
+    struct m32x16([i32 as m32; 16]);
+}
diff --git a/crates/core_simd/src/masks/wide/vectors_m64.rs b/crates/core_simd/src/masks/wide/vectors_m64.rs
new file mode 100644
index 000000000000..55c8687fcfc4
--- /dev/null
+++ b/crates/core_simd/src/masks/wide/vectors_m64.rs
@@ -0,0 +1,16 @@
+use super::m64;
+
+define_mask_vector! {
+    /// Vector of two `m64` values
+    struct m64x2([i64 as m64; 2]);
+}
+
+define_mask_vector! {
+    /// Vector of four `m64` values
+    struct m64x4([i64 as m64; 4]);
+}
+
+define_mask_vector! {
+    /// Vector of eight `m64` values
+    struct m64x8([i64 as m64; 8]);
+}
diff --git a/crates/core_simd/src/masks/wide/vectors_m8.rs b/crates/core_simd/src/masks/wide/vectors_m8.rs
new file mode 100644
index 000000000000..149e138739dc
--- /dev/null
+++ b/crates/core_simd/src/masks/wide/vectors_m8.rs
@@ -0,0 +1,21 @@
+use super::m8;
+
+define_mask_vector! {
+    /// Vector of eight `m8` values
+    struct m8x8([i8 as m8; 8]);
+}
+
+define_mask_vector! {
+    /// Vector of 16 `m8` values
+    struct m8x16([i8 as m8; 16]);
+}
+
+define_mask_vector! {
+    /// Vector of 32 `m8` values
+    struct m8x32([i8 as m8; 32]);
+}
+
+define_mask_vector! {
+    /// Vector of 64 `m8` values
+    struct m8x64([i8 as m8; 64]);
+}
diff --git a/crates/core_simd/src/masks/wide/vectors_msize.rs b/crates/core_simd/src/masks/wide/vectors_msize.rs
new file mode 100644
index 000000000000..497aba8ddbbf
--- /dev/null
+++ b/crates/core_simd/src/masks/wide/vectors_msize.rs
@@ -0,0 +1,16 @@
+use super::msize;
+
+define_mask_vector! {
+    /// Vector of two `msize` values
+    struct msizex2([isize as msize; 2]);
+}
+
+define_mask_vector! {
+    /// Vector of four `msize` values
+    struct msizex4([isize as msize; 4]);
+}
+
+define_mask_vector! {
+    /// Vector of eight `msize` values
+    struct msizex8([isize as msize; 8]);
+}
diff --git a/crates/core_simd/src/ops.rs b/crates/core_simd/src/ops.rs
index 5a186649821b..ac89feca9d66 100644
--- a/crates/core_simd/src/ops.rs
+++ b/crates/core_simd/src/ops.rs
@@ -96,7 +96,7 @@ macro_rules! impl_ref_ops {
 }
 
 /// Implements op traits for masks
-macro_rules! impl_mask_ops {
+macro_rules! impl_mask_element_ops {
     { $($mask:ty),* } => {
         $(
             impl_ref_ops! {
@@ -161,7 +161,15 @@ macro_rules! impl_mask_ops {
         )*
     }
 }
-impl_mask_ops! { crate::mask8, crate::mask16, crate::mask32, crate::mask64, crate::mask128, crate::masksize }
+
+impl_mask_element_ops! {
+    crate::masks::wide::m8,
+    crate::masks::wide::m16,
+    crate::masks::wide::m32,
+    crate::masks::wide::m64,
+    crate::masks::wide::m128,
+    crate::masks::wide::msize
+}
 
 /// Automatically implements operators over vectors and scalars for a particular vector.
 macro_rules! impl_op {
@@ -632,10 +640,10 @@ impl_float_ops! {
 }
 
 impl_mask_ops! {
-    crate::mask8    => crate::mask8x8,    crate::mask8x16,   crate::mask8x32,   crate::mask8x64;
-    crate::mask16   => crate::mask16x4,   crate::mask16x8,   crate::mask16x16,  crate::mask16x32;
-    crate::mask32   => crate::mask32x2,   crate::mask32x4,   crate::mask32x8,   crate::mask32x16;
-    crate::mask64   => crate::mask64x2,   crate::mask64x4,   crate::mask64x8;
-    crate::mask128  => crate::mask128x2,  crate::mask128x4;
-    crate::masksize => crate::masksizex2, crate::masksizex4, crate::masksizex8;
+    crate::masks::wide::m8    => crate::masks::wide::m8x8,    crate::masks::wide::m8x16,   crate::masks::wide::m8x32,   crate::masks::wide::m8x64;
+    crate::masks::wide::m16   => crate::masks::wide::m16x4,   crate::masks::wide::m16x8,   crate::masks::wide::m16x16,  crate::masks::wide::m16x32;
+    crate::masks::wide::m32   => crate::masks::wide::m32x2,   crate::masks::wide::m32x4,   crate::masks::wide::m32x8,   crate::masks::wide::m32x16;
+    crate::masks::wide::m64   => crate::masks::wide::m64x2,   crate::masks::wide::m64x4,   crate::masks::wide::m64x8;
+    crate::masks::wide::m128  => crate::masks::wide::m128x2,  crate::masks::wide::m128x4;
+    crate::masks::wide::msize => crate::masks::wide::msizex2, crate::masks::wide::msizex4, crate::masks::wide::msizex8;
 }
diff --git a/crates/core_simd/src/vectors_mask128.rs b/crates/core_simd/src/vectors_mask128.rs
deleted file mode 100644
index adf56a3684b3..000000000000
--- a/crates/core_simd/src/vectors_mask128.rs
+++ /dev/null
@@ -1,11 +0,0 @@
-use crate::mask128;
-
-define_mask_vector! {
-    /// Vector of two `mask128` values
-    struct mask128x2([i128 as mask128; 2]);
-}
-
-define_mask_vector! {
-    /// Vector of four `mask128` values
-    struct mask128x4([i128 as mask128; 4]);
-}
diff --git a/crates/core_simd/src/vectors_mask16.rs b/crates/core_simd/src/vectors_mask16.rs
deleted file mode 100644
index 406d7255a11e..000000000000
--- a/crates/core_simd/src/vectors_mask16.rs
+++ /dev/null
@@ -1,21 +0,0 @@
-use crate::mask16;
-
-define_mask_vector! {
-    /// Vector of four `mask16` values
-    struct mask16x4([i16 as mask16; 4]);
-}
-
-define_mask_vector! {
-    /// Vector of eight `mask16` values
-    struct mask16x8([i16 as mask16; 8]);
-}
-
-define_mask_vector! {
-    /// Vector of 16 `mask16` values
-    struct mask16x16([i16 as mask16; 16]);
-}
-
-define_mask_vector! {
-    /// Vector of 32 `mask16` values
-    struct mask16x32([i16 as mask16; 32]);
-}
diff --git a/crates/core_simd/src/vectors_mask32.rs b/crates/core_simd/src/vectors_mask32.rs
deleted file mode 100644
index fad191421f38..000000000000
--- a/crates/core_simd/src/vectors_mask32.rs
+++ /dev/null
@@ -1,21 +0,0 @@
-use crate::mask32;
-
-define_mask_vector! {
-    /// Vector of two `mask32` values
-    struct mask32x2([i32 as mask32; 2]);
-}
-
-define_mask_vector! {
-    /// Vector of four `mask32` values
-    struct mask32x4([i32 as mask32; 4]);
-}
-
-define_mask_vector! {
-    /// Vector of eight `mask32` values
-    struct mask32x8([i32 as mask32; 8]);
-}
-
-define_mask_vector! {
-    /// Vector of 16 `mask32` values
-    struct mask32x16([i32 as mask32; 16]);
-}
diff --git a/crates/core_simd/src/vectors_mask64.rs b/crates/core_simd/src/vectors_mask64.rs
deleted file mode 100644
index 554e731ccf24..000000000000
--- a/crates/core_simd/src/vectors_mask64.rs
+++ /dev/null
@@ -1,16 +0,0 @@
-use crate::mask64;
-
-define_mask_vector! {
-    /// Vector of two `mask64` values
-    struct mask64x2([i64 as mask64; 2]);
-}
-
-define_mask_vector! {
-    /// Vector of four `mask64` values
-    struct mask64x4([i64 as mask64; 4]);
-}
-
-define_mask_vector! {
-    /// Vector of eight `mask64` values
-    struct mask64x8([i64 as mask64; 8]);
-}
diff --git a/crates/core_simd/src/vectors_mask8.rs b/crates/core_simd/src/vectors_mask8.rs
deleted file mode 100644
index d038b3361044..000000000000
--- a/crates/core_simd/src/vectors_mask8.rs
+++ /dev/null
@@ -1,21 +0,0 @@
-use crate::mask8;
-
-define_mask_vector! {
-    /// Vector of eight `mask8` values
-    struct mask8x8([i8 as mask8; 8]);
-}
-
-define_mask_vector! {
-    /// Vector of 16 `mask8` values
-    struct mask8x16([i8 as mask8; 16]);
-}
-
-define_mask_vector! {
-    /// Vector of 32 `mask8` values
-    struct mask8x32([i8 as mask8; 32]);
-}
-
-define_mask_vector! {
-    /// Vector of 64 `mask8` values
-    struct mask8x64([i8 as mask8; 64]);
-}
diff --git a/crates/core_simd/src/vectors_masksize.rs b/crates/core_simd/src/vectors_masksize.rs
deleted file mode 100644
index a838aee51985..000000000000
--- a/crates/core_simd/src/vectors_masksize.rs
+++ /dev/null
@@ -1,16 +0,0 @@
-use crate::masksize;
-
-define_mask_vector! {
-    /// Vector of two `masksize` values
-    struct masksizex2([isize as masksize; 2]);
-}
-
-define_mask_vector! {
-    /// Vector of four `masksize` values
-    struct masksizex4([isize as masksize; 4]);
-}
-
-define_mask_vector! {
-    /// Vector of eight `masksize` values
-    struct masksizex8([isize as masksize; 8]);
-}

From 5bc5d7f0d12bc8d6e0175177e14b38e8c7c3d240 Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Wed, 28 Oct 2020 16:27:15 -0400
Subject: [PATCH 056/249] Add comparison ops

---
 crates/core_simd/src/intrinsics.rs |   7 +
 crates/core_simd/src/lib.rs        |   1 +
 crates/core_simd/src/macros.rs     |  18 ++
 crates/core_simd/src/masks/mod.rs  | 263 ++++++++++++++++++++---------
 crates/core_simd/src/masks/ops.rs  | 208 +++++++++++++++++++++++
 5 files changed, 414 insertions(+), 83 deletions(-)
 create mode 100644 crates/core_simd/src/masks/ops.rs

diff --git a/crates/core_simd/src/intrinsics.rs b/crates/core_simd/src/intrinsics.rs
index b2edc3747efc..3dfc77136f2d 100644
--- a/crates/core_simd/src/intrinsics.rs
+++ b/crates/core_simd/src/intrinsics.rs
@@ -45,4 +45,11 @@ extern "platform-intrinsic" {
 
     // ceil
     pub(crate) fn simd_ceil<T>(x: T) -> T;
+
+    pub(crate) fn simd_eq<T, U>(x: T, y: T) -> U;
+    pub(crate) fn simd_ne<T, U>(x: T, y: T) -> U;
+    pub(crate) fn simd_lt<T, U>(x: T, y: T) -> U;
+    pub(crate) fn simd_le<T, U>(x: T, y: T) -> U;
+    pub(crate) fn simd_gt<T, U>(x: T, y: T) -> U;
+    pub(crate) fn simd_ge<T, U>(x: T, y: T) -> U;
 }
diff --git a/crates/core_simd/src/lib.rs b/crates/core_simd/src/lib.rs
index fd4f9dd16fd5..3c581ad659be 100644
--- a/crates/core_simd/src/lib.rs
+++ b/crates/core_simd/src/lib.rs
@@ -11,6 +11,7 @@ mod intrinsics;
 mod ops;
 
 pub mod masks;
+pub use masks::opaque::*;
 
 mod vectors_u8;
 pub use vectors_u8::*;
diff --git a/crates/core_simd/src/macros.rs b/crates/core_simd/src/macros.rs
index b8324ffdb929..75584f58b784 100644
--- a/crates/core_simd/src/macros.rs
+++ b/crates/core_simd/src/macros.rs
@@ -336,6 +336,24 @@ macro_rules! define_mask_vector {
             call_repeat! { $lanes => define_mask_vector [$impl_type] splat $type | }
             call_counting_args! { $lanes => define_mask_vector => new $type | }
             call_counting_args! { $lanes => define_mask_vector => new_from_bool $type | }
+
+            /// Tests the value of the specified lane.
+            ///
+            /// # Panics
+            /// Panics if `lane` is greater than or equal to the number of lanes in the vector.
+            #[inline]
+            pub fn test(&self, lane: usize) -> bool {
+                self[lane].test()
+            }
+
+            /// Sets the value of the specified lane.
+            ///
+            /// # Panics
+            /// Panics if `lane` is greater than or equal to the number of lanes in the vector.
+            #[inline]
+            pub fn set(&mut self, lane: usize, value: bool) {
+                self[lane] = value.into();
+            }
         }
 
         base_vector_traits! { $name => [$type; $lanes] }
diff --git a/crates/core_simd/src/masks/mod.rs b/crates/core_simd/src/masks/mod.rs
index e138a1b4dd80..6688db290e25 100644
--- a/crates/core_simd/src/masks/mod.rs
+++ b/crates/core_simd/src/masks/mod.rs
@@ -2,7 +2,10 @@
 
 pub mod wide;
 
-trait MaskImpl {
+mod ops;
+pub use ops::*;
+
+pub(crate) trait MaskImpl {
     type Mask;
 }
 
@@ -93,15 +96,67 @@ macro_rules! define_opaque_mask {
     } => {
         $(#[$attr])*
         #[allow(non_camel_case_types)]
-        pub struct $name(<[$width; $lanes] as MaskImpl>::Mask);
+        pub struct $name(<[$width; $lanes] as crate::masks::MaskImpl>::Mask);
 
         impl $name {
+            pub(crate) fn new_from_inner(inner: <[$width; $lanes] as crate::masks::MaskImpl>::Mask) -> Self {
+                Self(inner)
+            }
+
             /// Construct a mask by setting all lanes to the given value.
             pub fn splat(value: bool) -> Self {
-                Self(<[$width; $lanes] as MaskImpl>::Mask::splat(value.into()))
+                Self(<[$width; $lanes] as crate::masks::MaskImpl>::Mask::splat(value.into()))
             }
 
             call_counting_args! { $lanes => define_opaque_mask => new [$width; $lanes] }
+
+            /// Tests the value of the specified lane.
+            ///
+            /// # Panics
+            /// Panics if `lane` is greater than or equal to the number of lanes in the vector.
+            #[inline]
+            pub fn test(&self, lane: usize) -> bool {
+                self.0.test(lane)
+            }
+
+            /// Sets the value of the specified lane.
+            ///
+            /// # Panics
+            /// Panics if `lane` is greater than or equal to the number of lanes in the vector.
+            #[inline]
+            pub fn set(&mut self, lane: usize, value: bool) {
+                self.0.set(lane, value);
+            }
+        }
+
+        impl Copy for $name {}
+
+        impl Clone for $name {
+            #[inline]
+            fn clone(&self) -> Self {
+                *self
+            }
+        }
+
+        impl Default for $name {
+            #[inline]
+            fn default() -> Self {
+                Self::splat(false)
+            }
+        }
+
+        impl PartialEq for $name {
+            #[inline]
+            fn eq(&self, other: &Self) -> bool {
+                self.0 == other.0
+            }
+        }
+
+        impl PartialOrd for $name {
+            #[inline]
+            fn partial_cmp(&self, other: &Self) -> Option<core::cmp::Ordering> {
+                self.0.partial_cmp(&other.0)
+            }
         }
     };
     { new [$width:ty; $lanes:tt] $($var:ident)* } => {
@@ -109,107 +164,149 @@ macro_rules! define_opaque_mask {
         #[allow(clippy::too_many_arguments)]
         #[inline]
         pub const fn new($($var: bool),*) -> Self {
-            Self(<[$width; $lanes] as MaskImpl>::Mask::new_from_bool($($var),*))
+            Self(<[$width; $lanes] as crate::masks::MaskImpl>::Mask::new_from_bool($($var),*))
         }
     }
 }
 
-define_opaque_mask! {
-    /// Mask for 8 8-bit lanes
-    struct mask8x8([u8; 8]);
-}
+pub(crate) mod opaque {
+    define_opaque_mask! {
+        /// Mask for 8 8-bit lanes.
+        ///
+        /// The layout of this type is unspecified.
+        struct mask8x8([u8; 8]);
+    }
 
-define_opaque_mask! {
-    /// Mask for 16 8-bit lanes
-    struct mask8x16([u8; 16]);
-}
+    define_opaque_mask! {
+        /// Mask for 16 8-bit lanes.
+        ///
+        /// The layout of this type is unspecified.
+        struct mask8x16([u8; 16]);
+    }
 
-define_opaque_mask! {
-    /// Mask for 32 8-bit lanes
-    struct mask8x32([u8; 32]);
-}
+    define_opaque_mask! {
+        /// Mask for 32 8-bit lanes.
+        ///
+        /// The layout of this type is unspecified.
+        struct mask8x32([u8; 32]);
+    }
 
-define_opaque_mask! {
-    /// Mask for 64 8-bit lanes
-    struct mask8x64([u8; 64]);
-}
+    define_opaque_mask! {
+        /// Mask for 64 8-bit lanes.
+        ///
+        /// The layout of this type is unspecified.
+        struct mask8x64([u8; 64]);
+    }
 
-define_opaque_mask! {
-    /// Mask for 4 16-bit lanes
-    struct mask16x4([u16; 4]);
-}
+    define_opaque_mask! {
+        /// Mask for 4 16-bit lanes.
+        ///
+        /// The layout of this type is unspecified.
+        struct mask16x4([u16; 4]);
+    }
 
-define_opaque_mask! {
-    /// Mask for 8 16-bit lanes
-    struct mask16x8([u16; 8]);
-}
+    define_opaque_mask! {
+        /// Mask for 8 16-bit lanes.
+        ///
+        /// The layout of this type is unspecified.
+        struct mask16x8([u16; 8]);
+    }
 
-define_opaque_mask! {
-    /// Mask for 16 16-bit lanes
-    struct mask16x16([u16; 16]);
-}
+    define_opaque_mask! {
+        /// Mask for 16 16-bit lanes.
+        ///
+        /// The layout of this type is unspecified.
+        struct mask16x16([u16; 16]);
+    }
 
-define_opaque_mask! {
-    /// Mask for 32 16-bit lanes
-    struct mask16x32([u16; 32]);
-}
+    define_opaque_mask! {
+        /// Mask for 32 16-bit lanes.
+        ///
+        /// The layout of this type is unspecified.
+        struct mask16x32([u16; 32]);
+    }
 
-define_opaque_mask! {
-    /// Mask for 2 32-bit lanes
-    struct mask32x2([u32; 2]);
-}
+    define_opaque_mask! {
+        /// Mask for 2 32-bit lanes.
+        ///
+        /// The layout of this type is unspecified.
+        struct mask32x2([u32; 2]);
+    }
 
-define_opaque_mask! {
-    /// Mask for 4 32-bit lanes
-    struct mask32x4([u32; 4]);
-}
+    define_opaque_mask! {
+        /// Mask for 4 32-bit lanes.
+        ///
+        /// The layout of this type is unspecified.
+        struct mask32x4([u32; 4]);
+    }
 
-define_opaque_mask! {
-    /// Mask for 8 32-bit lanes
-    struct mask32x8([u32; 8]);
-}
+    define_opaque_mask! {
+        /// Mask for 8 32-bit lanes.
+        ///
+        /// The layout of this type is unspecified.
+        struct mask32x8([u32; 8]);
+    }
 
-define_opaque_mask! {
-    /// Mask for 16 32-bit lanes
-    struct mask32x16([u32; 16]);
-}
+    define_opaque_mask! {
+        /// Mask for 16 32-bit lanes.
+        ///
+        /// The layout of this type is unspecified.
+        struct mask32x16([u32; 16]);
+    }
 
-define_opaque_mask! {
-    /// Mask for 2 64-bit lanes
-    struct mask64x2([u64; 2]);
-}
+    define_opaque_mask! {
+        /// Mask for 2 64-bit lanes.
+        ///
+        /// The layout of this type is unspecified.
+        struct mask64x2([u64; 2]);
+    }
 
-define_opaque_mask! {
-    /// Mask for 4 64-bit lanes
-    struct mask64x4([u64; 4]);
-}
+    define_opaque_mask! {
+        /// Mask for 4 64-bit lanes.
+        ///
+        /// The layout of this type is unspecified.
+        struct mask64x4([u64; 4]);
+    }
 
-define_opaque_mask! {
-    /// Mask for 8 64-bit lanes
-    struct mask64x8([u64; 8]);
-}
+    define_opaque_mask! {
+        /// Mask for 8 64-bit lanes.
+        ///
+        /// The layout of this type is unspecified.
+        struct mask64x8([u64; 8]);
+    }
 
-define_opaque_mask! {
-    /// Mask for 2 128-bit lanes
-    struct mask128x2([u128; 2]);
-}
+    define_opaque_mask! {
+        /// Mask for 2 128-bit lanes.
+        ///
+        /// The layout of this type is unspecified.
+        struct mask128x2([u128; 2]);
+    }
 
-define_opaque_mask! {
-    /// Mask for 4 128-bit lanes
-    struct mask128x4([u128; 4]);
-}
+    define_opaque_mask! {
+        /// Mask for 4 128-bit lanes.
+        ///
+        /// The layout of this type is unspecified.
+        struct mask128x4([u128; 4]);
+    }
 
-define_opaque_mask! {
-    /// Mask for 2 `isize`-wide lanes
-    struct masksizex2([usize; 2]);
-}
+    define_opaque_mask! {
+        /// Mask for 2 `isize`-wide lanes.
+        ///
+        /// The layout of this type is unspecified.
+        struct masksizex2([usize; 2]);
+    }
 
-define_opaque_mask! {
-    /// Mask for 4 `isize`-wide lanes
-    struct masksizex4([usize; 4]);
-}
+    define_opaque_mask! {
+        /// Mask for 4 `isize`-wide lanes.
+        ///
+        /// The layout of this type is unspecified.
+        struct masksizex4([usize; 4]);
+    }
 
-define_opaque_mask! {
-    /// Mask for 8 `isize`-wide lanes
-    struct masksizex8([usize; 8]);
+    define_opaque_mask! {
+        /// Mask for 8 `isize`-wide lanes.
+        ///
+        /// The layout of this type is unspecified.
+        struct masksizex8([usize; 8]);
+    }
 }
diff --git a/crates/core_simd/src/masks/ops.rs b/crates/core_simd/src/masks/ops.rs
new file mode 100644
index 000000000000..85ce955459a2
--- /dev/null
+++ b/crates/core_simd/src/masks/ops.rs
@@ -0,0 +1,208 @@
+/// Mask-related operations using a particular mask layout.
+pub trait MaskExt<Mask> {
+    /// Test if each lane is equal to the corresponding lane in `other`.
+    fn lanes_eq(self, other: Self) -> Mask;
+
+    /// Test if each lane is not equal to the corresponding lane in `other`.
+    fn lanes_ne(self, other: Self) -> Mask;
+
+    /// Test if each lane is less than the corresponding lane in `other`.
+    fn lanes_lt(self, other: Self) -> Mask;
+
+    /// Test if each lane is greater than the corresponding lane in `other`.
+    fn lanes_gt(self, other: Self) -> Mask;
+
+    /// Test if each lane is less than or equal to the corresponding lane in `other`.
+    fn lanes_le(self, other: Self) -> Mask;
+
+    /// Test if each lane is greater than or equal to the corresponding lane in `other`.
+    fn lanes_ge(self, other: Self) -> Mask;
+}
+
+macro_rules! implement_mask_ext {
+    { $($vector:ty => $($mask:ty),*;)* } => {
+        $( // vector
+            $( // mask
+                impl MaskExt<$mask> for $vector {
+                    #[inline]
+                    fn lanes_eq(self, other: Self) -> $mask {
+                        unsafe { crate::intrinsics::simd_eq(self, other) }
+                    }
+
+                    #[inline]
+                    fn lanes_ne(self, other: Self) -> $mask {
+                        unsafe { crate::intrinsics::simd_ne(self, other) }
+                    }
+
+                    #[inline]
+                    fn lanes_lt(self, other: Self) -> $mask {
+                        unsafe { crate::intrinsics::simd_lt(self, other) }
+                    }
+
+                    #[inline]
+                    fn lanes_gt(self, other: Self) -> $mask {
+                        unsafe { crate::intrinsics::simd_gt(self, other) }
+                    }
+
+                    #[inline]
+                    fn lanes_le(self, other: Self) -> $mask {
+                        unsafe { crate::intrinsics::simd_le(self, other) }
+                    }
+
+                    #[inline]
+                    fn lanes_ge(self, other: Self) -> $mask {
+                        unsafe { crate::intrinsics::simd_ge(self, other) }
+                    }
+                }
+            )*
+        )*
+    }
+}
+
+implement_mask_ext! {
+    crate::u8x8 => crate::masks::wide::m8x8;
+    crate::u8x16 => crate::masks::wide::m8x16;
+    crate::u8x32 => crate::masks::wide::m8x32;
+    crate::u8x64 => crate::masks::wide::m8x64;
+    crate::u16x4 => crate::masks::wide::m16x4;
+    crate::u16x8 => crate::masks::wide::m16x8;
+    crate::u16x16 => crate::masks::wide::m16x16;
+    crate::u16x32 => crate::masks::wide::m16x32;
+    crate::u32x2 => crate::masks::wide::m32x2;
+    crate::u32x4 => crate::masks::wide::m32x4;
+    crate::u32x8 => crate::masks::wide::m32x8;
+    crate::u32x16 => crate::masks::wide::m32x16;
+    crate::u64x2 => crate::masks::wide::m64x2;
+    crate::u64x4 => crate::masks::wide::m64x4;
+    crate::u64x8 => crate::masks::wide::m64x8;
+    crate::u128x2 => crate::masks::wide::m128x2;
+    crate::u128x4 => crate::masks::wide::m128x4;
+    crate::usizex2 => crate::masks::wide::msizex2;
+    crate::usizex4 => crate::masks::wide::msizex4;
+    crate::usizex8 => crate::masks::wide::msizex8;
+
+    crate::i8x8 => crate::masks::wide::m8x8;
+    crate::i8x16 => crate::masks::wide::m8x16;
+    crate::i8x32 => crate::masks::wide::m8x32;
+    crate::i8x64 => crate::masks::wide::m8x64;
+    crate::i16x4 => crate::masks::wide::m16x4;
+    crate::i16x8 => crate::masks::wide::m16x8;
+    crate::i16x16 => crate::masks::wide::m16x16;
+    crate::i16x32 => crate::masks::wide::m16x32;
+    crate::i32x2 => crate::masks::wide::m32x2;
+    crate::i32x4 => crate::masks::wide::m32x4;
+    crate::i32x8 => crate::masks::wide::m32x8;
+    crate::i32x16 => crate::masks::wide::m32x16;
+    crate::i64x2 => crate::masks::wide::m64x2;
+    crate::i64x4 => crate::masks::wide::m64x4;
+    crate::i64x8 => crate::masks::wide::m64x8;
+    crate::i128x2 => crate::masks::wide::m128x2;
+    crate::i128x4 => crate::masks::wide::m128x4;
+    crate::isizex2 => crate::masks::wide::msizex2;
+    crate::isizex4 => crate::masks::wide::msizex4;
+    crate::isizex8 => crate::masks::wide::msizex8;
+
+    crate::f32x2 => crate::masks::wide::m32x2;
+    crate::f32x4 => crate::masks::wide::m32x4;
+    crate::f32x8 => crate::masks::wide::m32x8;
+    crate::f32x16 => crate::masks::wide::m32x16;
+    crate::f64x2 => crate::masks::wide::m64x2;
+    crate::f64x4 => crate::masks::wide::m64x4;
+    crate::f64x8 => crate::masks::wide::m64x8;
+}
+
+macro_rules! implement_mask_ops {
+    { $($vector:ty => $mask:ty,)* } => {
+        $( // vector
+            impl $vector {
+                /// Test if each lane is equal to the corresponding lane in `other`.
+                #[inline]
+                pub fn lanes_eq(self, other: Self) -> $mask {
+                    <$mask>::new_from_inner(MaskExt::lanes_eq(self, other))
+                }
+
+                /// Test if each lane is not equal to the corresponding lane in `other`.
+                #[inline]
+                pub fn lanes_ne(self, other: Self) -> $mask {
+                    <$mask>::new_from_inner(MaskExt::lanes_ne(self, other))
+                }
+
+                /// Test if each lane is less than the corresponding lane in `other`.
+                #[inline]
+                pub fn lanes_lt(self, other: Self) -> $mask {
+                    <$mask>::new_from_inner(MaskExt::lanes_lt(self, other))
+                }
+
+                /// Test if each lane is greater than the corresponding lane in `other`.
+                #[inline]
+                pub fn lanes_gt(self, other: Self) -> $mask {
+                    <$mask>::new_from_inner(MaskExt::lanes_gt(self, other))
+                }
+
+                /// Test if each lane is less than or equal to the corresponding lane in `other`.
+                #[inline]
+                pub fn lanes_le(self, other: Self) -> $mask {
+                    <$mask>::new_from_inner(MaskExt::lanes_le(self, other))
+                }
+
+                /// Test if each lane is greater than or equal to the corresponding lane in `other`.
+                #[inline]
+                pub fn lanes_ge(self, other: Self) -> $mask {
+                    <$mask>::new_from_inner(MaskExt::lanes_ge(self, other))
+                }
+            }
+        )*
+    }
+}
+
+implement_mask_ops! {
+    crate::u8x8 => crate::mask8x8,
+    crate::u8x16 => crate::mask8x16,
+    crate::u8x32 => crate::mask8x32,
+    crate::u8x64 => crate::mask8x64,
+    crate::u16x4 => crate::mask16x4,
+    crate::u16x8 => crate::mask16x8,
+    crate::u16x16 => crate::mask16x16,
+    crate::u16x32 => crate::mask16x32,
+    crate::u32x2 => crate::mask32x2,
+    crate::u32x4 => crate::mask32x4,
+    crate::u32x8 => crate::mask32x8,
+    crate::u32x16 => crate::mask32x16,
+    crate::u64x2 => crate::mask64x2,
+    crate::u64x4 => crate::mask64x4,
+    crate::u64x8 => crate::mask64x8,
+    crate::u128x2 => crate::mask128x2,
+    crate::u128x4 => crate::mask128x4,
+    crate::usizex2 => crate::masksizex2,
+    crate::usizex4 => crate::masksizex4,
+    crate::usizex8 => crate::masksizex8,
+
+    crate::i8x8 => crate::mask8x8,
+    crate::i8x16 => crate::mask8x16,
+    crate::i8x32 => crate::mask8x32,
+    crate::i8x64 => crate::mask8x64,
+    crate::i16x4 => crate::mask16x4,
+    crate::i16x8 => crate::mask16x8,
+    crate::i16x16 => crate::mask16x16,
+    crate::i16x32 => crate::mask16x32,
+    crate::i32x2 => crate::mask32x2,
+    crate::i32x4 => crate::mask32x4,
+    crate::i32x8 => crate::mask32x8,
+    crate::i32x16 => crate::mask32x16,
+    crate::i64x2 => crate::mask64x2,
+    crate::i64x4 => crate::mask64x4,
+    crate::i64x8 => crate::mask64x8,
+    crate::i128x2 => crate::mask128x2,
+    crate::i128x4 => crate::mask128x4,
+    crate::isizex2 => crate::masksizex2,
+    crate::isizex4 => crate::masksizex4,
+    crate::isizex8 => crate::masksizex8,
+
+    crate::f32x2 => crate::mask32x2,
+    crate::f32x4 => crate::mask32x4,
+    crate::f32x8 => crate::mask32x8,
+    crate::f32x16 => crate::mask32x16,
+    crate::f64x2 => crate::mask64x2,
+    crate::f64x4 => crate::mask64x4,
+    crate::f64x8 => crate::mask64x8,
+}

From 78a8d615b51195c9154f95afdd5f870857ae512b Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Sun, 22 Nov 2020 15:18:31 -0500
Subject: [PATCH 057/249] Implement missing traits on opaque masks, fix tests

---
 crates/core_simd/src/masks/mod.rs             | 130 ++++++++++++++++++
 crates/core_simd/tests/helpers/biteq.rs       |  19 ++-
 crates/core_simd/tests/ops_impl/mask128.rs    |   6 +-
 crates/core_simd/tests/ops_impl/mask16.rs     |  10 +-
 crates/core_simd/tests/ops_impl/mask32.rs     |  10 +-
 crates/core_simd/tests/ops_impl/mask64.rs     |   8 +-
 crates/core_simd/tests/ops_impl/mask8.rs      |  10 +-
 .../core_simd/tests/ops_impl/mask_macros.rs   | 118 +++++++++-------
 crates/core_simd/tests/ops_impl/masksize.rs   |   8 +-
 9 files changed, 233 insertions(+), 86 deletions(-)

diff --git a/crates/core_simd/src/masks/mod.rs b/crates/core_simd/src/masks/mod.rs
index 6688db290e25..676a5560d2ff 100644
--- a/crates/core_simd/src/masks/mod.rs
+++ b/crates/core_simd/src/masks/mod.rs
@@ -158,6 +158,136 @@ macro_rules! define_opaque_mask {
                 self.0.partial_cmp(&other.0)
             }
         }
+
+        impl core::fmt::Debug for $name {
+            fn fmt(&self, f: &mut core::fmt::Formatter) -> core::fmt::Result {
+                f.debug_list()
+                    .entries((0..$lanes).map(|i| self.test(i)))
+                    .finish()
+            }
+        }
+
+        impl core::ops::BitAnd for $name {
+            type Output = Self;
+            #[inline]
+            fn bitand(self, rhs: Self) -> Self {
+                Self(self.0 & rhs.0)
+            }
+        }
+
+        impl core::ops::BitAnd<bool> for $name {
+            type Output = Self;
+            #[inline]
+            fn bitand(self, rhs: bool) -> Self {
+                self & Self::splat(rhs)
+            }
+        }
+
+        impl core::ops::BitAnd<$name> for bool {
+            type Output = $name;
+            #[inline]
+            fn bitand(self, rhs: $name) -> $name {
+                $name::splat(self) & rhs
+            }
+        }
+
+        impl core::ops::BitOr for $name {
+            type Output = Self;
+            #[inline]
+            fn bitor(self, rhs: Self) -> Self {
+                Self(self.0 | rhs.0)
+            }
+        }
+
+        impl core::ops::BitOr<bool> for $name {
+            type Output = Self;
+            #[inline]
+            fn bitor(self, rhs: bool) -> Self {
+                self | Self::splat(rhs)
+            }
+        }
+
+        impl core::ops::BitOr<$name> for bool {
+            type Output = $name;
+            #[inline]
+            fn bitor(self, rhs: $name) -> $name {
+                $name::splat(self) | rhs
+            }
+        }
+
+        impl core::ops::BitXor for $name {
+            type Output = Self;
+            #[inline]
+            fn bitxor(self, rhs: Self) -> Self::Output {
+                Self(self.0 ^ rhs.0)
+            }
+        }
+
+        impl core::ops::BitXor<bool> for $name {
+            type Output = Self;
+            #[inline]
+            fn bitxor(self, rhs: bool) -> Self::Output {
+                self ^ Self::splat(rhs)
+            }
+        }
+
+        impl core::ops::BitXor<$name> for bool {
+            type Output = $name;
+            #[inline]
+            fn bitxor(self, rhs: $name) -> Self::Output {
+                $name::splat(self) ^ rhs
+            }
+        }
+
+        impl core::ops::Not for $name {
+            type Output = $name;
+            #[inline]
+            fn not(self) -> Self::Output {
+                Self(!self.0)
+            }
+        }
+
+        impl core::ops::BitAndAssign for $name {
+            #[inline]
+            fn bitand_assign(&mut self, rhs: Self) {
+                self.0 &= rhs.0;
+            }
+        }
+
+        impl core::ops::BitAndAssign<bool> for $name {
+            #[inline]
+            fn bitand_assign(&mut self, rhs: bool) {
+                *self &= Self::splat(rhs);
+            }
+        }
+
+        impl core::ops::BitOrAssign for $name {
+            #[inline]
+            fn bitor_assign(&mut self, rhs: Self) {
+                self.0 |= rhs.0;
+            }
+        }
+
+        impl core::ops::BitOrAssign<bool> for $name {
+            #[inline]
+            fn bitor_assign(&mut self, rhs: bool) {
+                *self |= Self::splat(rhs);
+            }
+        }
+
+        impl core::ops::BitXorAssign for $name {
+            #[inline]
+            fn bitxor_assign(&mut self, rhs: Self) {
+                self.0 ^= rhs.0;
+            }
+        }
+
+        impl core::ops::BitXorAssign<bool> for $name {
+            #[inline]
+            fn bitxor_assign(&mut self, rhs: bool) {
+                *self ^= Self::splat(rhs);
+            }
+        }
     };
     { new [$width:ty; $lanes:tt] $($var:ident)* } => {
         /// Construct a vector by setting each lane to the given values.
diff --git a/crates/core_simd/tests/helpers/biteq.rs b/crates/core_simd/tests/helpers/biteq.rs
index f932eba907c3..00fc31f3d05f 100644
--- a/crates/core_simd/tests/helpers/biteq.rs
+++ b/crates/core_simd/tests/helpers/biteq.rs
@@ -70,7 +70,12 @@ impl_biteq! {
     integer impl BitEq for
         u8, u16, u32, u64, u128, usize,
         i8, i16, i32, i64, i128, isize,
-        core_simd::mask8, core_simd::mask16, core_simd::mask32, core_simd::mask64, core_simd::mask128, core_simd::masksize,
+        core_simd::masks::wide::m8,
+        core_simd::masks::wide::m16,
+        core_simd::masks::wide::m32,
+        core_simd::masks::wide::m64,
+        core_simd::masks::wide::m128,
+        core_simd::masks::wide::msize,
 }
 
 impl_biteq! {
@@ -93,12 +98,12 @@ impl_biteq! {
         core_simd::isizex2, core_simd::isizex4, core_simd::isizex8,
         core_simd::f32x2, core_simd::f32x4, core_simd::f32x8, core_simd::f32x16,
         core_simd::f64x2, core_simd::f64x4, core_simd::f64x8,
-        core_simd::mask8x8,    core_simd::mask8x16,   core_simd::mask8x32,   core_simd::mask8x64,
-        core_simd::mask16x4,   core_simd::mask16x8,   core_simd::mask16x16,  core_simd::mask16x32,
-        core_simd::mask32x2,   core_simd::mask32x4,   core_simd::mask32x8,   core_simd::mask32x16,
-        core_simd::mask64x2,   core_simd::mask64x4,   core_simd::mask64x8,
-        core_simd::mask128x2,  core_simd::mask128x4,
-        core_simd::masksizex2, core_simd::masksizex4, core_simd::masksizex8,
+        core_simd::masks::wide::m8x8,    core_simd::masks::wide::m8x16,   core_simd::masks::wide::m8x32,   core_simd::masks::wide::m8x64,
+        core_simd::masks::wide::m16x4,   core_simd::masks::wide::m16x8,   core_simd::masks::wide::m16x16,  core_simd::masks::wide::m16x32,
+        core_simd::masks::wide::m32x2,   core_simd::masks::wide::m32x4,   core_simd::masks::wide::m32x8,   core_simd::masks::wide::m32x16,
+        core_simd::masks::wide::m64x2,   core_simd::masks::wide::m64x4,   core_simd::masks::wide::m64x8,
+        core_simd::masks::wide::m128x2,  core_simd::masks::wide::m128x4,
+        core_simd::masks::wide::msizex2, core_simd::masks::wide::msizex4, core_simd::masks::wide::msizex8,
 }
 
 pub(crate) struct BitEqWrapper<'a, T>(pub(crate) &'a T);
diff --git a/crates/core_simd/tests/ops_impl/mask128.rs b/crates/core_simd/tests/ops_impl/mask128.rs
index f0bcdb4d4df9..27ba4e2d29fe 100644
--- a/crates/core_simd/tests/ops_impl/mask128.rs
+++ b/crates/core_simd/tests/ops_impl/mask128.rs
@@ -1,4 +1,2 @@
-use super::helpers;
-
-mask_tests! { mask128x2, mask128 }
-mask_tests! { mask128x4, mask128 }
+mask_tests! { mask128x2, 2 }
+mask_tests! { mask128x4, 4 }
diff --git a/crates/core_simd/tests/ops_impl/mask16.rs b/crates/core_simd/tests/ops_impl/mask16.rs
index 6f3f8e0ee02e..0fe82fa68047 100644
--- a/crates/core_simd/tests/ops_impl/mask16.rs
+++ b/crates/core_simd/tests/ops_impl/mask16.rs
@@ -1,6 +1,4 @@
-use super::helpers;
-
-mask_tests! { mask16x4, mask16 }
-mask_tests! { mask16x8, mask16 }
-mask_tests! { mask16x16, mask16 }
-mask_tests! { mask16x32, mask16 }
+mask_tests! { mask16x4, 4 }
+mask_tests! { mask16x8, 8 }
+mask_tests! { mask16x16, 16 }
+mask_tests! { mask16x32, 32 }
diff --git a/crates/core_simd/tests/ops_impl/mask32.rs b/crates/core_simd/tests/ops_impl/mask32.rs
index 5c35885a2f5b..66d987a43ce8 100644
--- a/crates/core_simd/tests/ops_impl/mask32.rs
+++ b/crates/core_simd/tests/ops_impl/mask32.rs
@@ -1,6 +1,4 @@
-use super::helpers;
-
-mask_tests! { mask32x2, mask32 }
-mask_tests! { mask32x4, mask32 }
-mask_tests! { mask32x8, mask32 }
-mask_tests! { mask32x16, mask32 }
+mask_tests! { mask32x2, 2 }
+mask_tests! { mask32x4, 4 }
+mask_tests! { mask32x8, 8 }
+mask_tests! { mask32x16, 16 }
diff --git a/crates/core_simd/tests/ops_impl/mask64.rs b/crates/core_simd/tests/ops_impl/mask64.rs
index 88d3211465c5..a1f1f67b2388 100644
--- a/crates/core_simd/tests/ops_impl/mask64.rs
+++ b/crates/core_simd/tests/ops_impl/mask64.rs
@@ -1,5 +1,3 @@
-use super::helpers;
-
-mask_tests! { mask64x2, mask64 }
-mask_tests! { mask64x4, mask64 }
-mask_tests! { mask64x8, mask64 }
+mask_tests! { mask64x2, 2 }
+mask_tests! { mask64x4, 4 }
+mask_tests! { mask64x8, 8 }
diff --git a/crates/core_simd/tests/ops_impl/mask8.rs b/crates/core_simd/tests/ops_impl/mask8.rs
index fa4bcf09f367..218fa9fe895e 100644
--- a/crates/core_simd/tests/ops_impl/mask8.rs
+++ b/crates/core_simd/tests/ops_impl/mask8.rs
@@ -1,6 +1,4 @@
-use super::helpers;
-
-mask_tests! { mask8x8, mask8 }
-mask_tests! { mask8x16, mask8 }
-mask_tests! { mask8x32, mask8 }
-mask_tests! { mask8x64, mask8 }
+mask_tests! { mask8x8, 8 }
+mask_tests! { mask8x16, 16 }
+mask_tests! { mask8x32, 32 }
+mask_tests! { mask8x64, 64 }
diff --git a/crates/core_simd/tests/ops_impl/mask_macros.rs b/crates/core_simd/tests/ops_impl/mask_macros.rs
index e6aee4c1d302..795f9e27c445 100644
--- a/crates/core_simd/tests/ops_impl/mask_macros.rs
+++ b/crates/core_simd/tests/ops_impl/mask_macros.rs
@@ -1,9 +1,9 @@
 macro_rules! mask_tests {
-    { $vector:ident, $scalar:ident } => {
+    { $vector:ident, $lanes:literal } => {
         #[cfg(test)]
         mod $vector {
-            use super::*;
-            use helpers::lanewise::*;
+            use core_simd::$vector as Vector;
+            const LANES: usize = $lanes;
 
             #[cfg(target_arch = "wasm32")]
             use wasm_bindgen_test::*;
@@ -11,15 +11,44 @@ macro_rules! mask_tests {
             #[cfg(target_arch = "wasm32")]
             wasm_bindgen_test_configure!(run_in_browser);
 
-            fn from_slice(slice: &[bool]) -> core_simd::$vector {
-                let mut value = core_simd::$vector::default();
-                let value_slice: &mut [_] = value.as_mut();
-                for (m, b) in value_slice.iter_mut().zip(slice.iter()) {
-                    *m = (*b).into();
+            fn from_slice(slice: &[bool]) -> Vector {
+                let mut value = Vector::default();
+                for (i, b) in slice.iter().take(LANES).enumerate() {
+                    value.set(i, *b);
                 }
                 value
             }
 
+            fn apply_unary_lanewise(x: Vector, f: impl Fn(bool) -> bool) -> Vector {
+                let mut value = Vector::default();
+                for i in 0..LANES {
+                    value.set(i, f(x.test(i)));
+                }
+                value
+            }
+
+            fn apply_binary_lanewise(x: Vector, y: Vector, f: impl Fn(bool, bool) -> bool) -> Vector {
+                let mut value = Vector::default();
+                for i in 0..LANES {
+                    value.set(i, f(x.test(i), y.test(i)));
+                }
+                value
+            }
+
+            fn apply_binary_scalar_lhs_lanewise(x: bool, mut y: Vector, f: impl Fn(bool, bool) -> bool) -> Vector {
+                for i in 0..LANES {
+                    y.set(i, f(x, y.test(i)));
+                }
+                y
+            }
+
+            fn apply_binary_scalar_rhs_lanewise(mut x: Vector, y: bool, f: impl Fn(bool, bool) -> bool) -> Vector {
+                for i in 0..LANES {
+                    x.set(i, f(x.test(i), y));
+                }
+                x
+            }
+
             const A: [bool; 64] = [
                 false, true, false, true, false, false, true, true,
                 false, true, false, true, false, false, true, true,
@@ -41,18 +70,13 @@ macro_rules! mask_tests {
                 false, false, true, true, false, true, false, true,
             ];
 
-            const SET_SCALAR: core_simd::$scalar = core_simd::$scalar::new(true);
-            const UNSET_SCALAR: core_simd::$scalar = core_simd::$scalar::new(false);
-            const SET_VECTOR: core_simd::$vector = core_simd::$vector::splat(SET_SCALAR);
-            const UNSET_VECTOR: core_simd::$vector = core_simd::$vector::splat(UNSET_SCALAR);
-
             #[test]
             #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn bitand() {
                 let a = from_slice(&A);
                 let b = from_slice(&B);
                 let expected = apply_binary_lanewise(a, b, core::ops::BitAnd::bitand);
-                assert_biteq!(a & b, expected);
+                assert_eq!(a & b, expected);
             }
 
             #[test]
@@ -62,7 +86,7 @@ macro_rules! mask_tests {
                 let b = from_slice(&B);
                 let expected = apply_binary_lanewise(a, b, core::ops::BitAnd::bitand);
                 a &= b;
-                assert_biteq!(a, expected);
+                assert_eq!(a, expected);
             }
 
             #[test]
@@ -70,8 +94,8 @@ macro_rules! mask_tests {
             fn bitand_scalar_rhs() {
                 let a = from_slice(&A);
                 let expected = a;
-                assert_biteq!(a & SET_SCALAR, expected);
-                assert_biteq!(a & UNSET_SCALAR, UNSET_VECTOR);
+                assert_eq!(a & true, expected);
+                assert_eq!(a & false, Vector::splat(false));
             }
 
             #[test]
@@ -79,8 +103,8 @@ macro_rules! mask_tests {
             fn bitand_scalar_lhs() {
                 let a = from_slice(&A);
                 let expected = a;
-                assert_biteq!(SET_SCALAR & a, expected);
-                assert_biteq!(UNSET_SCALAR & a, UNSET_VECTOR);
+                assert_eq!(true & a, expected);
+                assert_eq!(false & a, Vector::splat(false));
             }
 
             #[test]
@@ -88,10 +112,10 @@ macro_rules! mask_tests {
             fn bitand_assign_scalar() {
                 let mut a = from_slice(&A);
                 let expected = a;
-                a &= SET_SCALAR;
-                assert_biteq!(a, expected);
-                a &= UNSET_SCALAR;
-                assert_biteq!(a, UNSET_VECTOR);
+                a &= true;
+                assert_eq!(a, expected);
+                a &= false;
+                assert_eq!(a, Vector::splat(false));
             }
 
             #[test]
@@ -100,7 +124,7 @@ macro_rules! mask_tests {
                 let a = from_slice(&A);
                 let b = from_slice(&B);
                 let expected = apply_binary_lanewise(a, b, core::ops::BitOr::bitor);
-                assert_biteq!(a | b, expected);
+                assert_eq!(a | b, expected);
             }
 
             #[test]
@@ -110,23 +134,23 @@ macro_rules! mask_tests {
                 let b = from_slice(&B);
                 let expected = apply_binary_lanewise(a, b, core::ops::BitOr::bitor);
                 a |= b;
-                assert_biteq!(a, expected);
+                assert_eq!(a, expected);
             }
 
             #[test]
             #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn bitor_scalar_rhs() {
                 let a = from_slice(&A);
-                assert_biteq!(a | UNSET_SCALAR, a);
-                assert_biteq!(a | SET_SCALAR, SET_VECTOR);
+                assert_eq!(a | false, a);
+                assert_eq!(a | true, Vector::splat(true));
             }
 
             #[test]
             #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn bitor_scalar_lhs() {
                 let a = from_slice(&A);
-                assert_biteq!(UNSET_SCALAR | a, a);
-                assert_biteq!(SET_SCALAR | a, SET_VECTOR);
+                assert_eq!(false | a, a);
+                assert_eq!(true | a, Vector::splat(true));
             }
 
             #[test]
@@ -134,10 +158,10 @@ macro_rules! mask_tests {
             fn bitor_assign_scalar() {
                 let mut a = from_slice(&A);
                 let expected = a;
-                a |= UNSET_SCALAR;
-                assert_biteq!(a, expected);
-                a |= SET_SCALAR;
-                assert_biteq!(a, SET_VECTOR);
+                a |= false;
+                assert_eq!(a, expected);
+                a |= true;
+                assert_eq!(a, Vector::splat(true));
             }
 
             #[test]
@@ -146,7 +170,7 @@ macro_rules! mask_tests {
                 let a = from_slice(&A);
                 let b = from_slice(&B);
                 let expected = apply_binary_lanewise(a, b, core::ops::BitXor::bitxor);
-                assert_biteq!(a ^ b, expected);
+                assert_eq!(a ^ b, expected);
             }
 
             #[test]
@@ -156,25 +180,25 @@ macro_rules! mask_tests {
                 let b = from_slice(&B);
                 let expected = apply_binary_lanewise(a, b, core::ops::BitXor::bitxor);
                 a ^= b;
-                assert_biteq!(a, expected);
+                assert_eq!(a, expected);
             }
 
             #[test]
             #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn bitxor_scalar_rhs() {
                 let a = from_slice(&A);
-                let expected = apply_binary_scalar_rhs_lanewise(a, SET_SCALAR, core::ops::BitXor::bitxor);
-                assert_biteq!(a ^ UNSET_SCALAR, a);
-                assert_biteq!(a ^ SET_SCALAR, expected);
+                let expected = apply_binary_scalar_rhs_lanewise(a, true, core::ops::BitXor::bitxor);
+                assert_eq!(a ^ false, a);
+                assert_eq!(a ^ true, expected);
             }
 
             #[test]
             #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn bitxor_scalar_lhs() {
                 let a = from_slice(&A);
-                let expected = apply_binary_scalar_lhs_lanewise(SET_SCALAR, a, core::ops::BitXor::bitxor);
-                assert_biteq!(UNSET_SCALAR ^ a, a);
-                assert_biteq!(SET_SCALAR ^ a, expected);
+                let expected = apply_binary_scalar_lhs_lanewise(true, a, core::ops::BitXor::bitxor);
+                assert_eq!(false ^ a, a);
+                assert_eq!(true ^ a, expected);
             }
 
             #[test]
@@ -182,11 +206,11 @@ macro_rules! mask_tests {
             fn bitxor_assign_scalar() {
                 let mut a = from_slice(&A);
                 let expected_unset = a;
-                let expected_set = apply_binary_scalar_rhs_lanewise(a, SET_SCALAR, core::ops::BitXor::bitxor);
-                a ^= UNSET_SCALAR;
-                assert_biteq!(a, expected_unset);
-                a ^= SET_SCALAR;
-                assert_biteq!(a, expected_set);
+                let expected_set = apply_binary_scalar_rhs_lanewise(a, true, core::ops::BitXor::bitxor);
+                a ^= false;
+                assert_eq!(a, expected_unset);
+                a ^= true;
+                assert_eq!(a, expected_set);
             }
 
             #[test]
@@ -194,7 +218,7 @@ macro_rules! mask_tests {
             fn not() {
                 let v = from_slice(&A);
                 let expected = apply_unary_lanewise(v, core::ops::Not::not);
-                assert_biteq!(!v, expected);
+                assert_eq!(!v, expected);
             }
         }
     }
diff --git a/crates/core_simd/tests/ops_impl/masksize.rs b/crates/core_simd/tests/ops_impl/masksize.rs
index 76e333f3c154..e0a44d870ca5 100644
--- a/crates/core_simd/tests/ops_impl/masksize.rs
+++ b/crates/core_simd/tests/ops_impl/masksize.rs
@@ -1,5 +1,3 @@
-use super::helpers;
-
-mask_tests! { masksizex2, masksize }
-mask_tests! { masksizex4, masksize }
-mask_tests! { masksizex8, masksize }
+mask_tests! { masksizex2, 2 }
+mask_tests! { masksizex4, 4 }
+mask_tests! { masksizex8, 8 }

From 35b9ab9f5cf446712ea74ce8603267f9338c4440 Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Fri, 27 Nov 2020 00:23:49 -0500
Subject: [PATCH 058/249] Simplify some formatting

---
 crates/core_simd/src/masks/mod.rs      | 4 +---
 crates/core_simd/src/masks/wide/mod.rs | 8 ++++----
 2 files changed, 5 insertions(+), 7 deletions(-)

diff --git a/crates/core_simd/src/masks/mod.rs b/crates/core_simd/src/masks/mod.rs
index 676a5560d2ff..9fb3da006048 100644
--- a/crates/core_simd/src/masks/mod.rs
+++ b/crates/core_simd/src/masks/mod.rs
@@ -161,9 +161,7 @@ macro_rules! define_opaque_mask {
 
         impl core::fmt::Debug for $name {
             fn fmt(&self, f: &mut core::fmt::Formatter) -> core::fmt::Result {
-                f.debug_list()
-                    .entries((0..$lanes).map(|i| self.test(i)))
-                    .finish()
+                core::fmt::Debug::fmt(&self.0, f)
             }
         }
 
diff --git a/crates/core_simd/src/masks/wide/mod.rs b/crates/core_simd/src/masks/wide/mod.rs
index 1462992197dc..7df8ca7e53d5 100644
--- a/crates/core_simd/src/masks/wide/mod.rs
+++ b/crates/core_simd/src/masks/wide/mod.rs
@@ -84,25 +84,25 @@ macro_rules! define_mask {
 
         impl core::fmt::Binary for $name {
             fn fmt(&self, f: &mut core::fmt::Formatter) -> core::fmt::Result {
-                <$type as core::fmt::Binary>::fmt(&self.0, f)
+                core::fmt::Binary::fmt(&self.0, f)
             }
         }
 
         impl core::fmt::Octal for $name {
             fn fmt(&self, f: &mut core::fmt::Formatter) -> core::fmt::Result {
-                <$type as core::fmt::Octal>::fmt(&self.0, f)
+                core::fmt::Octal::fmt(&self.0, f)
             }
         }
 
         impl core::fmt::LowerHex for $name {
             fn fmt(&self, f: &mut core::fmt::Formatter) -> core::fmt::Result {
-                <$type as core::fmt::LowerHex>::fmt(&self.0, f)
+                core::fmt::LowerHex::fmt(&self.0, f)
             }
         }
 
         impl core::fmt::UpperHex for $name {
             fn fmt(&self, f: &mut core::fmt::Formatter) -> core::fmt::Result {
-                <$type as core::fmt::UpperHex>::fmt(&self.0, f)
+                core::fmt::UpperHex::fmt(&self.0, f)
             }
         }
     }

From 27e944231c3ca26add3c1a801fbe1c6de79f51fd Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Sat, 5 Dec 2020 00:03:20 -0500
Subject: [PATCH 059/249] Begin changing vectors to const generics

---
 crates/core_simd/src/fmt.rs           |  38 +---
 crates/core_simd/src/lib.rs           |  22 ++-
 crates/core_simd/src/macros.rs        | 263 +++++++++-----------------
 crates/core_simd/src/vectors_f32.rs   |  31 +--
 crates/core_simd/src/vectors_f64.rs   |  24 +--
 crates/core_simd/src/vectors_i128.rs  |  17 +-
 crates/core_simd/src/vectors_i16.rs   |  25 +--
 crates/core_simd/src/vectors_i32.rs   |  25 +--
 crates/core_simd/src/vectors_i64.rs   |  21 +-
 crates/core_simd/src/vectors_i8.rs    |  25 +--
 crates/core_simd/src/vectors_isize.rs |  21 +-
 crates/core_simd/src/vectors_u128.rs  |  17 +-
 crates/core_simd/src/vectors_u16.rs   |  25 +--
 crates/core_simd/src/vectors_u32.rs   |  25 +--
 crates/core_simd/src/vectors_u64.rs   |  21 +-
 crates/core_simd/src/vectors_u8.rs    |  25 +--
 crates/core_simd/src/vectors_usize.rs |  21 +-
 17 files changed, 241 insertions(+), 405 deletions(-)

diff --git a/crates/core_simd/src/fmt.rs b/crates/core_simd/src/fmt.rs
index 07332c1ccc8f..16b8f3b95d9a 100644
--- a/crates/core_simd/src/fmt.rs
+++ b/crates/core_simd/src/fmt.rs
@@ -30,10 +30,10 @@ debug_wrapper! {
 }
 
 macro_rules! impl_fmt_trait {
-    { $($type:ty => $(($trait:ident, $format:ident)),*;)* } => {
+    { $($type:ident => $(($trait:ident, $format:ident)),*;)* } => {
         $( // repeat type
             $( // repeat trait
-                impl core::fmt::$trait for $type {
+                impl<const LANES: usize> core::fmt::$trait for crate::$type<LANES> {
                     fn fmt(&self, f: &mut core::fmt::Formatter) -> core::fmt::Result {
                         $format(self.as_ref(), f)
                     }
@@ -41,7 +41,7 @@ macro_rules! impl_fmt_trait {
             )*
         )*
     };
-    { integers: $($type:ty,)* } => {
+    { integers: $($type:ident,)* } => {
         impl_fmt_trait! {
             $($type =>
               (Debug, format),
@@ -54,7 +54,7 @@ macro_rules! impl_fmt_trait {
             )*
         }
     };
-    { floats: $($type:ty,)* } => {
+    { floats: $($type:ident,)* } => {
         impl_fmt_trait! {
             $($type =>
               (Debug, format),
@@ -63,7 +63,7 @@ macro_rules! impl_fmt_trait {
             )*
         }
     };
-    { masks: $($type:ty,)* } => {
+    { masks: $($type:ident,)* } => {
         impl_fmt_trait! {
             $($type =>
               (Debug, format);
@@ -74,32 +74,12 @@ macro_rules! impl_fmt_trait {
 
 impl_fmt_trait! {
     integers:
-        crate::u8x8,    crate::u8x16,   crate::u8x32,   crate::u8x64,
-        crate::i8x8,    crate::i8x16,   crate::i8x32,   crate::i8x64,
-        crate::u16x4,   crate::u16x8,   crate::u16x16,  crate::u16x32,
-        crate::i16x4,   crate::i16x8,   crate::i16x16,  crate::i16x32,
-        crate::u32x2,   crate::u32x4,   crate::u32x8,   crate::u32x16,
-        crate::i32x2,   crate::i32x4,   crate::i32x8,   crate::i32x16,
-        crate::u64x2,   crate::u64x4,   crate::u64x8,
-        crate::i64x2,   crate::i64x4,   crate::i64x8,
-        crate::u128x2,  crate::u128x4,
-        crate::i128x2,  crate::i128x4,
-        crate::usizex2, crate::usizex4, crate::usizex8,
-        crate::isizex2, crate::isizex4, crate::isizex8,
+        SimdU8, SimdU16, SimdU32, SimdU64, SimdU128,
+        SimdI8, SimdI16, SimdI32, SimdI64, SimdI128,
+        SimdUsize, SimdIsize,
 }
 
 impl_fmt_trait! {
     floats:
-        crate::f32x2, crate::f32x4, crate::f32x8, crate::f32x16,
-        crate::f64x2, crate::f64x4, crate::f64x8,
-}
-
-impl_fmt_trait! {
-    masks:
-        crate::masks::wide::m8x8,    crate::masks::wide::m8x16,   crate::masks::wide::m8x32,   crate::masks::wide::m8x64,
-        crate::masks::wide::m16x4,   crate::masks::wide::m16x8,   crate::masks::wide::m16x16,  crate::masks::wide::m16x32,
-        crate::masks::wide::m32x2,   crate::masks::wide::m32x4,   crate::masks::wide::m32x8,   crate::masks::wide::m32x16,
-        crate::masks::wide::m64x2,   crate::masks::wide::m64x4,   crate::masks::wide::m64x8,
-        crate::masks::wide::m128x2,  crate::masks::wide::m128x4,
-        crate::masks::wide::msizex2, crate::masks::wide::msizex4, crate::masks::wide::msizex8,
+        SimdF32, SimdF64,
 }
diff --git a/crates/core_simd/src/lib.rs b/crates/core_simd/src/lib.rs
index 3c581ad659be..ba5e4db93128 100644
--- a/crates/core_simd/src/lib.rs
+++ b/crates/core_simd/src/lib.rs
@@ -1,5 +1,5 @@
 #![no_std]
-#![feature(repr_simd, platform_intrinsics, link_llvm_intrinsics, simd_ffi)]
+#![feature(repr_simd, platform_intrinsics, link_llvm_intrinsics, simd_ffi, min_const_generics)]
 #![warn(missing_docs)]
 //! Portable SIMD module.
 
@@ -8,10 +8,11 @@ mod macros;
 
 mod fmt;
 mod intrinsics;
-mod ops;
+//mod ops;
+//mod round;
 
-pub mod masks;
-pub use masks::opaque::*;
+//pub mod masks;
+//pub use masks::opaque::*;
 
 mod vectors_u8;
 pub use vectors_u8::*;
@@ -44,4 +45,15 @@ pub use vectors_f32::*;
 mod vectors_f64;
 pub use vectors_f64::*;
 
-mod round;
+//mod vectors_mask8;
+//pub use vectors_mask8::*;
+//mod vectors_mask16;
+//pub use vectors_mask16::*;
+//mod vectors_mask32;
+//pub use vectors_mask32::*;
+//mod vectors_mask64;
+//pub use vectors_mask64::*;
+//mod vectors_mask128;
+//pub use vectors_mask128::*;
+//mod vectors_masksize;
+//pub use vectors_masksize::*;
diff --git a/crates/core_simd/src/macros.rs b/crates/core_simd/src/macros.rs
index 75584f58b784..abf165c78b2a 100644
--- a/crates/core_simd/src/macros.rs
+++ b/crates/core_simd/src/macros.rs
@@ -136,73 +136,110 @@ macro_rules! call_counting_args {
 }
 
 /// Implements common traits on the specified vector `$name`, holding multiple `$lanes` of `$type`.
-macro_rules! base_vector_traits {
-    { $name:path => [$type:ty; $lanes:literal] } => {
-        impl Copy for $name {}
+macro_rules! impl_vector {
+    { $name:ident, $type:ty } => {
+        impl<const LANES: usize> $name<LANES> {
+            /// Construct a vector by setting all lanes to the given value.
+            pub const fn splat(value: $type) -> Self {
+                Self([value; LANES])
+            }
 
-        impl Clone for $name {
+            pub const fn as_slice(&self) -> &[$type] {
+                &self.0
+            }
+
+            pub fn as_mut_slice(&mut self) -> &mut [$type] {
+                &mut self.0
+            }
+
+            pub const fn as_ptr(&self) -> *const $type {
+                self.0.as_ptr()
+            }
+
+            pub fn as_mut_ptr(&mut self) -> *mut $type {
+                self.0.as_mut_ptr()
+            }
+
+            pub const fn from_array(array: [$type; LANES]) -> Self {
+                Self(array)
+            }
+
+            pub const fn to_array(self) -> [$type; LANES] {
+                self.0
+            }
+        }
+
+        impl<const LANES: usize> Copy for $name<LANES> {}
+
+        impl<const LANES: usize> Clone for $name<LANES> {
             #[inline]
             fn clone(&self) -> Self {
                 *self
             }
         }
 
-        impl Default for $name {
+        impl<const LANES: usize> Default for $name<LANES> {
             #[inline]
             fn default() -> Self {
                 Self::splat(<$type>::default())
             }
         }
 
-        impl PartialEq for $name {
+        impl<const LANES: usize> PartialEq for $name<LANES> {
             #[inline]
             fn eq(&self, other: &Self) -> bool {
-                AsRef::<[$type]>::as_ref(self) == AsRef::<[$type]>::as_ref(other)
+                // TODO use SIMD equality
+                self.to_array() == other.to_array()
             }
         }
 
-        impl PartialOrd for $name {
+        impl<const LANES: usize> PartialOrd for $name<LANES> {
             #[inline]
             fn partial_cmp(&self, other: &Self) -> Option<core::cmp::Ordering> {
-                AsRef::<[$type]>::as_ref(self).partial_cmp(AsRef::<[$type]>::as_ref(other))
+                // TODO use SIMD equalitya
+                self.to_array().partial_cmp(other.as_ref())
             }
         }
 
         // array references
-        impl AsRef<[$type; $lanes]> for $name {
+        impl<const LANES: usize> AsRef<[$type; LANES]> for $name<LANES> {
             #[inline]
-            fn as_ref(&self) -> &[$type; $lanes] {
-                unsafe { &*(self as *const _ as *const _) }
+            fn as_ref(&self) -> &[$type; LANES] {
+                &self.0
             }
         }
 
-        impl AsMut<[$type; $lanes]> for $name {
+        impl<const LANES: usize> AsMut<[$type; LANES]> for $name<LANES> {
             #[inline]
-            fn as_mut(&mut self) -> &mut [$type; $lanes] {
-                unsafe { &mut *(self as *mut _ as *mut _) }
+            fn as_mut(&mut self) -> &mut [$type; LANES] {
+                &mut self.0
             }
         }
 
         // slice references
-        impl AsRef<[$type]> for $name {
+        impl<const LANES: usize> AsRef<[$type]> for $name<LANES> {
             #[inline]
             fn as_ref(&self) -> &[$type] {
-                AsRef::<[$type; $lanes]>::as_ref(self)
+                &self.0
             }
         }
 
-        impl AsMut<[$type]> for $name {
+        impl<const LANES: usize> AsMut<[$type]> for $name<LANES> {
             #[inline]
             fn as_mut(&mut self) -> &mut [$type] {
-                AsMut::<[$type; $lanes]>::as_mut(self)
+                &mut self.0
             }
         }
 
         // vector/array conversion
-        from_transmute! { unsafe $name => [$type; $lanes] }
+        impl<const LANES: usize> From<[$type; LANES]> for $name<LANES> {
+            fn from(array: [$type; LANES]) -> Self {
+                Self(array)
+            }
+        }
 
         // splat
-        impl From<$type> for $name {
+        impl<const LANES: usize> From<$type> for $name<LANES> {
             #[inline]
             fn from(value: $type) -> Self {
                 Self::splat(value)
@@ -212,181 +249,59 @@ macro_rules! base_vector_traits {
 }
 
 /// Implements additional integer traits (Eq, Ord, Hash) on the specified vector `$name`, holding multiple `$lanes` of `$type`.
-macro_rules! integer_vector_traits {
-    { $name:path => [$type:ty; $lanes:literal] } => {
-        impl Eq for $name {}
+macro_rules! impl_integer_vector {
+    { $name:path, $type:ty } => {
+        impl_vector! { $name, $type }
 
-        impl Ord for $name {
+        impl<const LANES: usize> Eq for $name<LANES> {}
+
+        impl<const LANES: usize> Ord for $name<LANES> {
             #[inline]
             fn cmp(&self, other: &Self) -> core::cmp::Ordering {
-                AsRef::<[$type]>::as_ref(self).cmp(AsRef::<[$type]>::as_ref(other))
+                // TODO use SIMD cmp
+                self.to_array().cmp(other.as_ref())
             }
         }
 
-        impl core::hash::Hash for $name {
+        impl<const LANES: usize> core::hash::Hash for $name<LANES> {
             #[inline]
             fn hash<H>(&self, state: &mut H)
             where
                 H: core::hash::Hasher
             {
-                AsRef::<[$type]>::as_ref(self).hash(state)
+                self.as_slice().hash(state)
             }
         }
     }
 }
 
-/// Defines a vector `$name` containing multiple `$lanes` of `$type`.
-macro_rules! define_vector {
-    { $(#[$attr:meta])* struct $name:ident([$type:ty; $lanes:tt]); } => {
-        call_repeat! { $lanes => define_vector [$type] def $(#[$attr])* | $name | }
-
-        impl $name {
-            call_repeat! { $lanes => define_vector [$type] splat $type | }
-            call_counting_args! { $lanes => define_vector => new $type | }
-        }
-
-        base_vector_traits! { $name => [$type; $lanes] }
-    };
-    { def $(#[$attr:meta])* | $name:ident | $($itype:ty)* } => {
-        $(#[$attr])*
-        #[allow(non_camel_case_types)]
-        #[repr(simd)]
-        pub struct $name($($itype),*);
-    };
-    { splat $type:ty | $($itype:ty)* } => {
-        /// Construct a vector by setting all lanes to the given value.
-        #[inline]
-        pub const fn splat(value: $type) -> Self {
-            Self($(value as $itype),*)
-        }
-    };
-    { new $type:ty | $($var:ident)* } => {
-        /// Construct a vector by setting each lane to the given values.
-        #[allow(clippy::too_many_arguments)]
-        #[inline]
-        pub const fn new($($var: $type),*) -> Self {
-            Self($($var),*)
-        }
-    }
-}
-
 /// Implements inherent methods for a float vector `$name` containing multiple
 /// `$lanes` of float `$type`, which uses `$bits_ty` as its binary
 /// representation. Called from `define_float_vector!`.
 macro_rules! impl_float_vector {
     { $name:path => [$type:ty; $lanes:literal]; bits $bits_ty:ty; } => {
         impl $name {
-            /// Raw transmutation to an unsigned integer vector type with the
-            /// same size and number of lanes.
-            #[inline]
-            pub fn to_bits(self) -> $bits_ty {
-                unsafe { core::mem::transmute(self) }
-            }
-
-            /// Raw transmutation from an unsigned integer vector type with the
-            /// same size and number of lanes.
-            #[inline]
-            pub fn from_bits(bits: $bits_ty) -> Self {
-                unsafe { core::mem::transmute(bits) }
-            }
-
-            /// Produces a vector where every lane has the absolute value of the
-            /// equivalently-indexed lane in `self`.
-            #[inline]
-            pub fn abs(self) -> Self {
-                let no_sign = <$bits_ty>::splat(!0 >> 1);
-                Self::from_bits(self.to_bits() & no_sign)
-            }
+//            /// Raw transmutation to an unsigned integer vector type with the
+//            /// same size and number of lanes.
+//            #[inline]
+//            pub fn to_bits(self) -> $bits_ty {
+//                unsafe { core::mem::transmute(self) }
+//            }
+//
+//            /// Raw transmutation from an unsigned integer vector type with the
+//            /// same size and number of lanes.
+//            #[inline]
+//            pub fn from_bits(bits: $bits_ty) -> Self {
+//                unsafe { core::mem::transmute(bits) }
+//            }
+//
+//            /// Produces a vector where every lane has the absolute value of the
+//            /// equivalently-indexed lane in `self`.
+//            #[inline]
+//            pub fn abs(self) -> Self {
+//                let no_sign = <$bits_ty>::splat(!0 >> 1);
+//                Self::from_bits(self.to_bits() & no_sign)
+//            }
         }
     };
 }
-
-/// Defines a float vector `$name` containing multiple `$lanes` of float
-/// `$type`, which uses `$bits_ty` as its binary representation.
-macro_rules! define_float_vector {
-    { $(#[$attr:meta])* struct $name:ident([$type:ty; $lanes:tt]); bits $bits_ty:ty; } => {
-        define_vector! {
-            $(#[$attr])*
-            struct $name([$type; $lanes]);
-        }
-
-        impl_float_vector! { $name => [$type; $lanes]; bits $bits_ty; }
-    }
-}
-
-/// Defines an integer vector `$name` containing multiple `$lanes` of integer `$type`.
-macro_rules! define_integer_vector {
-    { $(#[$attr:meta])* struct $name:ident([$type:ty; $lanes:tt]); } => {
-        define_vector! {
-            $(#[$attr])*
-            struct $name([$type; $lanes]);
-        }
-
-        integer_vector_traits! { $name => [$type; $lanes] }
-    }
-}
-
-/// Defines a mask vector `$name` containing multiple `$lanes` of `$type`, represented by the
-/// underlying type `$impl_type`.
-macro_rules! define_mask_vector {
-    { $(#[$attr:meta])* struct $name:ident([$impl_type:ty as $type:ty; $lanes:tt]); } => {
-        call_repeat! { $lanes => define_mask_vector [$impl_type] def $(#[$attr])* | $name | }
-
-        impl $name {
-            call_repeat! { $lanes => define_mask_vector [$impl_type] splat $type | }
-            call_counting_args! { $lanes => define_mask_vector => new $type | }
-            call_counting_args! { $lanes => define_mask_vector => new_from_bool $type | }
-
-            /// Tests the value of the specified lane.
-            ///
-            /// # Panics
-            /// Panics if `lane` is greater than or equal to the number of lanes in the vector.
-            #[inline]
-            pub fn test(&self, lane: usize) -> bool {
-                self[lane].test()
-            }
-
-            /// Sets the value of the specified lane.
-            ///
-            /// # Panics
-            /// Panics if `lane` is greater than or equal to the number of lanes in the vector.
-            #[inline]
-            pub fn set(&mut self, lane: usize, value: bool) {
-                self[lane] = value.into();
-            }
-        }
-
-        base_vector_traits! { $name => [$type; $lanes] }
-        integer_vector_traits! { $name => [$type; $lanes] }
-    };
-    { def $(#[$attr:meta])* | $name:ident | $($itype:ty)* } => {
-        $(#[$attr])*
-        #[allow(non_camel_case_types)]
-        #[repr(simd)]
-        pub struct $name($($itype),*);
-    };
-    { splat $type:ty | $($itype:ty)* } => {
-        /// Construct a vector by setting all lanes to the given value.
-        #[inline]
-        pub const fn splat(value: $type) -> Self {
-            Self($(value.0 as $itype),*)
-        }
-    };
-    { new $type:ty | $($var:ident)* } => {
-        /// Construct a vector by setting each lane to the given values.
-        #[allow(clippy::too_many_arguments)]
-        #[inline]
-        pub const fn new($($var: $type),*) -> Self {
-            Self($($var.0),*)
-        }
-    };
-    { new_from_bool $type:ty | $($var:ident)* } => {
-        /// Used internally (since we can't use the Into trait in `const fn`s)
-        #[allow(clippy::too_many_arguments)]
-        #[allow(unused)]
-        #[inline]
-        pub(crate) const fn new_from_bool($($var: bool),*) -> Self {
-            Self($(<$type>::new($var).0),*)
-        }
-    }
-}
diff --git a/crates/core_simd/src/vectors_f32.rs b/crates/core_simd/src/vectors_f32.rs
index 17b382ee739e..fbe89bb853bd 100644
--- a/crates/core_simd/src/vectors_f32.rs
+++ b/crates/core_simd/src/vectors_f32.rs
@@ -1,29 +1,16 @@
-define_float_vector! {
-    /// Vector of two `f32` values
-    struct f32x2([f32; 2]);
-    bits crate::u32x2;
-}
+#![allow(non_camel_case_types)]
 
-define_float_vector! {
-    /// Vector of four `f32` values
-    struct f32x4([f32; 4]);
-    bits crate::u32x4;
-}
+/// A SIMD vector of containing `LANES` lanes of `f32`.
+#[repr(simd)]
+pub struct SimdF32<const LANES: usize>([f32; LANES]);
 
-define_float_vector! {
-    /// Vector of eight `f32` values
-    struct f32x8([f32; 8]);
-    bits crate::u32x8;
-}
+impl_vector! { SimdF32, f32 }
 
-define_float_vector! {
-    /// Vector of 16 `f32` values
-    struct f32x16([f32; 16]);
-    bits crate::u32x16;
-}
+pub type f32x2 = SimdF32<2>;
+pub type f32x4 = SimdF32<4>;
+pub type f32x8 = SimdF32<8>;
+pub type f32x16 = SimdF32<16>;
 
 from_transmute_x86! { unsafe f32x4 => __m128 }
 from_transmute_x86! { unsafe f32x8 => __m256 }
 //from_transmute_x86! { unsafe f32x16 => __m512 }
-
-
diff --git a/crates/core_simd/src/vectors_f64.rs b/crates/core_simd/src/vectors_f64.rs
index b41923ca6f10..f588aa4b15d3 100644
--- a/crates/core_simd/src/vectors_f64.rs
+++ b/crates/core_simd/src/vectors_f64.rs
@@ -1,20 +1,14 @@
-define_float_vector! {
-    /// Vector of two `f64` values
-    struct f64x2([f64; 2]);
-    bits crate::u64x2;
-}
+#![allow(non_camel_case_types)]
 
-define_float_vector! {
-    /// Vector of four `f64` values
-    struct f64x4([f64; 4]);
-    bits crate::u64x4;
-}
+/// A SIMD vector of containing `LANES` lanes of `f64`.
+#[repr(simd)]
+pub struct SimdF64<const LANES: usize>([f64; LANES]);
 
-define_float_vector! {
-    /// Vector of eight `f64` values
-    struct f64x8([f64; 8]);
-    bits crate::u64x8;
-}
+impl_vector! { SimdF64, f64 }
+
+pub type f64x2 = SimdF64<2>;
+pub type f64x4 = SimdF64<4>;
+pub type f64x8 = SimdF64<8>;
 
 from_transmute_x86! { unsafe f64x2 => __m128d }
 from_transmute_x86! { unsafe f64x4 => __m256d }
diff --git a/crates/core_simd/src/vectors_i128.rs b/crates/core_simd/src/vectors_i128.rs
index 5c8354070e81..46c17fa12fae 100644
--- a/crates/core_simd/src/vectors_i128.rs
+++ b/crates/core_simd/src/vectors_i128.rs
@@ -1,12 +1,13 @@
-define_integer_vector! {
-    /// Vector of two `i128` values
-    struct i128x2([i128; 2]);
-}
+#![allow(non_camel_case_types)]
 
-define_integer_vector! {
-    /// Vector of four `i128` values
-    struct i128x4([i128; 4]);
-}
+/// A SIMD vector of containing `LANES` lanes of `i128`.
+#[repr(simd)]
+pub struct SimdI128<const LANES: usize>([i128; LANES]);
+
+impl_vector! { SimdI128, i128 }
+
+pub type i128x2 = SimdI128<2>;
+pub type i128x4 = SimdI128<4>;
 
 from_transmute_x86! { unsafe i128x2 => __m256i }
 //from_transmute_x86! { unsafe i128x4 => __m512i }
diff --git a/crates/core_simd/src/vectors_i16.rs b/crates/core_simd/src/vectors_i16.rs
index 8aabd136b104..d3eefdb745c9 100644
--- a/crates/core_simd/src/vectors_i16.rs
+++ b/crates/core_simd/src/vectors_i16.rs
@@ -1,22 +1,15 @@
-define_integer_vector! {
-    /// Vector of four `i16` values
-    struct i16x4([i16; 4]);
-}
+#![allow(non_camel_case_types)]
 
-define_integer_vector! {
-    /// Vector of eight `i16` values
-    struct i16x8([i16; 8]);
-}
+/// A SIMD vector of containing `LANES` lanes of `i16`.
+#[repr(simd)]
+pub struct SimdI16<const LANES: usize>([i16; LANES]);
 
-define_integer_vector! {
-    /// Vector of 16 `i16` values
-    struct i16x16([i16; 16]);
-}
+impl_vector! { SimdI16, i16 }
 
-define_integer_vector! {
-    /// Vector of 32 `i16` values
-    struct i16x32([i16; 32]);
-}
+pub type i16x4 = SimdI16<4>;
+pub type i16x8 = SimdI16<8>;
+pub type i16x16 = SimdI16<16>;
+pub type i16x32 = SimdI16<32>;
 
 from_transmute_x86! { unsafe i16x8 => __m128i }
 from_transmute_x86! { unsafe i16x16 => __m256i }
diff --git a/crates/core_simd/src/vectors_i32.rs b/crates/core_simd/src/vectors_i32.rs
index 9aa9bc8e9dc8..63e70393d7ff 100644
--- a/crates/core_simd/src/vectors_i32.rs
+++ b/crates/core_simd/src/vectors_i32.rs
@@ -1,22 +1,15 @@
-define_integer_vector! {
-    /// Vector of two `i32` values
-    struct i32x2([i32; 2]);
-}
+#![allow(non_camel_case_types)]
 
-define_integer_vector! {
-    /// Vector of four `i32` values
-    struct i32x4([i32; 4]);
-}
+/// A SIMD vector of containing `LANES` lanes of `i32`.
+#[repr(simd)]
+pub struct SimdI32<const LANES: usize>([i32; LANES]);
 
-define_integer_vector! {
-    /// Vector of eight `i32` values
-    struct i32x8([i32; 8]);
-}
+impl_vector! { SimdI32, i32 }
 
-define_integer_vector! {
-    /// Vector of 16 `i32` values
-    struct i32x16([i32; 16]);
-}
+pub type i32x2 = SimdI32<2>;
+pub type i32x4 = SimdI32<4>;
+pub type i32x8 = SimdI32<8>;
+pub type i32x16 = SimdI32<16>;
 
 from_transmute_x86! { unsafe i32x4 => __m128i }
 from_transmute_x86! { unsafe i32x8 => __m256i }
diff --git a/crates/core_simd/src/vectors_i64.rs b/crates/core_simd/src/vectors_i64.rs
index ba66aba2095d..1d5e9b89f9a6 100644
--- a/crates/core_simd/src/vectors_i64.rs
+++ b/crates/core_simd/src/vectors_i64.rs
@@ -1,17 +1,14 @@
-define_integer_vector! {
-    /// Vector of two `i64` values
-    struct i64x2([i64; 2]);
-}
+#![allow(non_camel_case_types)]
 
-define_integer_vector! {
-    /// Vector of four `i64` values
-    struct i64x4([i64; 4]);
-}
+/// A SIMD vector of containing `LANES` lanes of `i64`.
+#[repr(simd)]
+pub struct SimdI64<const LANES: usize>([i64; LANES]);
 
-define_integer_vector! {
-    /// Vector of eight `i64` values
-    struct i64x8([i64; 8]);
-}
+impl_vector! { SimdI64, i64 }
+
+pub type i64x2 = SimdI64<2>;
+pub type i64x4 = SimdI64<4>;
+pub type i64x8 = SimdI64<8>;
 
 from_transmute_x86! { unsafe i64x2 => __m128i }
 from_transmute_x86! { unsafe i64x4 => __m256i }
diff --git a/crates/core_simd/src/vectors_i8.rs b/crates/core_simd/src/vectors_i8.rs
index 3e52d894cc22..6a38bd42122e 100644
--- a/crates/core_simd/src/vectors_i8.rs
+++ b/crates/core_simd/src/vectors_i8.rs
@@ -1,22 +1,15 @@
-define_integer_vector! {
-    /// Vector of eight `i8` values
-    struct i8x8([i8; 8]);
-}
+#![allow(non_camel_case_types)]
 
-define_integer_vector! {
-    /// Vector of 16 `i8` values
-    struct i8x16([i8; 16]);
-}
+/// A SIMD vector of containing `LANES` lanes of `i8`.
+#[repr(simd)]
+pub struct SimdI8<const LANES: usize>([i8; LANES]);
 
-define_integer_vector! {
-    /// Vector of 32 `i8` values
-    struct i8x32([i8; 32]);
-}
+impl_vector! { SimdI8, i8 }
 
-define_integer_vector! {
-    /// Vector of 64 `i8` values
-    struct i8x64([i8; 64]);
-}
+pub type i8x8 = SimdI8<8>;
+pub type i8x16 = SimdI8<16>;
+pub type i8x32 = SimdI8<32>;
+pub type i8x64 = SimdI8<64>;
 
 from_transmute_x86! { unsafe i8x16 => __m128i }
 from_transmute_x86! { unsafe i8x32 => __m256i }
diff --git a/crates/core_simd/src/vectors_isize.rs b/crates/core_simd/src/vectors_isize.rs
index 35dac8bcbd45..805aade88bbe 100644
--- a/crates/core_simd/src/vectors_isize.rs
+++ b/crates/core_simd/src/vectors_isize.rs
@@ -1,17 +1,14 @@
-define_integer_vector! {
-    /// Vector of two `isize` values
-    struct isizex2([isize; 2]);
-}
+#![allow(non_camel_case_types)]
 
-define_integer_vector! {
-    /// Vector of four `isize` values
-    struct isizex4([isize; 4]);
-}
+/// A SIMD vector of containing `LANES` lanes of `isize`.
+#[repr(simd)]
+pub struct SimdIsize<const LANES: usize>([isize; LANES]);
 
-define_integer_vector! {
-    /// Vector of eight `isize` values
-    struct isizex8([isize; 8]);
-}
+impl_vector! { SimdIsize, isize }
+
+pub type isizex2 = SimdIsize<2>;
+pub type isizex4 = SimdIsize<4>;
+pub type isizex8 = SimdIsize<8>;
 
 #[cfg(target_pointer_width = "32")]
 from_transmute_x86! { unsafe isizex4 => __m128i }
diff --git a/crates/core_simd/src/vectors_u128.rs b/crates/core_simd/src/vectors_u128.rs
index eec7bde1722f..06617876ce0b 100644
--- a/crates/core_simd/src/vectors_u128.rs
+++ b/crates/core_simd/src/vectors_u128.rs
@@ -1,12 +1,13 @@
-define_integer_vector! {
-    /// Vector of two `u128` values
-    struct u128x2([u128; 2]);
-}
+#![allow(non_camel_case_types)]
 
-define_integer_vector! {
-    /// Vector of four `u128` values
-    struct u128x4([u128; 4]);
-}
+/// A SIMD vector of containing `LANES` lanes of `u128`.
+#[repr(simd)]
+pub struct SimdU128<const LANES: usize>([u128; LANES]);
+
+impl_vector! { SimdU128, u128 }
+
+pub type u128x2 = SimdU128<2>;
+pub type u128x4 = SimdU128<4>;
 
 from_transmute_x86! { unsafe u128x2 => __m256i }
 //from_transmute_x86! { unsafe u128x4 => __m512i }
diff --git a/crates/core_simd/src/vectors_u16.rs b/crates/core_simd/src/vectors_u16.rs
index 809ab10383cd..208c0e36aa3f 100644
--- a/crates/core_simd/src/vectors_u16.rs
+++ b/crates/core_simd/src/vectors_u16.rs
@@ -1,22 +1,15 @@
-define_integer_vector! {
-    /// Vector of four `u16` values
-    struct u16x4([u16; 4]);
-}
+#![allow(non_camel_case_types)]
 
-define_integer_vector! {
-    /// Vector of eight `u16` values
-    struct u16x8([u16; 8]);
-}
+/// A SIMD vector of containing `LANES` lanes of `u16`.
+#[repr(simd)]
+pub struct SimdU16<const LANES: usize>([u16; LANES]);
 
-define_integer_vector! {
-    /// Vector of 16 `u16` values
-    struct u16x16([u16; 16]);
-}
+impl_vector! { SimdU16, u16 }
 
-define_integer_vector! {
-    /// Vector of 32 `u16` values
-    struct u16x32([u16; 32]);
-}
+pub type u16x4 = SimdU16<4>;
+pub type u16x8 = SimdU16<8>;
+pub type u16x16 = SimdU16<16>;
+pub type u16x32 = SimdU16<32>;
 
 from_transmute_x86! { unsafe u16x8 => __m128i }
 from_transmute_x86! { unsafe u16x16 => __m256i }
diff --git a/crates/core_simd/src/vectors_u32.rs b/crates/core_simd/src/vectors_u32.rs
index b00c63d9058f..8a5239e7daf8 100644
--- a/crates/core_simd/src/vectors_u32.rs
+++ b/crates/core_simd/src/vectors_u32.rs
@@ -1,22 +1,15 @@
-define_integer_vector! {
-    /// Vector of two `u32` values
-    struct u32x2([u32; 2]);
-}
+#![allow(non_camel_case_types)]
 
-define_integer_vector! {
-    /// Vector of four `u32` values
-    struct u32x4([u32; 4]);
-}
+/// A SIMD vector of containing `LANES` lanes of `u32`.
+#[repr(simd)]
+pub struct SimdU32<const LANES: usize>([u32; LANES]);
 
-define_integer_vector! {
-    /// Vector of eight `u32` values
-    struct u32x8([u32; 8]);
-}
+impl_vector! { SimdU32, u32 }
 
-define_integer_vector! {
-    /// Vector of 16 `u32` values
-    struct u32x16([u32; 16]);
-}
+pub type u32x2 = SimdU32<2>;
+pub type u32x4 = SimdU32<4>;
+pub type u32x8 = SimdU32<8>;
+pub type u32x16 = SimdU32<16>;
 
 from_transmute_x86! { unsafe u32x4 => __m128i }
 from_transmute_x86! { unsafe u32x8 => __m256i }
diff --git a/crates/core_simd/src/vectors_u64.rs b/crates/core_simd/src/vectors_u64.rs
index 0bcf28ebc265..48b8a9ef3908 100644
--- a/crates/core_simd/src/vectors_u64.rs
+++ b/crates/core_simd/src/vectors_u64.rs
@@ -1,17 +1,14 @@
-define_integer_vector! {
-    /// Vector of two `u64` values
-    struct u64x2([u64; 2]);
-}
+#![allow(non_camel_case_types)]
 
-define_integer_vector! {
-    /// Vector of four `u64` values
-    struct u64x4([u64; 4]);
-}
+/// A SIMD vector of containing `LANES` lanes of `u64`.
+#[repr(simd)]
+pub struct SimdU64<const LANES: usize>([u64; LANES]);
 
-define_integer_vector! {
-    /// Vector of eight `u64` values
-    struct u64x8([u64; 8]);
-}
+impl_vector! { SimdU64, u64 }
+
+pub type u64x2 = SimdU64<2>;
+pub type u64x4 = SimdU64<4>;
+pub type u64x8 = SimdU64<8>;
 
 from_transmute_x86! { unsafe u64x2 => __m128i }
 from_transmute_x86! { unsafe u64x4 => __m256i }
diff --git a/crates/core_simd/src/vectors_u8.rs b/crates/core_simd/src/vectors_u8.rs
index a187bc6f7b42..83a179eff575 100644
--- a/crates/core_simd/src/vectors_u8.rs
+++ b/crates/core_simd/src/vectors_u8.rs
@@ -1,22 +1,15 @@
-define_integer_vector! {
-    /// Vector of eight `u8` values
-    struct u8x8([u8; 8]);
-}
+#![allow(non_camel_case_types)]
 
-define_integer_vector! {
-    /// Vector of 16 `u8` values
-    struct u8x16([u8; 16]);
-}
+/// A SIMD vector of containing `LANES` lanes of `u8`.
+#[repr(simd)]
+pub struct SimdU8<const LANES: usize>([u8; LANES]);
 
-define_integer_vector! {
-    /// Vector of 32 `u8` values
-    struct u8x32([u8; 32]);
-}
+impl_vector! { SimdU8, u8 }
 
-define_integer_vector! {
-    /// Vector of 64 `u8` values
-    struct u8x64([u8; 64]);
-}
+pub type u8x8 = SimdU8<8>;
+pub type u8x16 = SimdU8<16>;
+pub type u8x32 = SimdU8<32>;
+pub type u8x64 = SimdU8<64>;
 
 from_transmute_x86! { unsafe u8x16 => __m128i }
 from_transmute_x86! { unsafe u8x32 => __m256i }
diff --git a/crates/core_simd/src/vectors_usize.rs b/crates/core_simd/src/vectors_usize.rs
index 84a4b8e509b3..096b6cea08d0 100644
--- a/crates/core_simd/src/vectors_usize.rs
+++ b/crates/core_simd/src/vectors_usize.rs
@@ -1,17 +1,14 @@
-define_integer_vector! {
-    /// Vector of two `usize` values
-    struct usizex2([usize; 2]);
-}
+#![allow(non_camel_case_types)]
 
-define_integer_vector! {
-    /// Vector of four `usize` values
-    struct usizex4([usize; 4]);
-}
+/// A SIMD vector of containing `LANES` lanes of `usize`.
+#[repr(simd)]
+pub struct SimdUsize<const LANES: usize>([usize; LANES]);
 
-define_integer_vector! {
-    /// Vector of eight `usize` values
-    struct usizex8([usize; 8]);
-}
+impl_vector! { SimdUsize, usize }
+
+pub type usizex2 = SimdUsize<2>;
+pub type usizex4 = SimdUsize<4>;
+pub type usizex8 = SimdUsize<8>;
 
 #[cfg(target_pointer_width = "32")]
 from_transmute_x86! { unsafe usizex4 => __m128i }

From 22576bb6e0080fc8eb00d40168b3b608d3756735 Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Sat, 5 Dec 2020 23:49:42 -0500
Subject: [PATCH 060/249] Implement additional functions

---
 crates/core_simd/src/macros.rs        | 52 +++++++++++++--------------
 crates/core_simd/src/vectors_f32.rs   |  2 +-
 crates/core_simd/src/vectors_f64.rs   |  2 +-
 crates/core_simd/src/vectors_i128.rs  |  2 +-
 crates/core_simd/src/vectors_i16.rs   |  2 +-
 crates/core_simd/src/vectors_i32.rs   |  2 +-
 crates/core_simd/src/vectors_i64.rs   |  2 +-
 crates/core_simd/src/vectors_i8.rs    |  2 +-
 crates/core_simd/src/vectors_isize.rs |  2 +-
 crates/core_simd/src/vectors_u128.rs  |  2 +-
 crates/core_simd/src/vectors_u16.rs   |  2 +-
 crates/core_simd/src/vectors_u32.rs   |  2 +-
 crates/core_simd/src/vectors_u64.rs   |  2 +-
 crates/core_simd/src/vectors_u8.rs    |  2 +-
 crates/core_simd/src/vectors_usize.rs |  2 +-
 15 files changed, 40 insertions(+), 40 deletions(-)

diff --git a/crates/core_simd/src/macros.rs b/crates/core_simd/src/macros.rs
index abf165c78b2a..c588505daf87 100644
--- a/crates/core_simd/src/macros.rs
+++ b/crates/core_simd/src/macros.rs
@@ -139,31 +139,27 @@ macro_rules! call_counting_args {
 macro_rules! impl_vector {
     { $name:ident, $type:ty } => {
         impl<const LANES: usize> $name<LANES> {
-            /// Construct a vector by setting all lanes to the given value.
+            /// Construct a SIMD vector by setting all lanes to the given value.
             pub const fn splat(value: $type) -> Self {
                 Self([value; LANES])
             }
 
+            /// Returns a slice containing the entire SIMD vector.
             pub const fn as_slice(&self) -> &[$type] {
                 &self.0
             }
 
+            /// Returns a mutable slice containing the entire SIMD vector.
             pub fn as_mut_slice(&mut self) -> &mut [$type] {
                 &mut self.0
             }
 
-            pub const fn as_ptr(&self) -> *const $type {
-                self.0.as_ptr()
-            }
-
-            pub fn as_mut_ptr(&mut self) -> *mut $type {
-                self.0.as_mut_ptr()
-            }
-
+            /// Converts an array to a SIMD vector.
             pub const fn from_array(array: [$type; LANES]) -> Self {
                 Self(array)
             }
 
+            /// Converts a SIMD vector to an array.
             pub const fn to_array(self) -> [$type; LANES] {
                 self.0
             }
@@ -250,7 +246,7 @@ macro_rules! impl_vector {
 
 /// Implements additional integer traits (Eq, Ord, Hash) on the specified vector `$name`, holding multiple `$lanes` of `$type`.
 macro_rules! impl_integer_vector {
-    { $name:path, $type:ty } => {
+    { $name:ident, $type:ty } => {
         impl_vector! { $name, $type }
 
         impl<const LANES: usize> Eq for $name<LANES> {}
@@ -279,22 +275,26 @@ macro_rules! impl_integer_vector {
 /// `$lanes` of float `$type`, which uses `$bits_ty` as its binary
 /// representation. Called from `define_float_vector!`.
 macro_rules! impl_float_vector {
-    { $name:path => [$type:ty; $lanes:literal]; bits $bits_ty:ty; } => {
-        impl $name {
-//            /// Raw transmutation to an unsigned integer vector type with the
-//            /// same size and number of lanes.
-//            #[inline]
-//            pub fn to_bits(self) -> $bits_ty {
-//                unsafe { core::mem::transmute(self) }
-//            }
-//
-//            /// Raw transmutation from an unsigned integer vector type with the
-//            /// same size and number of lanes.
-//            #[inline]
-//            pub fn from_bits(bits: $bits_ty) -> Self {
-//                unsafe { core::mem::transmute(bits) }
-//            }
-//
+    { $name:ident, $type:ty, $bits_ty:ident } => {
+        impl_vector! { $name, $type }
+
+        impl<const LANES: usize> $name<LANES> {
+            /// Raw transmutation to an unsigned integer vector type with the
+            /// same size and number of lanes.
+            #[inline]
+            pub fn to_bits(self) -> crate::$bits_ty<LANES> {
+                assert_eq!(core::mem::size_of::<Self>(), core::mem::size_of::<crate::$bits_ty<LANES>>());
+                unsafe { core::mem::transmute_copy(&self) }
+            }
+
+            /// Raw transmutation from an unsigned integer vector type with the
+            /// same size and number of lanes.
+            #[inline]
+            pub fn from_bits(bits: crate::$bits_ty<LANES>) -> Self {
+                assert_eq!(core::mem::size_of::<Self>(), core::mem::size_of::<crate::$bits_ty<LANES>>());
+                unsafe { core::mem::transmute_copy(&bits) }
+            }
+
 //            /// Produces a vector where every lane has the absolute value of the
 //            /// equivalently-indexed lane in `self`.
 //            #[inline]
diff --git a/crates/core_simd/src/vectors_f32.rs b/crates/core_simd/src/vectors_f32.rs
index fbe89bb853bd..b1e13408cc95 100644
--- a/crates/core_simd/src/vectors_f32.rs
+++ b/crates/core_simd/src/vectors_f32.rs
@@ -4,7 +4,7 @@
 #[repr(simd)]
 pub struct SimdF32<const LANES: usize>([f32; LANES]);
 
-impl_vector! { SimdF32, f32 }
+impl_float_vector! { SimdF32, f32, SimdU32 }
 
 pub type f32x2 = SimdF32<2>;
 pub type f32x4 = SimdF32<4>;
diff --git a/crates/core_simd/src/vectors_f64.rs b/crates/core_simd/src/vectors_f64.rs
index f588aa4b15d3..4297c9d636c3 100644
--- a/crates/core_simd/src/vectors_f64.rs
+++ b/crates/core_simd/src/vectors_f64.rs
@@ -4,7 +4,7 @@
 #[repr(simd)]
 pub struct SimdF64<const LANES: usize>([f64; LANES]);
 
-impl_vector! { SimdF64, f64 }
+impl_float_vector! { SimdF64, f64, SimdU64 }
 
 pub type f64x2 = SimdF64<2>;
 pub type f64x4 = SimdF64<4>;
diff --git a/crates/core_simd/src/vectors_i128.rs b/crates/core_simd/src/vectors_i128.rs
index 46c17fa12fae..a48c823cbd66 100644
--- a/crates/core_simd/src/vectors_i128.rs
+++ b/crates/core_simd/src/vectors_i128.rs
@@ -4,7 +4,7 @@
 #[repr(simd)]
 pub struct SimdI128<const LANES: usize>([i128; LANES]);
 
-impl_vector! { SimdI128, i128 }
+impl_integer_vector! { SimdI128, i128 }
 
 pub type i128x2 = SimdI128<2>;
 pub type i128x4 = SimdI128<4>;
diff --git a/crates/core_simd/src/vectors_i16.rs b/crates/core_simd/src/vectors_i16.rs
index d3eefdb745c9..7bc522287a3b 100644
--- a/crates/core_simd/src/vectors_i16.rs
+++ b/crates/core_simd/src/vectors_i16.rs
@@ -4,7 +4,7 @@
 #[repr(simd)]
 pub struct SimdI16<const LANES: usize>([i16; LANES]);
 
-impl_vector! { SimdI16, i16 }
+impl_integer_vector! { SimdI16, i16 }
 
 pub type i16x4 = SimdI16<4>;
 pub type i16x8 = SimdI16<8>;
diff --git a/crates/core_simd/src/vectors_i32.rs b/crates/core_simd/src/vectors_i32.rs
index 63e70393d7ff..05533bb0b6d7 100644
--- a/crates/core_simd/src/vectors_i32.rs
+++ b/crates/core_simd/src/vectors_i32.rs
@@ -4,7 +4,7 @@
 #[repr(simd)]
 pub struct SimdI32<const LANES: usize>([i32; LANES]);
 
-impl_vector! { SimdI32, i32 }
+impl_integer_vector! { SimdI32, i32 }
 
 pub type i32x2 = SimdI32<2>;
 pub type i32x4 = SimdI32<4>;
diff --git a/crates/core_simd/src/vectors_i64.rs b/crates/core_simd/src/vectors_i64.rs
index 1d5e9b89f9a6..e669e8a367c0 100644
--- a/crates/core_simd/src/vectors_i64.rs
+++ b/crates/core_simd/src/vectors_i64.rs
@@ -4,7 +4,7 @@
 #[repr(simd)]
 pub struct SimdI64<const LANES: usize>([i64; LANES]);
 
-impl_vector! { SimdI64, i64 }
+impl_integer_vector! { SimdI64, i64 }
 
 pub type i64x2 = SimdI64<2>;
 pub type i64x4 = SimdI64<4>;
diff --git a/crates/core_simd/src/vectors_i8.rs b/crates/core_simd/src/vectors_i8.rs
index 6a38bd42122e..55a440cc92fa 100644
--- a/crates/core_simd/src/vectors_i8.rs
+++ b/crates/core_simd/src/vectors_i8.rs
@@ -4,7 +4,7 @@
 #[repr(simd)]
 pub struct SimdI8<const LANES: usize>([i8; LANES]);
 
-impl_vector! { SimdI8, i8 }
+impl_integer_vector! { SimdI8, i8 }
 
 pub type i8x8 = SimdI8<8>;
 pub type i8x16 = SimdI8<16>;
diff --git a/crates/core_simd/src/vectors_isize.rs b/crates/core_simd/src/vectors_isize.rs
index 805aade88bbe..6d9b20615326 100644
--- a/crates/core_simd/src/vectors_isize.rs
+++ b/crates/core_simd/src/vectors_isize.rs
@@ -4,7 +4,7 @@
 #[repr(simd)]
 pub struct SimdIsize<const LANES: usize>([isize; LANES]);
 
-impl_vector! { SimdIsize, isize }
+impl_integer_vector! { SimdIsize, isize }
 
 pub type isizex2 = SimdIsize<2>;
 pub type isizex4 = SimdIsize<4>;
diff --git a/crates/core_simd/src/vectors_u128.rs b/crates/core_simd/src/vectors_u128.rs
index 06617876ce0b..54ad6e191f73 100644
--- a/crates/core_simd/src/vectors_u128.rs
+++ b/crates/core_simd/src/vectors_u128.rs
@@ -4,7 +4,7 @@
 #[repr(simd)]
 pub struct SimdU128<const LANES: usize>([u128; LANES]);
 
-impl_vector! { SimdU128, u128 }
+impl_integer_vector! { SimdU128, u128 }
 
 pub type u128x2 = SimdU128<2>;
 pub type u128x4 = SimdU128<4>;
diff --git a/crates/core_simd/src/vectors_u16.rs b/crates/core_simd/src/vectors_u16.rs
index 208c0e36aa3f..7b0e345ef159 100644
--- a/crates/core_simd/src/vectors_u16.rs
+++ b/crates/core_simd/src/vectors_u16.rs
@@ -4,7 +4,7 @@
 #[repr(simd)]
 pub struct SimdU16<const LANES: usize>([u16; LANES]);
 
-impl_vector! { SimdU16, u16 }
+impl_integer_vector! { SimdU16, u16 }
 
 pub type u16x4 = SimdU16<4>;
 pub type u16x8 = SimdU16<8>;
diff --git a/crates/core_simd/src/vectors_u32.rs b/crates/core_simd/src/vectors_u32.rs
index 8a5239e7daf8..f80efbc59ebb 100644
--- a/crates/core_simd/src/vectors_u32.rs
+++ b/crates/core_simd/src/vectors_u32.rs
@@ -4,7 +4,7 @@
 #[repr(simd)]
 pub struct SimdU32<const LANES: usize>([u32; LANES]);
 
-impl_vector! { SimdU32, u32 }
+impl_integer_vector! { SimdU32, u32 }
 
 pub type u32x2 = SimdU32<2>;
 pub type u32x4 = SimdU32<4>;
diff --git a/crates/core_simd/src/vectors_u64.rs b/crates/core_simd/src/vectors_u64.rs
index 48b8a9ef3908..848d90faaa73 100644
--- a/crates/core_simd/src/vectors_u64.rs
+++ b/crates/core_simd/src/vectors_u64.rs
@@ -4,7 +4,7 @@
 #[repr(simd)]
 pub struct SimdU64<const LANES: usize>([u64; LANES]);
 
-impl_vector! { SimdU64, u64 }
+impl_integer_vector! { SimdU64, u64 }
 
 pub type u64x2 = SimdU64<2>;
 pub type u64x4 = SimdU64<4>;
diff --git a/crates/core_simd/src/vectors_u8.rs b/crates/core_simd/src/vectors_u8.rs
index 83a179eff575..b172801aa993 100644
--- a/crates/core_simd/src/vectors_u8.rs
+++ b/crates/core_simd/src/vectors_u8.rs
@@ -4,7 +4,7 @@
 #[repr(simd)]
 pub struct SimdU8<const LANES: usize>([u8; LANES]);
 
-impl_vector! { SimdU8, u8 }
+impl_integer_vector! { SimdU8, u8 }
 
 pub type u8x8 = SimdU8<8>;
 pub type u8x16 = SimdU8<16>;
diff --git a/crates/core_simd/src/vectors_usize.rs b/crates/core_simd/src/vectors_usize.rs
index 096b6cea08d0..b0655ab311b1 100644
--- a/crates/core_simd/src/vectors_usize.rs
+++ b/crates/core_simd/src/vectors_usize.rs
@@ -4,7 +4,7 @@
 #[repr(simd)]
 pub struct SimdUsize<const LANES: usize>([usize; LANES]);
 
-impl_vector! { SimdUsize, usize }
+impl_integer_vector! { SimdUsize, usize }
 
 pub type usizex2 = SimdUsize<2>;
 pub type usizex4 = SimdUsize<4>;

From 25c7640fb331112d5bf445c0893af89ee3e9ea9d Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Sun, 6 Dec 2020 00:36:33 -0500
Subject: [PATCH 061/249] Reenable ops and fix tests

---
 crates/core_simd/src/lib.rs                   |   2 +-
 crates/core_simd/src/macros.rs                |  14 +-
 crates/core_simd/src/ops.rs                   | 283 ++++++------------
 crates/core_simd/tests/helpers/biteq.rs       |  12 -
 .../core_simd/tests/ops_impl/float_macros.rs  |   3 +
 .../core_simd/tests/ops_impl/mask_macros.rs   |   2 +
 6 files changed, 112 insertions(+), 204 deletions(-)

diff --git a/crates/core_simd/src/lib.rs b/crates/core_simd/src/lib.rs
index ba5e4db93128..d23e5ad21ba2 100644
--- a/crates/core_simd/src/lib.rs
+++ b/crates/core_simd/src/lib.rs
@@ -8,7 +8,7 @@ mod macros;
 
 mod fmt;
 mod intrinsics;
-//mod ops;
+mod ops;
 //mod round;
 
 //pub mod masks;
diff --git a/crates/core_simd/src/macros.rs b/crates/core_simd/src/macros.rs
index c588505daf87..99adb669bc51 100644
--- a/crates/core_simd/src/macros.rs
+++ b/crates/core_simd/src/macros.rs
@@ -295,13 +295,13 @@ macro_rules! impl_float_vector {
                 unsafe { core::mem::transmute_copy(&bits) }
             }
 
-//            /// Produces a vector where every lane has the absolute value of the
-//            /// equivalently-indexed lane in `self`.
-//            #[inline]
-//            pub fn abs(self) -> Self {
-//                let no_sign = <$bits_ty>::splat(!0 >> 1);
-//                Self::from_bits(self.to_bits() & no_sign)
-//            }
+            /// Produces a vector where every lane has the absolute value of the
+            /// equivalently-indexed lane in `self`.
+            #[inline]
+            pub fn abs(self) -> Self {
+                let no_sign = crate::$bits_ty::splat(!0 >> 1);
+                Self::from_bits(self.to_bits() & no_sign)
+            }
         }
     };
 }
diff --git a/crates/core_simd/src/ops.rs b/crates/core_simd/src/ops.rs
index ac89feca9d66..942d071de44d 100644
--- a/crates/core_simd/src/ops.rs
+++ b/crates/core_simd/src/ops.rs
@@ -12,21 +12,21 @@ where
 macro_rules! impl_ref_ops {
     // binary op
     {
-        impl core::ops::$trait:ident<$rhs:ty> for $type:ty {
+        impl<const $lanes:ident: usize> core::ops::$trait:ident<$rhs:ty> for $type:ty {
             type Output = $output:ty;
 
             $(#[$attrs:meta])*
             fn $fn:ident($self_tok:ident, $rhs_arg:ident: $rhs_arg_ty:ty) -> Self::Output $body:tt
         }
     } => {
-        impl core::ops::$trait<$rhs> for $type {
+        impl<const $lanes: usize> core::ops::$trait<$rhs> for $type {
             type Output = $output;
 
             $(#[$attrs])*
             fn $fn($self_tok, $rhs_arg: $rhs_arg_ty) -> Self::Output $body
         }
 
-        impl core::ops::$trait<&'_ $rhs> for $type {
+        impl<const $lanes: usize> core::ops::$trait<&'_ $rhs> for $type {
             type Output = <$type as core::ops::$trait<$rhs>>::Output;
 
             $(#[$attrs])*
@@ -35,7 +35,7 @@ macro_rules! impl_ref_ops {
             }
         }
 
-        impl core::ops::$trait<$rhs> for &'_ $type {
+        impl<const $lanes: usize> core::ops::$trait<$rhs> for &'_ $type {
             type Output = <$type as core::ops::$trait<$rhs>>::Output;
 
             $(#[$attrs])*
@@ -44,7 +44,7 @@ macro_rules! impl_ref_ops {
             }
         }
 
-        impl core::ops::$trait<&'_ $rhs> for &'_ $type {
+        impl<const $lanes: usize> core::ops::$trait<&'_ $rhs> for &'_ $type {
             type Output = <$type as core::ops::$trait<$rhs>>::Output;
 
             $(#[$attrs])*
@@ -56,17 +56,17 @@ macro_rules! impl_ref_ops {
 
     // binary assignment op
     {
-        impl core::ops::$trait:ident<$rhs:ty> for $type:ty {
+        impl<const $lanes:ident: usize> core::ops::$trait:ident<$rhs:ty> for $type:ty {
             $(#[$attrs:meta])*
             fn $fn:ident(&mut $self_tok:ident, $rhs_arg:ident: $rhs_arg_ty:ty) $body:tt
         }
     } => {
-        impl core::ops::$trait<$rhs> for $type {
+        impl<const $lanes: usize> core::ops::$trait<$rhs> for $type {
             $(#[$attrs])*
             fn $fn(&mut $self_tok, $rhs_arg: $rhs_arg_ty) $body
         }
 
-        impl core::ops::$trait<&'_ $rhs> for $type {
+        impl<const $lanes: usize> core::ops::$trait<&'_ $rhs> for $type {
             $(#[$attrs])*
             fn $fn(&mut $self_tok, $rhs_arg: &$rhs_arg_ty) {
                 core::ops::$trait::$fn($self_tok, *$rhs_arg)
@@ -76,17 +76,17 @@ macro_rules! impl_ref_ops {
 
     // unary op
     {
-        impl core::ops::$trait:ident for $type:ty {
+        impl<const $lanes:ident: usize> core::ops::$trait:ident for $type:ty {
             type Output = $output:ty;
             fn $fn:ident($self_tok:ident) -> Self::Output $body:tt
         }
     } => {
-        impl core::ops::$trait for $type {
+        impl<const $lanes: usize> core::ops::$trait for $type {
             type Output = $output;
             fn $fn($self_tok) -> Self::Output $body
         }
 
-        impl core::ops::$trait for &'_ $type {
+        impl<const $lanes: usize> core::ops::$trait for &'_ $type {
             type Output = <$type as core::ops::$trait>::Output;
             fn $fn($self_tok) -> Self::Output {
                 core::ops::$trait::$fn(*$self_tok)
@@ -95,152 +95,76 @@ macro_rules! impl_ref_ops {
     }
 }
 
-/// Implements op traits for masks
-macro_rules! impl_mask_element_ops {
-    { $($mask:ty),* } => {
-        $(
-            impl_ref_ops! {
-                impl core::ops::BitAnd<$mask> for $mask {
-                    type Output = Self;
-                    fn bitand(self, rhs: Self) -> Self::Output {
-                        Self(self.0 & rhs.0)
-                    }
-                }
-            }
-
-            impl_ref_ops! {
-                impl core::ops::BitAndAssign<$mask> for $mask {
-                    fn bitand_assign(&mut self, rhs: Self) {
-                        *self = *self & rhs;
-                    }
-                }
-            }
-
-            impl_ref_ops! {
-                impl core::ops::BitOr<$mask> for $mask {
-                    type Output = Self;
-                    fn bitor(self, rhs: Self) -> Self::Output {
-                        Self(self.0 | rhs.0)
-                    }
-                }
-            }
-
-            impl_ref_ops! {
-                impl core::ops::BitOrAssign<$mask> for $mask {
-                    fn bitor_assign(&mut self, rhs: Self) {
-                        *self = *self | rhs;
-                    }
-                }
-            }
-
-            impl_ref_ops! {
-                impl core::ops::BitXor<$mask> for $mask {
-                    type Output = Self;
-                    fn bitxor(self, rhs: Self) -> Self::Output {
-                        Self(self.0 ^ rhs.0)
-                    }
-                }
-            }
-
-            impl_ref_ops! {
-                impl core::ops::BitXorAssign<$mask> for $mask {
-                    fn bitxor_assign(&mut self, rhs: Self) {
-                        *self = *self ^ rhs;
-                    }
-                }
-            }
-
-            impl_ref_ops! {
-                impl core::ops::Not for $mask {
-                    type Output = Self;
-                    fn not(self) -> Self::Output {
-                        Self(!self.0)
-                    }
-                }
-            }
-        )*
-    }
-}
-
-impl_mask_element_ops! {
-    crate::masks::wide::m8,
-    crate::masks::wide::m16,
-    crate::masks::wide::m32,
-    crate::masks::wide::m64,
-    crate::masks::wide::m128,
-    crate::masks::wide::msize
-}
-
 /// Automatically implements operators over vectors and scalars for a particular vector.
 macro_rules! impl_op {
-    { impl Add for $type:ty, $scalar:ty } => {
+    { impl Add for $type:ident, $scalar:ty } => {
         impl_op! { @binary $type, $scalar, Add::add, AddAssign::add_assign, simd_add }
     };
-    { impl Sub for $type:ty, $scalar:ty } => {
+    { impl Sub for $type:ident, $scalar:ty } => {
         impl_op! { @binary $type, $scalar, Sub::sub, SubAssign::sub_assign, simd_sub }
     };
-    { impl Mul for $type:ty, $scalar:ty } => {
+    { impl Mul for $type:ident, $scalar:ty } => {
         impl_op! { @binary $type, $scalar, Mul::mul, MulAssign::mul_assign, simd_mul }
     };
-    { impl Div for $type:ty, $scalar:ty } => {
+    { impl Div for $type:ident, $scalar:ty } => {
         impl_op! { @binary $type, $scalar, Div::div, DivAssign::div_assign, simd_div }
     };
-    { impl Rem for $type:ty, $scalar:ty } => {
+    { impl Rem for $type:ident, $scalar:ty } => {
         impl_op! { @binary $type, $scalar, Rem::rem, RemAssign::rem_assign, simd_rem }
     };
-    { impl Shl for $type:ty, $scalar:ty } => {
+    { impl Shl for $type:ident, $scalar:ty } => {
         impl_op! { @binary $type, $scalar, Shl::shl, ShlAssign::shl_assign, simd_shl }
     };
-    { impl Shr for $type:ty, $scalar:ty } => {
+    { impl Shr for $type:ident, $scalar:ty } => {
         impl_op! { @binary $type, $scalar, Shr::shr, ShrAssign::shr_assign, simd_shr }
     };
-    { impl BitAnd for $type:ty, $scalar:ty } => {
+    { impl BitAnd for $type:ident, $scalar:ty } => {
         impl_op! { @binary $type, $scalar, BitAnd::bitand, BitAndAssign::bitand_assign, simd_and }
     };
-    { impl BitOr for $type:ty, $scalar:ty } => {
+    { impl BitOr for $type:ident, $scalar:ty } => {
         impl_op! { @binary $type, $scalar, BitOr::bitor, BitOrAssign::bitor_assign, simd_or }
     };
-    { impl BitXor for $type:ty, $scalar:ty } => {
+    { impl BitXor for $type:ident, $scalar:ty } => {
         impl_op! { @binary $type, $scalar, BitXor::bitxor, BitXorAssign::bitxor_assign, simd_xor }
     };
 
-    { impl Not for $type:ty, $scalar:ty } => {
+    { impl Not for $type:ident, $scalar:ty } => {
         impl_ref_ops! {
-            impl core::ops::Not for $type {
+            impl<const LANES: usize> core::ops::Not for crate::$type<LANES> {
                 type Output = Self;
                 fn not(self) -> Self::Output {
-                    self ^ <$type>::splat(!<$scalar>::default())
+                    self ^ Self::splat(!<$scalar>::default())
                 }
             }
         }
     };
 
-    { impl Neg for $type:ty, $scalar:ty } => {
+    { impl Neg for $type:ident, $scalar:ty } => {
         impl_ref_ops! {
-            impl core::ops::Neg for $type {
+            impl<const LANES: usize> core::ops::Neg for crate::$type<LANES> {
                 type Output = Self;
                 fn neg(self) -> Self::Output {
-                    <$type>::splat(0) - self
+                    Self::splat(0) - self
                 }
             }
         }
     };
 
-    { impl Neg for $type:ty, $scalar:ty, @float } => {
+    { impl Neg for $type:ident, $scalar:ty, @float } => {
         impl_ref_ops! {
-            impl core::ops::Neg for $type {
+            impl<const LANES: usize> core::ops::Neg for crate::$type<LANES> {
                 type Output = Self;
                 fn neg(self) -> Self::Output {
                     // FIXME: Replace this with fneg intrinsic once available.
                     // https://github.com/rust-lang/stdsimd/issues/32
-                    Self::from_bits(<$type>::splat(-0.0).to_bits() ^ self.to_bits())
+                    Self::from_bits(Self::splat(-0.0).to_bits() ^ self.to_bits())
                 }
             }
         }
     };
 
-    { impl Index for $type:ty, $scalar:ty } => {
-        impl<I> core::ops::Index<I> for $type
+    { impl Index for $type:ident, $scalar:ty } => {
+        impl<I, const LANES: usize> core::ops::Index<I> for crate::$type<LANES>
         where
             I: core::slice::SliceIndex<[$scalar]>,
         {
@@ -251,7 +175,7 @@ macro_rules! impl_op {
             }
         }
 
-        impl<I> core::ops::IndexMut<I> for $type
+        impl<I, const LANES: usize> core::ops::IndexMut<I> for crate::$type<LANES>
         where
             I: core::slice::SliceIndex<[$scalar]>,
         {
@@ -263,13 +187,13 @@ macro_rules! impl_op {
     };
 
     // generic binary op with assignment when output is `Self`
-    { @binary $type:ty, $scalar:ty, $trait:ident :: $trait_fn:ident, $assign_trait:ident :: $assign_trait_fn:ident, $intrinsic:ident } => {
+    { @binary $type:ident, $scalar:ty, $trait:ident :: $trait_fn:ident, $assign_trait:ident :: $assign_trait_fn:ident, $intrinsic:ident } => {
         impl_ref_ops! {
-            impl core::ops::$trait<$type> for $type {
-                type Output = $type;
+            impl<const LANES: usize> core::ops::$trait<Self> for crate::$type<LANES> {
+                type Output = Self;
 
                 #[inline]
-                fn $trait_fn(self, rhs: $type) -> Self::Output {
+                fn $trait_fn(self, rhs: Self) -> Self::Output {
                     unsafe {
                         crate::intrinsics::$intrinsic(self, rhs)
                     }
@@ -278,31 +202,31 @@ macro_rules! impl_op {
         }
 
         impl_ref_ops! {
-            impl core::ops::$trait<$scalar> for $type {
-                type Output = $type;
+            impl<const LANES: usize> core::ops::$trait<$scalar> for crate::$type<LANES> {
+                type Output = Self;
 
                 #[inline]
                 fn $trait_fn(self, rhs: $scalar) -> Self::Output {
-                    core::ops::$trait::$trait_fn(self, <$type>::splat(rhs))
+                    core::ops::$trait::$trait_fn(self, Self::splat(rhs))
                 }
             }
         }
 
         impl_ref_ops! {
-            impl core::ops::$trait<$type> for $scalar {
-                type Output = $type;
+            impl<const LANES: usize> core::ops::$trait<crate::$type<LANES>> for $scalar {
+                type Output = crate::$type<LANES>;
 
                 #[inline]
-                fn $trait_fn(self, rhs: $type) -> Self::Output {
-                    core::ops::$trait::$trait_fn(<$type>::splat(self), rhs)
+                fn $trait_fn(self, rhs: crate::$type<LANES>) -> Self::Output {
+                    core::ops::$trait::$trait_fn(crate::$type::splat(self), rhs)
                 }
             }
         }
 
         impl_ref_ops! {
-            impl core::ops::$assign_trait<$type> for $type {
+            impl<const LANES: usize> core::ops::$assign_trait<Self> for crate::$type<LANES> {
                 #[inline]
-                fn $assign_trait_fn(&mut self, rhs: $type) {
+                fn $assign_trait_fn(&mut self, rhs: Self) {
                     unsafe {
                         *self = crate::intrinsics::$intrinsic(*self, rhs);
                     }
@@ -311,10 +235,10 @@ macro_rules! impl_op {
         }
 
         impl_ref_ops! {
-            impl core::ops::$assign_trait<$scalar> for $type {
+            impl<const LANES: usize> core::ops::$assign_trait<$scalar> for crate::$type<LANES> {
                 #[inline]
                 fn $assign_trait_fn(&mut self, rhs: $scalar) {
-                    core::ops::$assign_trait::$assign_trait_fn(self, <$type>::splat(rhs));
+                    core::ops::$assign_trait::$assign_trait_fn(self, Self::splat(rhs));
                 }
             }
         }
@@ -323,7 +247,7 @@ macro_rules! impl_op {
 
 /// Implements floating-point operators for the provided types.
 macro_rules! impl_float_ops {
-    { $($scalar:ty => $($vector:ty),*;)* } => {
+    { $($scalar:ty => $($vector:ident),*;)* } => {
         $( // scalar
             $( // vector
                 impl_op! { impl Add for $vector, $scalar }
@@ -340,7 +264,7 @@ macro_rules! impl_float_ops {
 
 /// Implements mask operators for the provided types.
 macro_rules! impl_mask_ops {
-    { $($scalar:ty => $($vector:ty),*;)* } => {
+    { $($scalar:ty => $($vector:ident),*;)* } => {
         $( // scalar
             $( // vector
                 impl_op! { impl BitAnd for $vector, $scalar }
@@ -355,7 +279,7 @@ macro_rules! impl_mask_ops {
 
 /// Implements unsigned integer operators for the provided types.
 macro_rules! impl_unsigned_int_ops {
-    { $($scalar:ty => $($vector:ty),*;)* } => {
+    { $($scalar:ty => $($vector:ident),*;)* } => {
         $( // scalar
             $( // vector
                 impl_op! { impl Add for $vector, $scalar }
@@ -369,11 +293,11 @@ macro_rules! impl_unsigned_int_ops {
 
                 // Integers panic on divide by 0
                 impl_ref_ops! {
-                    impl core::ops::Div<$vector> for $vector {
+                    impl<const LANES: usize> core::ops::Div<Self> for crate::$vector<LANES> {
                         type Output = Self;
 
                         #[inline]
-                        fn div(self, rhs: $vector) -> Self::Output {
+                        fn div(self, rhs: Self) -> Self::Output {
                             // TODO there is probably a better way of doing this
                             if AsRef::<[$scalar]>::as_ref(&rhs)
                                 .iter()
@@ -387,8 +311,8 @@ macro_rules! impl_unsigned_int_ops {
                 }
 
                 impl_ref_ops! {
-                    impl core::ops::Div<$scalar> for $vector {
-                        type Output = $vector;
+                    impl<const LANES: usize> core::ops::Div<$scalar> for crate::$vector<LANES> {
+                        type Output = Self;
 
                         #[inline]
                         fn div(self, rhs: $scalar) -> Self::Output {
@@ -402,18 +326,18 @@ macro_rules! impl_unsigned_int_ops {
                 }
 
                 impl_ref_ops! {
-                    impl core::ops::Div<$vector> for $scalar {
-                        type Output = $vector;
+                    impl<const LANES: usize> core::ops::Div<crate::$vector<LANES>> for $scalar {
+                        type Output = crate::$vector<LANES>;
 
                         #[inline]
-                        fn div(self, rhs: $vector) -> Self::Output {
-                            <$vector>::splat(self) / rhs
+                        fn div(self, rhs: crate::$vector<LANES>) -> Self::Output {
+                            crate::$vector::splat(self) / rhs
                         }
                     }
                 }
 
                 impl_ref_ops! {
-                    impl core::ops::DivAssign<$vector> for $vector {
+                    impl<const LANES: usize> core::ops::DivAssign<Self> for crate::$vector<LANES> {
                         #[inline]
                         fn div_assign(&mut self, rhs: Self) {
                             *self = *self / rhs;
@@ -422,7 +346,7 @@ macro_rules! impl_unsigned_int_ops {
                 }
 
                 impl_ref_ops! {
-                    impl core::ops::DivAssign<$scalar> for $vector {
+                    impl<const LANES: usize> core::ops::DivAssign<$scalar> for crate::$vector<LANES> {
                         #[inline]
                         fn div_assign(&mut self, rhs: $scalar) {
                             *self = *self / rhs;
@@ -432,11 +356,11 @@ macro_rules! impl_unsigned_int_ops {
 
                 // remainder panics on zero divisor
                 impl_ref_ops! {
-                    impl core::ops::Rem<$vector> for $vector {
+                    impl<const LANES: usize> core::ops::Rem<Self> for crate::$vector<LANES> {
                         type Output = Self;
 
                         #[inline]
-                        fn rem(self, rhs: $vector) -> Self::Output {
+                        fn rem(self, rhs: Self) -> Self::Output {
                             // TODO there is probably a better way of doing this
                             if AsRef::<[$scalar]>::as_ref(&rhs)
                                 .iter()
@@ -450,8 +374,8 @@ macro_rules! impl_unsigned_int_ops {
                 }
 
                 impl_ref_ops! {
-                    impl core::ops::Rem<$scalar> for $vector {
-                        type Output = $vector;
+                    impl<const LANES: usize> core::ops::Rem<$scalar> for crate::$vector<LANES> {
+                        type Output = Self;
 
                         #[inline]
                         fn rem(self, rhs: $scalar) -> Self::Output {
@@ -465,18 +389,18 @@ macro_rules! impl_unsigned_int_ops {
                 }
 
                 impl_ref_ops! {
-                    impl core::ops::Rem<$vector> for $scalar {
-                        type Output = $vector;
+                    impl<const LANES: usize> core::ops::Rem<crate::$vector<LANES>> for $scalar {
+                        type Output = crate::$vector<LANES>;
 
                         #[inline]
-                        fn rem(self, rhs: $vector) -> Self::Output {
-                            <$vector>::splat(self) % rhs
+                        fn rem(self, rhs: crate::$vector<LANES>) -> Self::Output {
+                            crate::$vector::splat(self) % rhs
                         }
                     }
                 }
 
                 impl_ref_ops! {
-                    impl core::ops::RemAssign<$vector> for $vector {
+                    impl<const LANES: usize> core::ops::RemAssign<Self> for crate::$vector<LANES> {
                         #[inline]
                         fn rem_assign(&mut self, rhs: Self) {
                             *self = *self % rhs;
@@ -485,7 +409,7 @@ macro_rules! impl_unsigned_int_ops {
                 }
 
                 impl_ref_ops! {
-                    impl core::ops::RemAssign<$scalar> for $vector {
+                    impl<const LANES: usize> core::ops::RemAssign<$scalar> for crate::$vector<LANES> {
                         #[inline]
                         fn rem_assign(&mut self, rhs: $scalar) {
                             *self = *self % rhs;
@@ -495,11 +419,11 @@ macro_rules! impl_unsigned_int_ops {
 
                 // shifts panic on overflow
                 impl_ref_ops! {
-                    impl core::ops::Shl<$vector> for $vector {
+                    impl<const LANES: usize> core::ops::Shl<Self> for crate::$vector<LANES> {
                         type Output = Self;
 
                         #[inline]
-                        fn shl(self, rhs: $vector) -> Self::Output {
+                        fn shl(self, rhs: Self) -> Self::Output {
                             // TODO there is probably a better way of doing this
                             if AsRef::<[$scalar]>::as_ref(&rhs)
                                 .iter()
@@ -514,8 +438,8 @@ macro_rules! impl_unsigned_int_ops {
                 }
 
                 impl_ref_ops! {
-                    impl core::ops::Shl<$scalar> for $vector {
-                        type Output = $vector;
+                    impl<const LANES: usize> core::ops::Shl<$scalar> for crate::$vector<LANES> {
+                        type Output = Self;
 
                         #[inline]
                         fn shl(self, rhs: $scalar) -> Self::Output {
@@ -530,7 +454,7 @@ macro_rules! impl_unsigned_int_ops {
 
 
                 impl_ref_ops! {
-                    impl core::ops::ShlAssign<$vector> for $vector {
+                    impl<const LANES: usize> core::ops::ShlAssign<Self> for crate::$vector<LANES> {
                         #[inline]
                         fn shl_assign(&mut self, rhs: Self) {
                             *self = *self << rhs;
@@ -539,7 +463,7 @@ macro_rules! impl_unsigned_int_ops {
                 }
 
                 impl_ref_ops! {
-                    impl core::ops::ShlAssign<$scalar> for $vector {
+                    impl<const LANES: usize> core::ops::ShlAssign<$scalar> for crate::$vector<LANES> {
                         #[inline]
                         fn shl_assign(&mut self, rhs: $scalar) {
                             *self = *self << rhs;
@@ -548,13 +472,13 @@ macro_rules! impl_unsigned_int_ops {
                 }
 
                 impl_ref_ops! {
-                    impl core::ops::Shr<$vector> for $vector {
+                    impl<const LANES: usize> core::ops::Shr<Self> for crate::$vector<LANES> {
                         type Output = Self;
 
                         #[inline]
-                        fn shr(self, rhs: $vector) -> Self::Output {
+                        fn shr(self, rhs: Self) -> Self::Output {
                             // TODO there is probably a better way of doing this
-                            if AsRef::<[$scalar]>::as_ref(&rhs)
+                            if rhs.as_slice()
                                 .iter()
                                 .copied()
                                 .any(invalid_shift_rhs)
@@ -567,8 +491,8 @@ macro_rules! impl_unsigned_int_ops {
                 }
 
                 impl_ref_ops! {
-                    impl core::ops::Shr<$scalar> for $vector {
-                        type Output = $vector;
+                    impl<const LANES: usize> core::ops::Shr<$scalar> for crate::$vector<LANES> {
+                        type Output = Self;
 
                         #[inline]
                         fn shr(self, rhs: $scalar) -> Self::Output {
@@ -583,7 +507,7 @@ macro_rules! impl_unsigned_int_ops {
 
 
                 impl_ref_ops! {
-                    impl core::ops::ShrAssign<$vector> for $vector {
+                    impl<const LANES: usize> core::ops::ShrAssign<Self> for crate::$vector<LANES> {
                         #[inline]
                         fn shr_assign(&mut self, rhs: Self) {
                             *self = *self >> rhs;
@@ -592,7 +516,7 @@ macro_rules! impl_unsigned_int_ops {
                 }
 
                 impl_ref_ops! {
-                    impl core::ops::ShrAssign<$scalar> for $vector {
+                    impl<const LANES: usize> core::ops::ShrAssign<$scalar> for crate::$vector<LANES> {
                         #[inline]
                         fn shr_assign(&mut self, rhs: $scalar) {
                             *self = *self >> rhs;
@@ -606,7 +530,7 @@ macro_rules! impl_unsigned_int_ops {
 
 /// Implements unsigned integer operators for the provided types.
 macro_rules! impl_signed_int_ops {
-    { $($scalar:ty => $($vector:ty),*;)* } => {
+    { $($scalar:ty => $($vector:ident),*;)* } => {
         impl_unsigned_int_ops! { $($scalar => $($vector),*;)* }
         $( // scalar
             $( // vector
@@ -617,33 +541,24 @@ macro_rules! impl_signed_int_ops {
 }
 
 impl_unsigned_int_ops! {
-    u8    => crate::u8x8,    crate::u8x16,   crate::u8x32,   crate::u8x64;
-    u16   => crate::u16x4,   crate::u16x8,   crate::u16x16,  crate::u16x32;
-    u32   => crate::u32x2,   crate::u32x4,   crate::u32x8,   crate::u32x16;
-    u64   => crate::u64x2,   crate::u64x4,   crate::u64x8;
-    u128  => crate::u128x2,  crate::u128x4;
-    usize => crate::usizex2, crate::usizex4, crate::usizex8;
+    u8 => SimdU8;
+    u16 => SimdU16;
+    u32 => SimdU32;
+    u64 => SimdU64;
+    u128 => SimdU128;
+    usize => SimdUsize;
 }
 
 impl_signed_int_ops! {
-    i8    => crate::i8x8,    crate::i8x16,   crate::i8x32,   crate::i8x64;
-    i16   => crate::i16x4,   crate::i16x8,   crate::i16x16,  crate::i16x32;
-    i32   => crate::i32x2,   crate::i32x4,   crate::i32x8,   crate::i32x16;
-    i64   => crate::i64x2,   crate::i64x4,   crate::i64x8;
-    i128  => crate::i128x2,  crate::i128x4;
-    isize => crate::isizex2, crate::isizex4, crate::isizex8;
+    i8 => SimdI8;
+    i16 => SimdI16;
+    i32 => SimdI32;
+    i64 => SimdI64;
+    i128 => SimdI128;
+    isize => SimdIsize;
 }
 
 impl_float_ops! {
-    f32 => crate::f32x2, crate::f32x4, crate::f32x8, crate::f32x16;
-    f64 => crate::f64x2, crate::f64x4, crate::f64x8;
-}
-
-impl_mask_ops! {
-    crate::masks::wide::m8    => crate::masks::wide::m8x8,    crate::masks::wide::m8x16,   crate::masks::wide::m8x32,   crate::masks::wide::m8x64;
-    crate::masks::wide::m16   => crate::masks::wide::m16x4,   crate::masks::wide::m16x8,   crate::masks::wide::m16x16,  crate::masks::wide::m16x32;
-    crate::masks::wide::m32   => crate::masks::wide::m32x2,   crate::masks::wide::m32x4,   crate::masks::wide::m32x8,   crate::masks::wide::m32x16;
-    crate::masks::wide::m64   => crate::masks::wide::m64x2,   crate::masks::wide::m64x4,   crate::masks::wide::m64x8;
-    crate::masks::wide::m128  => crate::masks::wide::m128x2,  crate::masks::wide::m128x4;
-    crate::masks::wide::msize => crate::masks::wide::msizex2, crate::masks::wide::msizex4, crate::masks::wide::msizex8;
+    f32 => SimdF32;
+    f64 => SimdF64;
 }
diff --git a/crates/core_simd/tests/helpers/biteq.rs b/crates/core_simd/tests/helpers/biteq.rs
index 00fc31f3d05f..9da2bdfce42e 100644
--- a/crates/core_simd/tests/helpers/biteq.rs
+++ b/crates/core_simd/tests/helpers/biteq.rs
@@ -70,12 +70,6 @@ impl_biteq! {
     integer impl BitEq for
         u8, u16, u32, u64, u128, usize,
         i8, i16, i32, i64, i128, isize,
-        core_simd::masks::wide::m8,
-        core_simd::masks::wide::m16,
-        core_simd::masks::wide::m32,
-        core_simd::masks::wide::m64,
-        core_simd::masks::wide::m128,
-        core_simd::masks::wide::msize,
 }
 
 impl_biteq! {
@@ -98,12 +92,6 @@ impl_biteq! {
         core_simd::isizex2, core_simd::isizex4, core_simd::isizex8,
         core_simd::f32x2, core_simd::f32x4, core_simd::f32x8, core_simd::f32x16,
         core_simd::f64x2, core_simd::f64x4, core_simd::f64x8,
-        core_simd::masks::wide::m8x8,    core_simd::masks::wide::m8x16,   core_simd::masks::wide::m8x32,   core_simd::masks::wide::m8x64,
-        core_simd::masks::wide::m16x4,   core_simd::masks::wide::m16x8,   core_simd::masks::wide::m16x16,  core_simd::masks::wide::m16x32,
-        core_simd::masks::wide::m32x2,   core_simd::masks::wide::m32x4,   core_simd::masks::wide::m32x8,   core_simd::masks::wide::m32x16,
-        core_simd::masks::wide::m64x2,   core_simd::masks::wide::m64x4,   core_simd::masks::wide::m64x8,
-        core_simd::masks::wide::m128x2,  core_simd::masks::wide::m128x4,
-        core_simd::masks::wide::msizex2, core_simd::masks::wide::msizex4, core_simd::masks::wide::msizex8,
 }
 
 pub(crate) struct BitEqWrapper<'a, T>(pub(crate) &'a T);
diff --git a/crates/core_simd/tests/ops_impl/float_macros.rs b/crates/core_simd/tests/ops_impl/float_macros.rs
index fe347a5362da..a46367d0cc2f 100644
--- a/crates/core_simd/tests/ops_impl/float_macros.rs
+++ b/crates/core_simd/tests/ops_impl/float_macros.rs
@@ -335,6 +335,8 @@ macro_rules! float_tests {
                 }
             }
 
+            // TODO reenable after converting float ops to platform intrinsics
+            /*
             #[test]
             #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn ceil_odd_floats() {
@@ -413,6 +415,7 @@ macro_rules! float_tests {
                     assert_biteq!(core_simd::$vector::round_from_int(v), expected);
                 }
             }
+            */
         }
     }
 }
diff --git a/crates/core_simd/tests/ops_impl/mask_macros.rs b/crates/core_simd/tests/ops_impl/mask_macros.rs
index 795f9e27c445..3aaa036b9940 100644
--- a/crates/core_simd/tests/ops_impl/mask_macros.rs
+++ b/crates/core_simd/tests/ops_impl/mask_macros.rs
@@ -1,5 +1,6 @@
 macro_rules! mask_tests {
     { $vector:ident, $lanes:literal } => {
+        /*
         #[cfg(test)]
         mod $vector {
             use core_simd::$vector as Vector;
@@ -221,5 +222,6 @@ macro_rules! mask_tests {
                 assert_eq!(!v, expected);
             }
         }
+        */
     }
 }

From 9cc3deaa9256060868bb952ea5f850a910633f19 Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Sun, 13 Dec 2020 23:58:33 -0500
Subject: [PATCH 062/249] Finish refactoring vector types

---
 crates/core_simd/src/intrinsics.rs            |   2 +-
 crates/core_simd/src/lib.rs                   |   4 +-
 crates/core_simd/src/masks/full_masks/mod.rs  | 199 +++++
 .../{wide => full_masks}/vectors_m128.rs      |   0
 .../masks/{wide => full_masks}/vectors_m16.rs |   0
 .../masks/{wide => full_masks}/vectors_m32.rs |   0
 .../masks/{wide => full_masks}/vectors_m64.rs |   0
 .../masks/{wide => full_masks}/vectors_m8.rs  |   3 +
 .../{wide => full_masks}/vectors_msize.rs     |   0
 crates/core_simd/src/masks/mod.rs             | 682 ++++++++----------
 crates/core_simd/src/masks/ops.rs             | 208 ------
 crates/core_simd/src/masks/wide/mod.rs        | 139 ----
 crates/core_simd/src/vectors_f32.rs           |   9 +-
 crates/core_simd/src/vectors_f64.rs           |   7 +-
 crates/core_simd/src/vectors_i128.rs          |   5 +-
 crates/core_simd/src/vectors_i16.rs           |   9 +-
 crates/core_simd/src/vectors_i32.rs           |   9 +-
 crates/core_simd/src/vectors_i64.rs           |   7 +-
 crates/core_simd/src/vectors_i8.rs            |   9 +-
 crates/core_simd/src/vectors_isize.rs         |   7 +-
 crates/core_simd/src/vectors_u128.rs          |   5 +-
 crates/core_simd/src/vectors_u16.rs           |   9 +-
 crates/core_simd/src/vectors_u32.rs           |   9 +-
 crates/core_simd/src/vectors_u64.rs           |   7 +-
 crates/core_simd/src/vectors_u8.rs            |   9 +-
 crates/core_simd/src/vectors_usize.rs         |   7 +-
 .../core_simd/tests/ops_impl/mask_macros.rs   |   2 -
 27 files changed, 615 insertions(+), 732 deletions(-)
 create mode 100644 crates/core_simd/src/masks/full_masks/mod.rs
 rename crates/core_simd/src/masks/{wide => full_masks}/vectors_m128.rs (100%)
 rename crates/core_simd/src/masks/{wide => full_masks}/vectors_m16.rs (100%)
 rename crates/core_simd/src/masks/{wide => full_masks}/vectors_m32.rs (100%)
 rename crates/core_simd/src/masks/{wide => full_masks}/vectors_m64.rs (100%)
 rename crates/core_simd/src/masks/{wide => full_masks}/vectors_m8.rs (82%)
 rename crates/core_simd/src/masks/{wide => full_masks}/vectors_msize.rs (100%)
 delete mode 100644 crates/core_simd/src/masks/ops.rs
 delete mode 100644 crates/core_simd/src/masks/wide/mod.rs

diff --git a/crates/core_simd/src/intrinsics.rs b/crates/core_simd/src/intrinsics.rs
index 3dfc77136f2d..51689cd97bea 100644
--- a/crates/core_simd/src/intrinsics.rs
+++ b/crates/core_simd/src/intrinsics.rs
@@ -1,7 +1,7 @@
 //! This module contains the LLVM intrinsics bindings that provide the functionality for this
 //! crate.
 //!
-//! The LLVM assembly language is documented here: https://llvm.org/docs/LangRef.html
+//! The LLVM assembly language is documented here: <https://llvm.org/docs/LangRef.html>
 
 /// These intrinsics aren't linked directly from LLVM and are mostly undocumented, however they are
 /// simply lowered to the matching LLVM instructions by the compiler.  The associated instruction
diff --git a/crates/core_simd/src/lib.rs b/crates/core_simd/src/lib.rs
index d23e5ad21ba2..9d4ce683f22a 100644
--- a/crates/core_simd/src/lib.rs
+++ b/crates/core_simd/src/lib.rs
@@ -11,8 +11,8 @@ mod intrinsics;
 mod ops;
 //mod round;
 
-//pub mod masks;
-//pub use masks::opaque::*;
+mod masks;
+pub use masks::*;
 
 mod vectors_u8;
 pub use vectors_u8::*;
diff --git a/crates/core_simd/src/masks/full_masks/mod.rs b/crates/core_simd/src/masks/full_masks/mod.rs
new file mode 100644
index 000000000000..829174669c2a
--- /dev/null
+++ b/crates/core_simd/src/masks/full_masks/mod.rs
@@ -0,0 +1,199 @@
+//! Masks that take up full SIMD vector registers.
+
+/// The error type returned when converting an integer to a mask fails.
+#[derive(Debug, Copy, Clone, PartialEq, Eq)]
+pub struct TryFromMaskError(());
+
+impl core::fmt::Display for TryFromMaskError {
+    fn fmt(&self, f: &mut core::fmt::Formatter) -> core::fmt::Result {
+        write!(f, "mask vector must have all bits set or unset in each lane")
+    }
+}
+
+macro_rules! define_mask {
+    { $(#[$attr:meta])* struct $name:ident<const $lanes:ident: usize>($type:ty); } => {
+        $(#[$attr])*
+        #[derive(Copy, Clone, Default, PartialEq, PartialOrd, Eq, Ord, Hash)]
+        #[repr(transparent)]
+        pub struct $name<const $lanes: usize>($type);
+
+        delegate_ops_to_inner! { $name }
+
+        impl<const $lanes: usize> $name<$lanes> {
+            /// Construct a mask by setting all lanes to the given value.
+            pub fn splat(value: bool) -> Self {
+                Self(<$type>::splat(value.into()))
+            }
+
+            /// Tests the value of the specified lane.
+            ///
+            /// # Panics
+            /// Panics if `lane` is greater than or equal to the number of lanes in the vector.
+            #[inline]
+            pub fn test(&self, lane: usize) -> bool {
+                self.0[lane] > 0
+            }
+
+            /// Sets the value of the specified lane.
+            ///
+            /// # Panics
+            /// Panics if `lane` is greater than or equal to the number of lanes in the vector.
+            #[inline]
+            pub fn set(&mut self, lane: usize, value: bool) {
+                self.0[lane] = if value {
+                    !0
+                } else {
+                    0
+                }
+            }
+        }
+
+        impl<const $lanes: usize> core::convert::From<bool> for $name<$lanes> {
+            fn from(value: bool) -> Self {
+                Self::splat(value)
+            }
+        }
+
+        impl<const $lanes: usize> core::convert::TryFrom<$type> for $name<$lanes> {
+            type Error = TryFromMaskError;
+            fn try_from(value: $type) -> Result<Self, Self::Error> {
+                if value.as_slice().iter().all(|x| *x == 0 || !*x == 0) {
+                    Ok(Self(value))
+                } else {
+                    Err(TryFromMaskError(()))
+                }
+            }
+        }
+
+        impl<const $lanes: usize> core::convert::From<$name<$lanes>> for $type {
+            fn from(value: $name<$lanes>) -> Self {
+                value.0
+            }
+        }
+
+        impl<const $lanes: usize> core::fmt::Debug for $name<$lanes> {
+            fn fmt(&self, f: &mut core::fmt::Formatter) -> core::fmt::Result {
+                f.debug_list()
+                    .entries((0..LANES).map(|lane| self.test(lane)))
+                    .finish()
+            }
+        }
+
+        impl<const $lanes: usize> core::fmt::Binary for $name<$lanes> {
+            fn fmt(&self, f: &mut core::fmt::Formatter) -> core::fmt::Result {
+                core::fmt::Binary::fmt(&self.0, f)
+            }
+        }
+
+        impl<const $lanes: usize> core::fmt::Octal for $name<$lanes> {
+            fn fmt(&self, f: &mut core::fmt::Formatter) -> core::fmt::Result {
+                core::fmt::Octal::fmt(&self.0, f)
+            }
+        }
+
+        impl<const $lanes: usize> core::fmt::LowerHex for $name<$lanes> {
+            fn fmt(&self, f: &mut core::fmt::Formatter) -> core::fmt::Result {
+                core::fmt::LowerHex::fmt(&self.0, f)
+            }
+        }
+
+        impl<const $lanes: usize> core::fmt::UpperHex for $name<$lanes> {
+            fn fmt(&self, f: &mut core::fmt::Formatter) -> core::fmt::Result {
+                core::fmt::UpperHex::fmt(&self.0, f)
+            }
+        }
+    }
+}
+
+define_mask! {
+    /// A mask equivalent to [SimdI8](crate::SimdI8), where all bits in the lane must be either set
+    /// or unset.
+    struct SimdI8Mask<const LANES: usize>(crate::SimdI8<LANES>);
+}
+
+define_mask! {
+    /// A mask equivalent to [SimdI16](crate::SimdI16), where all bits in the lane must be either set
+    /// or unset.
+    struct SimdI16Mask<const LANES: usize>(crate::SimdI16<LANES>);
+}
+
+define_mask! {
+    /// A mask equivalent to [SimdI32](crate::SimdI32), where all bits in the lane must be either set
+    /// or unset.
+    struct SimdI32Mask<const LANES: usize>(crate::SimdI32<LANES>);
+}
+
+define_mask! {
+    /// A mask equivalent to [SimdI64](crate::SimdI64), where all bits in the lane must be either set
+    /// or unset.
+    struct SimdI64Mask<const LANES: usize>(crate::SimdI64<LANES>);
+}
+
+define_mask! {
+    /// A mask equivalent to [SimdI128](crate::SimdI128), where all bits in the lane must be either set
+    /// or unset.
+    struct SimdI128Mask<const LANES: usize>(crate::SimdI64<LANES>);
+}
+
+define_mask! {
+    /// A mask equivalent to [SimdIsize](crate::SimdIsize), where all bits in the lane must be either set
+    /// or unset.
+    struct SimdIsizeMask<const LANES: usize>(crate::SimdI64<LANES>);
+}
+
+macro_rules! implement_mask_ext {
+    { $($vector:ident => $mask:ident,)* } => {
+        $(
+            impl<const LANES: usize> crate::masks::MaskExt<$mask<LANES>> for crate::$vector<LANES> {
+                #[inline]
+                fn lanes_eq(&self, other: &Self) -> $mask<LANES> {
+                    unsafe { crate::intrinsics::simd_eq(self, other) }
+                }
+
+                #[inline]
+                fn lanes_ne(&self, other: &Self) -> $mask<LANES> {
+                    unsafe { crate::intrinsics::simd_ne(self, other) }
+                }
+
+                #[inline]
+                fn lanes_lt(&self, other: &Self) -> $mask<LANES> {
+                    unsafe { crate::intrinsics::simd_lt(self, other) }
+                }
+
+                #[inline]
+                fn lanes_gt(&self, other: &Self) -> $mask<LANES> {
+                    unsafe { crate::intrinsics::simd_gt(self, other) }
+                }
+
+                #[inline]
+                fn lanes_le(&self, other: &Self) -> $mask<LANES> {
+                    unsafe { crate::intrinsics::simd_le(self, other) }
+                }
+
+                #[inline]
+                fn lanes_ge(&self, other: &Self) -> $mask<LANES> {
+                    unsafe { crate::intrinsics::simd_ge(self, other) }
+                }
+            }
+        )*
+    }
+}
+
+implement_mask_ext! {
+    SimdI8 => SimdI8Mask,
+    SimdI16 => SimdI16Mask,
+    SimdI32 => SimdI32Mask,
+    SimdI64 => SimdI64Mask,
+    SimdI128 => SimdI128Mask,
+    SimdIsize => SimdIsizeMask,
+
+    SimdU8 => SimdI8Mask,
+    SimdU16 => SimdI16Mask,
+    SimdU32 => SimdI32Mask,
+    SimdU64 => SimdI64Mask,
+    SimdU128 => SimdI128Mask,
+    SimdUsize => SimdIsizeMask,
+
+    SimdF32 => SimdI32Mask,
+    SimdF64 => SimdI64Mask,
+}
diff --git a/crates/core_simd/src/masks/wide/vectors_m128.rs b/crates/core_simd/src/masks/full_masks/vectors_m128.rs
similarity index 100%
rename from crates/core_simd/src/masks/wide/vectors_m128.rs
rename to crates/core_simd/src/masks/full_masks/vectors_m128.rs
diff --git a/crates/core_simd/src/masks/wide/vectors_m16.rs b/crates/core_simd/src/masks/full_masks/vectors_m16.rs
similarity index 100%
rename from crates/core_simd/src/masks/wide/vectors_m16.rs
rename to crates/core_simd/src/masks/full_masks/vectors_m16.rs
diff --git a/crates/core_simd/src/masks/wide/vectors_m32.rs b/crates/core_simd/src/masks/full_masks/vectors_m32.rs
similarity index 100%
rename from crates/core_simd/src/masks/wide/vectors_m32.rs
rename to crates/core_simd/src/masks/full_masks/vectors_m32.rs
diff --git a/crates/core_simd/src/masks/wide/vectors_m64.rs b/crates/core_simd/src/masks/full_masks/vectors_m64.rs
similarity index 100%
rename from crates/core_simd/src/masks/wide/vectors_m64.rs
rename to crates/core_simd/src/masks/full_masks/vectors_m64.rs
diff --git a/crates/core_simd/src/masks/wide/vectors_m8.rs b/crates/core_simd/src/masks/full_masks/vectors_m8.rs
similarity index 82%
rename from crates/core_simd/src/masks/wide/vectors_m8.rs
rename to crates/core_simd/src/masks/full_masks/vectors_m8.rs
index 149e138739dc..85506dd93e13 100644
--- a/crates/core_simd/src/masks/wide/vectors_m8.rs
+++ b/crates/core_simd/src/masks/full_masks/vectors_m8.rs
@@ -19,3 +19,6 @@ define_mask_vector! {
     /// Vector of 64 `m8` values
     struct m8x64([i8 as m8; 64]);
 }
+
+#[repr(transparent)]
+struct VectorMask8<const LANES: usize>(crate::SimdI8<LANES>);
diff --git a/crates/core_simd/src/masks/wide/vectors_msize.rs b/crates/core_simd/src/masks/full_masks/vectors_msize.rs
similarity index 100%
rename from crates/core_simd/src/masks/wide/vectors_msize.rs
rename to crates/core_simd/src/masks/full_masks/vectors_msize.rs
diff --git a/crates/core_simd/src/masks/mod.rs b/crates/core_simd/src/masks/mod.rs
index 9fb3da006048..9c90373fb472 100644
--- a/crates/core_simd/src/masks/mod.rs
+++ b/crates/core_simd/src/masks/mod.rs
@@ -1,115 +1,151 @@
 //! Types and traits associated with masking lanes of vectors.
+#![allow(non_camel_case_types)]
 
-pub mod wide;
+macro_rules! delegate_ops_to_inner {
+    { $name:ident } => {
+        impl<const LANES: usize> core::ops::BitAnd for $name<LANES> {
+            type Output = Self;
+            #[inline]
+            fn bitand(self, rhs: Self) -> Self {
+                Self(self.0 & rhs.0)
+            }
+        }
 
-mod ops;
-pub use ops::*;
+        impl<const LANES: usize> core::ops::BitAnd<bool> for $name<LANES> {
+            type Output = Self;
+            #[inline]
+            fn bitand(self, rhs: bool) -> Self {
+                self & Self::splat(rhs)
+            }
+        }
 
-pub(crate) trait MaskImpl {
-    type Mask;
+        impl<const LANES: usize> core::ops::BitAnd<$name<LANES>> for bool {
+            type Output = $name<LANES>;
+            #[inline]
+            fn bitand(self, rhs: $name<LANES>) -> $name<LANES> {
+                $name::<LANES>::splat(self) & rhs
+            }
+        }
+
+        impl<const LANES: usize> core::ops::BitOr for $name<LANES> {
+            type Output = Self;
+            #[inline]
+            fn bitor(self, rhs: Self) -> Self {
+                Self(self.0 | rhs.0)
+            }
+        }
+
+        impl<const LANES: usize> core::ops::BitOr<bool> for $name<LANES> {
+            type Output = Self;
+            #[inline]
+            fn bitor(self, rhs: bool) -> Self {
+                self | Self::splat(rhs)
+            }
+        }
+
+        impl<const LANES: usize> core::ops::BitOr<$name<LANES>> for bool {
+            type Output = $name<LANES>;
+            #[inline]
+            fn bitor(self, rhs: $name<LANES>) -> $name<LANES> {
+                $name::<LANES>::splat(self) | rhs
+            }
+        }
+
+        impl<const LANES: usize> core::ops::BitXor for $name<LANES> {
+            type Output = Self;
+            #[inline]
+            fn bitxor(self, rhs: Self) -> Self::Output {
+                Self(self.0 ^ rhs.0)
+            }
+        }
+
+        impl<const LANES: usize> core::ops::BitXor<bool> for $name<LANES> {
+            type Output = Self;
+            #[inline]
+            fn bitxor(self, rhs: bool) -> Self::Output {
+                self ^ Self::splat(rhs)
+            }
+        }
+
+        impl<const LANES: usize> core::ops::BitXor<$name<LANES>> for bool {
+            type Output = $name<LANES>;
+            #[inline]
+            fn bitxor(self, rhs: $name<LANES>) -> Self::Output {
+                $name::<LANES>::splat(self) ^ rhs
+            }
+        }
+
+        impl<const LANES: usize> core::ops::Not for $name<LANES> {
+            type Output = $name<LANES>;
+            #[inline]
+            fn not(self) -> Self::Output {
+                Self(!self.0)
+            }
+        }
+
+        impl<const LANES: usize> core::ops::BitAndAssign for $name<LANES> {
+            #[inline]
+            fn bitand_assign(&mut self, rhs: Self) {
+                self.0 &= rhs.0;
+            }
+        }
+
+        impl<const LANES: usize> core::ops::BitAndAssign<bool> for $name<LANES> {
+            #[inline]
+            fn bitand_assign(&mut self, rhs: bool) {
+                *self &= Self::splat(rhs);
+            }
+        }
+
+        impl<const LANES: usize> core::ops::BitOrAssign for $name<LANES> {
+            #[inline]
+            fn bitor_assign(&mut self, rhs: Self) {
+                self.0 |= rhs.0;
+            }
+        }
+
+        impl<const LANES: usize> core::ops::BitOrAssign<bool> for $name<LANES> {
+            #[inline]
+            fn bitor_assign(&mut self, rhs: bool) {
+                *self |= Self::splat(rhs);
+            }
+        }
+
+        impl<const LANES: usize> core::ops::BitXorAssign for $name<LANES> {
+            #[inline]
+            fn bitxor_assign(&mut self, rhs: Self) {
+                self.0 ^= rhs.0;
+            }
+        }
+
+        impl<const LANES: usize> core::ops::BitXorAssign<bool> for $name<LANES> {
+            #[inline]
+            fn bitxor_assign(&mut self, rhs: bool) {
+                *self ^= Self::splat(rhs);
+            }
+        }
+    }
 }
 
-impl MaskImpl for [u8; 8] {
-    type Mask = wide::m8x8;
-}
-
-impl MaskImpl for [u8; 16] {
-    type Mask = wide::m8x16;
-}
-
-impl MaskImpl for [u8; 32] {
-    type Mask = wide::m8x32;
-}
-
-impl MaskImpl for [u8; 64] {
-    type Mask = wide::m8x64;
-}
-
-impl MaskImpl for [u16; 4] {
-    type Mask = wide::m16x4;
-}
-
-impl MaskImpl for [u16; 8] {
-    type Mask = wide::m16x8;
-}
-
-impl MaskImpl for [u16; 16] {
-    type Mask = wide::m16x16;
-}
-
-impl MaskImpl for [u16; 32] {
-    type Mask = wide::m16x32;
-}
-
-impl MaskImpl for [u32; 2] {
-    type Mask = wide::m32x2;
-}
-
-impl MaskImpl for [u32; 4] {
-    type Mask = wide::m32x4;
-}
-
-impl MaskImpl for [u32; 8] {
-    type Mask = wide::m32x8;
-}
-
-impl MaskImpl for [u32; 16] {
-    type Mask = wide::m32x16;
-}
-
-impl MaskImpl for [u64; 2] {
-    type Mask = wide::m64x2;
-}
-
-impl MaskImpl for [u64; 4] {
-    type Mask = wide::m64x4;
-}
-
-impl MaskImpl for [u64; 8] {
-    type Mask = wide::m64x8;
-}
-
-impl MaskImpl for [u128; 2] {
-    type Mask = wide::m128x2;
-}
-
-impl MaskImpl for [u128; 4] {
-    type Mask = wide::m128x4;
-}
-
-impl MaskImpl for [usize; 2] {
-    type Mask = wide::msizex2;
-}
-
-impl MaskImpl for [usize; 4] {
-    type Mask = wide::msizex4;
-}
-
-impl MaskImpl for [usize; 8] {
-    type Mask = wide::msizex8;
-}
+pub mod full_masks;
 
 macro_rules! define_opaque_mask {
     {
         $(#[$attr:meta])*
-        struct $name:ident([$width:ty; $lanes:tt]);
+        struct $name:ident<const $lanes:ident: usize>($inner_ty:ty);
     } => {
         $(#[$attr])*
         #[allow(non_camel_case_types)]
-        pub struct $name(<[$width; $lanes] as crate::masks::MaskImpl>::Mask);
+        pub struct $name<const $lanes: usize>($inner_ty);
 
-        impl $name {
-            pub(crate) fn new_from_inner(inner: <[$width; $lanes] as crate::masks::MaskImpl>::Mask) -> Self {
-                Self(inner)
-            }
+        delegate_ops_to_inner! { $name }
 
+        impl<const $lanes: usize> $name<$lanes> {
             /// Construct a mask by setting all lanes to the given value.
             pub fn splat(value: bool) -> Self {
-                Self(<[$width; $lanes] as crate::masks::MaskImpl>::Mask::splat(value.into()))
+                Self(<$inner_ty>::splat(value))
             }
 
-            call_counting_args! { $lanes => define_opaque_mask => new [$width; $lanes] }
-
             /// Tests the value of the specified lane.
             ///
             /// # Panics
@@ -129,312 +165,226 @@ macro_rules! define_opaque_mask {
             }
         }
 
-        impl Copy for $name {}
+        impl<const $lanes: usize> Copy for $name<$lanes> {}
 
-        impl Clone for $name {
+        impl<const $lanes: usize> Clone for $name<$lanes> {
             #[inline]
             fn clone(&self) -> Self {
                 *self
             }
         }
 
-        impl Default for $name {
+        impl<const $lanes: usize> Default for $name<$lanes> {
             #[inline]
             fn default() -> Self {
                 Self::splat(false)
             }
         }
 
-        impl PartialEq for $name {
+        impl<const $lanes: usize> PartialEq for $name<$lanes> {
             #[inline]
             fn eq(&self, other: &Self) -> bool {
                 self.0 == other.0
             }
         }
 
-        impl PartialOrd for $name {
+        impl<const $lanes: usize> PartialOrd for $name<$lanes> {
             #[inline]
             fn partial_cmp(&self, other: &Self) -> Option<core::cmp::Ordering> {
                 self.0.partial_cmp(&other.0)
             }
         }
 
-        impl core::fmt::Debug for $name {
+        impl<const $lanes: usize> core::fmt::Debug for $name<$lanes> {
             fn fmt(&self, f: &mut core::fmt::Formatter) -> core::fmt::Result {
                 core::fmt::Debug::fmt(&self.0, f)
             }
         }
-
-        impl core::ops::BitAnd for $name {
-            type Output = Self;
-            #[inline]
-            fn bitand(self, rhs: Self) -> Self {
-                Self(self.0 & rhs.0)
-            }
-        }
-
-        impl core::ops::BitAnd<bool> for $name {
-            type Output = Self;
-            #[inline]
-            fn bitand(self, rhs: bool) -> Self {
-                self & Self::splat(rhs)
-            }
-        }
-
-        impl core::ops::BitAnd<$name> for bool {
-            type Output = $name;
-            #[inline]
-            fn bitand(self, rhs: $name) -> $name {
-                $name::splat(self) & rhs
-            }
-        }
-
-        impl core::ops::BitOr for $name {
-            type Output = Self;
-            #[inline]
-            fn bitor(self, rhs: Self) -> Self {
-                Self(self.0 | rhs.0)
-            }
-        }
-
-        impl core::ops::BitOr<bool> for $name {
-            type Output = Self;
-            #[inline]
-            fn bitor(self, rhs: bool) -> Self {
-                self | Self::splat(rhs)
-            }
-        }
-
-        impl core::ops::BitOr<$name> for bool {
-            type Output = $name;
-            #[inline]
-            fn bitor(self, rhs: $name) -> $name {
-                $name::splat(self) | rhs
-            }
-        }
-
-        impl core::ops::BitXor for $name {
-            type Output = Self;
-            #[inline]
-            fn bitxor(self, rhs: Self) -> Self::Output {
-                Self(self.0 ^ rhs.0)
-            }
-        }
-
-        impl core::ops::BitXor<bool> for $name {
-            type Output = Self;
-            #[inline]
-            fn bitxor(self, rhs: bool) -> Self::Output {
-                self ^ Self::splat(rhs)
-            }
-        }
-
-        impl core::ops::BitXor<$name> for bool {
-            type Output = $name;
-            #[inline]
-            fn bitxor(self, rhs: $name) -> Self::Output {
-                $name::splat(self) ^ rhs
-            }
-        }
-
-        impl core::ops::Not for $name {
-            type Output = $name;
-            #[inline]
-            fn not(self) -> Self::Output {
-                Self(!self.0)
-            }
-        }
-
-        impl core::ops::BitAndAssign for $name {
-            #[inline]
-            fn bitand_assign(&mut self, rhs: Self) {
-                self.0 &= rhs.0;
-            }
-        }
-
-        impl core::ops::BitAndAssign<bool> for $name {
-            #[inline]
-            fn bitand_assign(&mut self, rhs: bool) {
-                *self &= Self::splat(rhs);
-            }
-        }
-
-        impl core::ops::BitOrAssign for $name {
-            #[inline]
-            fn bitor_assign(&mut self, rhs: Self) {
-                self.0 |= rhs.0;
-            }
-        }
-
-        impl core::ops::BitOrAssign<bool> for $name {
-            #[inline]
-            fn bitor_assign(&mut self, rhs: bool) {
-                *self |= Self::splat(rhs);
-            }
-        }
-
-        impl core::ops::BitXorAssign for $name {
-            #[inline]
-            fn bitxor_assign(&mut self, rhs: Self) {
-                self.0 ^= rhs.0;
-            }
-        }
-
-        impl core::ops::BitXorAssign<bool> for $name {
-            #[inline]
-            fn bitxor_assign(&mut self, rhs: bool) {
-                *self ^= Self::splat(rhs);
-            }
-        }
     };
-    { new [$width:ty; $lanes:tt] $($var:ident)* } => {
-        /// Construct a vector by setting each lane to the given values.
-        #[allow(clippy::too_many_arguments)]
-        #[inline]
-        pub const fn new($($var: bool),*) -> Self {
-            Self(<[$width; $lanes] as crate::masks::MaskImpl>::Mask::new_from_bool($($var),*))
-        }
+}
+
+define_opaque_mask! {
+    /// Mask for vectors with `LANES` 8-bit elements.
+    ///
+    /// The layout of this type is unspecified.
+    struct Mask8<const LANES: usize>(full_masks::SimdI8Mask<LANES>);
+}
+
+define_opaque_mask! {
+    /// Mask for vectors with `LANES` 16-bit elements.
+    ///
+    /// The layout of this type is unspecified.
+    struct Mask16<const LANES: usize>(full_masks::SimdI16Mask<LANES>);
+}
+
+define_opaque_mask! {
+    /// Mask for vectors with `LANES` 32-bit elements.
+    ///
+    /// The layout of this type is unspecified.
+    struct Mask32<const LANES: usize>(full_masks::SimdI32Mask<LANES>);
+}
+
+define_opaque_mask! {
+    /// Mask for vectors with `LANES` 64-bit elements.
+    ///
+    /// The layout of this type is unspecified.
+    struct Mask64<const LANES: usize>(full_masks::SimdI64Mask<LANES>);
+}
+
+define_opaque_mask! {
+    /// Mask for vectors with `LANES` 128-bit elements.
+    ///
+    /// The layout of this type is unspecified.
+    struct Mask128<const LANES: usize>(full_masks::SimdI128Mask<LANES>);
+}
+
+define_opaque_mask! {
+    /// Mask for vectors with `LANES` pointer-width elements.
+    ///
+    /// The layout of this type is unspecified.
+    struct MaskSize<const LANES: usize>(full_masks::SimdIsizeMask<LANES>);
+}
+
+/// Mask-related operations using a particular mask layout.
+pub trait MaskExt<Mask> {
+    /// Test if each lane is equal to the corresponding lane in `other`.
+    fn lanes_eq(&self, other: &Self) -> Mask;
+
+    /// Test if each lane is not equal to the corresponding lane in `other`.
+    fn lanes_ne(&self, other: &Self) -> Mask;
+
+    /// Test if each lane is less than the corresponding lane in `other`.
+    fn lanes_lt(&self, other: &Self) -> Mask;
+
+    /// Test if each lane is greater than the corresponding lane in `other`.
+    fn lanes_gt(&self, other: &Self) -> Mask;
+
+    /// Test if each lane is less than or equal to the corresponding lane in `other`.
+    fn lanes_le(&self, other: &Self) -> Mask;
+
+    /// Test if each lane is greater than or equal to the corresponding lane in `other`.
+    fn lanes_ge(&self, other: &Self) -> Mask;
+}
+
+macro_rules! implement_mask_ops {
+    { $($vector:ident => $mask:ident,)* } => {
+        $(
+            impl<const LANES: usize> crate::$vector<LANES> {
+                /// Test if each lane is equal to the corresponding lane in `other`.
+                #[inline]
+                pub fn lanes_eq(&self, other: &Self) -> $mask<LANES> {
+                    $mask(MaskExt::lanes_eq(self, other))
+                }
+
+                /// Test if each lane is not equal to the corresponding lane in `other`.
+                #[inline]
+                pub fn lanes_ne(&self, other: &Self) -> $mask<LANES> {
+                    $mask(MaskExt::lanes_ne(self, other))
+                }
+
+                /// Test if each lane is less than the corresponding lane in `other`.
+                #[inline]
+                pub fn lanes_lt(&self, other: &Self) -> $mask<LANES> {
+                    $mask(MaskExt::lanes_lt(self, other))
+                }
+
+                /// Test if each lane is greater than the corresponding lane in `other`.
+                #[inline]
+                pub fn lanes_gt(&self, other: &Self) -> $mask<LANES> {
+                    $mask(MaskExt::lanes_gt(self, other))
+                }
+
+                /// Test if each lane is less than or equal to the corresponding lane in `other`.
+                #[inline]
+                pub fn lanes_le(&self, other: &Self) -> $mask<LANES> {
+                    $mask(MaskExt::lanes_le(self, other))
+                }
+
+                /// Test if each lane is greater than or equal to the corresponding lane in `other`.
+                #[inline]
+                pub fn lanes_ge(&self, other: &Self) -> $mask<LANES> {
+                    $mask(MaskExt::lanes_ge(self, other))
+                }
+            }
+        )*
     }
 }
 
-pub(crate) mod opaque {
-    define_opaque_mask! {
-        /// Mask for 8 8-bit lanes.
-        ///
-        /// The layout of this type is unspecified.
-        struct mask8x8([u8; 8]);
-    }
+implement_mask_ops! {
+    SimdI8 => Mask8,
+    SimdI16 => Mask16,
+    SimdI32 => Mask32,
+    SimdI64 => Mask64,
+    SimdI128 => Mask128,
+    SimdIsize => MaskSize,
 
-    define_opaque_mask! {
-        /// Mask for 16 8-bit lanes.
-        ///
-        /// The layout of this type is unspecified.
-        struct mask8x16([u8; 16]);
-    }
+    SimdU8 => Mask8,
+    SimdU16 => Mask16,
+    SimdU32 => Mask32,
+    SimdU64 => Mask64,
+    SimdU128 => Mask128,
+    SimdUsize => MaskSize,
 
-    define_opaque_mask! {
-        /// Mask for 32 8-bit lanes.
-        ///
-        /// The layout of this type is unspecified.
-        struct mask8x32([u8; 32]);
-    }
-
-    define_opaque_mask! {
-        /// Mask for 64 8-bit lanes.
-        ///
-        /// The layout of this type is unspecified.
-        struct mask8x64([u8; 64]);
-    }
-
-    define_opaque_mask! {
-        /// Mask for 4 16-bit lanes.
-        ///
-        /// The layout of this type is unspecified.
-        struct mask16x4([u16; 4]);
-    }
-
-    define_opaque_mask! {
-        /// Mask for 8 16-bit lanes.
-        ///
-        /// The layout of this type is unspecified.
-        struct mask16x8([u16; 8]);
-    }
-
-    define_opaque_mask! {
-        /// Mask for 16 16-bit lanes.
-        ///
-        /// The layout of this type is unspecified.
-        struct mask16x16([u16; 16]);
-    }
-
-    define_opaque_mask! {
-        /// Mask for 32 16-bit lanes.
-        ///
-        /// The layout of this type is unspecified.
-        struct mask16x32([u16; 32]);
-    }
-
-    define_opaque_mask! {
-        /// Mask for 2 32-bit lanes.
-        ///
-        /// The layout of this type is unspecified.
-        struct mask32x2([u32; 2]);
-    }
-
-    define_opaque_mask! {
-        /// Mask for 4 32-bit lanes.
-        ///
-        /// The layout of this type is unspecified.
-        struct mask32x4([u32; 4]);
-    }
-
-    define_opaque_mask! {
-        /// Mask for 8 32-bit lanes.
-        ///
-        /// The layout of this type is unspecified.
-        struct mask32x8([u32; 8]);
-    }
-
-    define_opaque_mask! {
-        /// Mask for 16 32-bit lanes.
-        ///
-        /// The layout of this type is unspecified.
-        struct mask32x16([u32; 16]);
-    }
-
-    define_opaque_mask! {
-        /// Mask for 2 64-bit lanes.
-        ///
-        /// The layout of this type is unspecified.
-        struct mask64x2([u64; 2]);
-    }
-
-    define_opaque_mask! {
-        /// Mask for 4 64-bit lanes.
-        ///
-        /// The layout of this type is unspecified.
-        struct mask64x4([u64; 4]);
-    }
-
-    define_opaque_mask! {
-        /// Mask for 8 64-bit lanes.
-        ///
-        /// The layout of this type is unspecified.
-        struct mask64x8([u64; 8]);
-    }
-
-    define_opaque_mask! {
-        /// Mask for 2 128-bit lanes.
-        ///
-        /// The layout of this type is unspecified.
-        struct mask128x2([u128; 2]);
-    }
-
-    define_opaque_mask! {
-        /// Mask for 4 128-bit lanes.
-        ///
-        /// The layout of this type is unspecified.
-        struct mask128x4([u128; 4]);
-    }
-
-    define_opaque_mask! {
-        /// Mask for 2 `isize`-wide lanes.
-        ///
-        /// The layout of this type is unspecified.
-        struct masksizex2([usize; 2]);
-    }
-
-    define_opaque_mask! {
-        /// Mask for 4 `isize`-wide lanes.
-        ///
-        /// The layout of this type is unspecified.
-        struct masksizex4([usize; 4]);
-    }
-
-    define_opaque_mask! {
-        /// Mask for 8 `isize`-wide lanes.
-        ///
-        /// The layout of this type is unspecified.
-        struct masksizex8([usize; 8]);
-    }
+    SimdF32 => Mask32,
+    SimdF64 => Mask64,
 }
+
+/// Vector of eight 8-bit masks
+pub type mask8x8 = Mask8<8>;
+
+/// Vector of 16 8-bit masks
+pub type mask8x16 = Mask8<16>;
+
+/// Vector of 32 8-bit masks
+pub type mask8x32 = Mask8<32>;
+
+/// Vector of 16 8-bit masks
+pub type mask8x64 = Mask8<64>;
+
+/// Vector of four 16-bit masks
+pub type mask16x4 = Mask16<4>;
+
+/// Vector of eight 16-bit masks
+pub type mask16x8 = Mask16<8>;
+
+/// Vector of 16 16-bit masks
+pub type mask16x16 = Mask16<16>;
+
+/// Vector of 32 16-bit masks
+pub type mask16x32 = Mask32<32>;
+
+/// Vector of two 32-bit masks
+pub type mask32x2 = Mask32<2>;
+
+/// Vector of four 32-bit masks
+pub type mask32x4 = Mask32<4>;
+
+/// Vector of eight 32-bit masks
+pub type mask32x8 = Mask32<8>;
+
+/// Vector of 16 32-bit masks
+pub type mask32x16 = Mask32<16>;
+
+/// Vector of two 64-bit masks
+pub type mask64x2 = Mask64<2>;
+
+/// Vector of four 64-bit masks
+pub type mask64x4 = Mask64<4>;
+
+/// Vector of eight 64-bit masks
+pub type mask64x8 = Mask64<8>;
+
+/// Vector of two 128-bit masks
+pub type mask128x2 = Mask128<2>;
+
+/// Vector of four 128-bit masks
+pub type mask128x4 = Mask128<4>;
+
+/// Vector of two pointer-width masks
+pub type masksizex2 = MaskSize<2>;
+
+/// Vector of four pointer-width masks
+pub type masksizex4 = MaskSize<4>;
+
+/// Vector of eight pointer-width masks
+pub type masksizex8 = MaskSize<8>;
diff --git a/crates/core_simd/src/masks/ops.rs b/crates/core_simd/src/masks/ops.rs
deleted file mode 100644
index 85ce955459a2..000000000000
--- a/crates/core_simd/src/masks/ops.rs
+++ /dev/null
@@ -1,208 +0,0 @@
-/// Mask-related operations using a particular mask layout.
-pub trait MaskExt<Mask> {
-    /// Test if each lane is equal to the corresponding lane in `other`.
-    fn lanes_eq(self, other: Self) -> Mask;
-
-    /// Test if each lane is not equal to the corresponding lane in `other`.
-    fn lanes_ne(self, other: Self) -> Mask;
-
-    /// Test if each lane is less than the corresponding lane in `other`.
-    fn lanes_lt(self, other: Self) -> Mask;
-
-    /// Test if each lane is greater than the corresponding lane in `other`.
-    fn lanes_gt(self, other: Self) -> Mask;
-
-    /// Test if each lane is less than or equal to the corresponding lane in `other`.
-    fn lanes_le(self, other: Self) -> Mask;
-
-    /// Test if each lane is greater than or equal to the corresponding lane in `other`.
-    fn lanes_ge(self, other: Self) -> Mask;
-}
-
-macro_rules! implement_mask_ext {
-    { $($vector:ty => $($mask:ty),*;)* } => {
-        $( // vector
-            $( // mask
-                impl MaskExt<$mask> for $vector {
-                    #[inline]
-                    fn lanes_eq(self, other: Self) -> $mask {
-                        unsafe { crate::intrinsics::simd_eq(self, other) }
-                    }
-
-                    #[inline]
-                    fn lanes_ne(self, other: Self) -> $mask {
-                        unsafe { crate::intrinsics::simd_ne(self, other) }
-                    }
-
-                    #[inline]
-                    fn lanes_lt(self, other: Self) -> $mask {
-                        unsafe { crate::intrinsics::simd_lt(self, other) }
-                    }
-
-                    #[inline]
-                    fn lanes_gt(self, other: Self) -> $mask {
-                        unsafe { crate::intrinsics::simd_gt(self, other) }
-                    }
-
-                    #[inline]
-                    fn lanes_le(self, other: Self) -> $mask {
-                        unsafe { crate::intrinsics::simd_le(self, other) }
-                    }
-
-                    #[inline]
-                    fn lanes_ge(self, other: Self) -> $mask {
-                        unsafe { crate::intrinsics::simd_ge(self, other) }
-                    }
-                }
-            )*
-        )*
-    }
-}
-
-implement_mask_ext! {
-    crate::u8x8 => crate::masks::wide::m8x8;
-    crate::u8x16 => crate::masks::wide::m8x16;
-    crate::u8x32 => crate::masks::wide::m8x32;
-    crate::u8x64 => crate::masks::wide::m8x64;
-    crate::u16x4 => crate::masks::wide::m16x4;
-    crate::u16x8 => crate::masks::wide::m16x8;
-    crate::u16x16 => crate::masks::wide::m16x16;
-    crate::u16x32 => crate::masks::wide::m16x32;
-    crate::u32x2 => crate::masks::wide::m32x2;
-    crate::u32x4 => crate::masks::wide::m32x4;
-    crate::u32x8 => crate::masks::wide::m32x8;
-    crate::u32x16 => crate::masks::wide::m32x16;
-    crate::u64x2 => crate::masks::wide::m64x2;
-    crate::u64x4 => crate::masks::wide::m64x4;
-    crate::u64x8 => crate::masks::wide::m64x8;
-    crate::u128x2 => crate::masks::wide::m128x2;
-    crate::u128x4 => crate::masks::wide::m128x4;
-    crate::usizex2 => crate::masks::wide::msizex2;
-    crate::usizex4 => crate::masks::wide::msizex4;
-    crate::usizex8 => crate::masks::wide::msizex8;
-
-    crate::i8x8 => crate::masks::wide::m8x8;
-    crate::i8x16 => crate::masks::wide::m8x16;
-    crate::i8x32 => crate::masks::wide::m8x32;
-    crate::i8x64 => crate::masks::wide::m8x64;
-    crate::i16x4 => crate::masks::wide::m16x4;
-    crate::i16x8 => crate::masks::wide::m16x8;
-    crate::i16x16 => crate::masks::wide::m16x16;
-    crate::i16x32 => crate::masks::wide::m16x32;
-    crate::i32x2 => crate::masks::wide::m32x2;
-    crate::i32x4 => crate::masks::wide::m32x4;
-    crate::i32x8 => crate::masks::wide::m32x8;
-    crate::i32x16 => crate::masks::wide::m32x16;
-    crate::i64x2 => crate::masks::wide::m64x2;
-    crate::i64x4 => crate::masks::wide::m64x4;
-    crate::i64x8 => crate::masks::wide::m64x8;
-    crate::i128x2 => crate::masks::wide::m128x2;
-    crate::i128x4 => crate::masks::wide::m128x4;
-    crate::isizex2 => crate::masks::wide::msizex2;
-    crate::isizex4 => crate::masks::wide::msizex4;
-    crate::isizex8 => crate::masks::wide::msizex8;
-
-    crate::f32x2 => crate::masks::wide::m32x2;
-    crate::f32x4 => crate::masks::wide::m32x4;
-    crate::f32x8 => crate::masks::wide::m32x8;
-    crate::f32x16 => crate::masks::wide::m32x16;
-    crate::f64x2 => crate::masks::wide::m64x2;
-    crate::f64x4 => crate::masks::wide::m64x4;
-    crate::f64x8 => crate::masks::wide::m64x8;
-}
-
-macro_rules! implement_mask_ops {
-    { $($vector:ty => $mask:ty,)* } => {
-        $( // vector
-            impl $vector {
-                /// Test if each lane is equal to the corresponding lane in `other`.
-                #[inline]
-                pub fn lanes_eq(self, other: Self) -> $mask {
-                    <$mask>::new_from_inner(MaskExt::lanes_eq(self, other))
-                }
-
-                /// Test if each lane is not equal to the corresponding lane in `other`.
-                #[inline]
-                pub fn lanes_ne(self, other: Self) -> $mask {
-                    <$mask>::new_from_inner(MaskExt::lanes_ne(self, other))
-                }
-
-                /// Test if each lane is less than the corresponding lane in `other`.
-                #[inline]
-                pub fn lanes_lt(self, other: Self) -> $mask {
-                    <$mask>::new_from_inner(MaskExt::lanes_lt(self, other))
-                }
-
-                /// Test if each lane is greater than the corresponding lane in `other`.
-                #[inline]
-                pub fn lanes_gt(self, other: Self) -> $mask {
-                    <$mask>::new_from_inner(MaskExt::lanes_gt(self, other))
-                }
-
-                /// Test if each lane is less than or equal to the corresponding lane in `other`.
-                #[inline]
-                pub fn lanes_le(self, other: Self) -> $mask {
-                    <$mask>::new_from_inner(MaskExt::lanes_le(self, other))
-                }
-
-                /// Test if each lane is greater than or equal to the corresponding lane in `other`.
-                #[inline]
-                pub fn lanes_ge(self, other: Self) -> $mask {
-                    <$mask>::new_from_inner(MaskExt::lanes_ge(self, other))
-                }
-            }
-        )*
-    }
-}
-
-implement_mask_ops! {
-    crate::u8x8 => crate::mask8x8,
-    crate::u8x16 => crate::mask8x16,
-    crate::u8x32 => crate::mask8x32,
-    crate::u8x64 => crate::mask8x64,
-    crate::u16x4 => crate::mask16x4,
-    crate::u16x8 => crate::mask16x8,
-    crate::u16x16 => crate::mask16x16,
-    crate::u16x32 => crate::mask16x32,
-    crate::u32x2 => crate::mask32x2,
-    crate::u32x4 => crate::mask32x4,
-    crate::u32x8 => crate::mask32x8,
-    crate::u32x16 => crate::mask32x16,
-    crate::u64x2 => crate::mask64x2,
-    crate::u64x4 => crate::mask64x4,
-    crate::u64x8 => crate::mask64x8,
-    crate::u128x2 => crate::mask128x2,
-    crate::u128x4 => crate::mask128x4,
-    crate::usizex2 => crate::masksizex2,
-    crate::usizex4 => crate::masksizex4,
-    crate::usizex8 => crate::masksizex8,
-
-    crate::i8x8 => crate::mask8x8,
-    crate::i8x16 => crate::mask8x16,
-    crate::i8x32 => crate::mask8x32,
-    crate::i8x64 => crate::mask8x64,
-    crate::i16x4 => crate::mask16x4,
-    crate::i16x8 => crate::mask16x8,
-    crate::i16x16 => crate::mask16x16,
-    crate::i16x32 => crate::mask16x32,
-    crate::i32x2 => crate::mask32x2,
-    crate::i32x4 => crate::mask32x4,
-    crate::i32x8 => crate::mask32x8,
-    crate::i32x16 => crate::mask32x16,
-    crate::i64x2 => crate::mask64x2,
-    crate::i64x4 => crate::mask64x4,
-    crate::i64x8 => crate::mask64x8,
-    crate::i128x2 => crate::mask128x2,
-    crate::i128x4 => crate::mask128x4,
-    crate::isizex2 => crate::masksizex2,
-    crate::isizex4 => crate::masksizex4,
-    crate::isizex8 => crate::masksizex8,
-
-    crate::f32x2 => crate::mask32x2,
-    crate::f32x4 => crate::mask32x4,
-    crate::f32x8 => crate::mask32x8,
-    crate::f32x16 => crate::mask32x16,
-    crate::f64x2 => crate::mask64x2,
-    crate::f64x4 => crate::mask64x4,
-    crate::f64x8 => crate::mask64x8,
-}
diff --git a/crates/core_simd/src/masks/wide/mod.rs b/crates/core_simd/src/masks/wide/mod.rs
deleted file mode 100644
index 7df8ca7e53d5..000000000000
--- a/crates/core_simd/src/masks/wide/mod.rs
+++ /dev/null
@@ -1,139 +0,0 @@
-//! Masks that take up full vector registers.
-
-mod vectors_m8;
-pub use vectors_m8::*;
-mod vectors_m16;
-pub use vectors_m16::*;
-mod vectors_m32;
-pub use vectors_m32::*;
-mod vectors_m64;
-pub use vectors_m64::*;
-mod vectors_m128;
-pub use vectors_m128::*;
-mod vectors_msize;
-pub use vectors_msize::*;
-
-/// The error type returned when converting an integer to a mask fails.
-#[derive(Debug, Copy, Clone, PartialEq, Eq)]
-pub struct TryFromMaskError(());
-
-impl core::fmt::Display for TryFromMaskError {
-    fn fmt(&self, f: &mut core::fmt::Formatter) -> core::fmt::Result {
-        write!(f, "mask must have all bits set or unset")
-    }
-}
-
-macro_rules! define_mask {
-    { $(#[$attr:meta])* struct $name:ident($type:ty); } => {
-        $(#[$attr])*
-        #[allow(non_camel_case_types)]
-        #[derive(Copy, Clone, Default, PartialEq, PartialOrd, Eq, Ord, Hash)]
-        #[repr(transparent)]
-        pub struct $name(pub(crate) $type);
-
-        impl $name {
-            /// Construct a mask from the given value.
-            pub const fn new(value: bool) -> Self {
-                if value {
-                    Self(!0)
-                } else {
-                    Self(0)
-                }
-            }
-
-            /// Test if the mask is set.
-            pub const fn test(&self) -> bool {
-                self.0 != 0
-            }
-        }
-
-        impl core::convert::From<bool> for $name {
-            fn from(value: bool) -> Self {
-                Self::new(value)
-            }
-        }
-
-        impl core::convert::From<$name> for bool {
-            fn from(mask: $name) -> Self {
-                mask.test()
-            }
-        }
-
-        impl core::convert::TryFrom<$type> for $name {
-            type Error = TryFromMaskError;
-            fn try_from(value: $type) -> Result<Self, Self::Error> {
-                if value == 0 || !value == 0 {
-                    Ok(Self(value))
-                } else {
-                    Err(TryFromMaskError(()))
-                }
-            }
-        }
-
-        impl core::convert::From<$name> for $type {
-            fn from(value: $name) -> Self {
-                value.0
-            }
-        }
-
-        impl core::fmt::Debug for $name {
-            fn fmt(&self, f: &mut core::fmt::Formatter) -> core::fmt::Result {
-                self.test().fmt(f)
-            }
-        }
-
-        impl core::fmt::Binary for $name {
-            fn fmt(&self, f: &mut core::fmt::Formatter) -> core::fmt::Result {
-                core::fmt::Binary::fmt(&self.0, f)
-            }
-        }
-
-        impl core::fmt::Octal for $name {
-            fn fmt(&self, f: &mut core::fmt::Formatter) -> core::fmt::Result {
-                core::fmt::Octal::fmt(&self.0, f)
-            }
-        }
-
-        impl core::fmt::LowerHex for $name {
-            fn fmt(&self, f: &mut core::fmt::Formatter) -> core::fmt::Result {
-                core::fmt::LowerHex::fmt(&self.0, f)
-            }
-        }
-
-        impl core::fmt::UpperHex for $name {
-            fn fmt(&self, f: &mut core::fmt::Formatter) -> core::fmt::Result {
-                core::fmt::UpperHex::fmt(&self.0, f)
-            }
-        }
-    }
-}
-
-define_mask! {
-    /// 8-bit mask
-    struct m8(i8);
-}
-
-define_mask! {
-    /// 16-bit mask
-    struct m16(i16);
-}
-
-define_mask! {
-    /// 32-bit mask
-    struct m32(i32);
-}
-
-define_mask! {
-    /// 64-bit mask
-    struct m64(i64);
-}
-
-define_mask! {
-    /// 128-bit mask
-    struct m128(i128);
-}
-
-define_mask! {
-    /// `isize`-wide mask
-    struct msize(isize);
-}
diff --git a/crates/core_simd/src/vectors_f32.rs b/crates/core_simd/src/vectors_f32.rs
index b1e13408cc95..0b5d8c6ec49b 100644
--- a/crates/core_simd/src/vectors_f32.rs
+++ b/crates/core_simd/src/vectors_f32.rs
@@ -1,14 +1,21 @@
 #![allow(non_camel_case_types)]
 
-/// A SIMD vector of containing `LANES` lanes of `f32`.
+/// A SIMD vector of containing `LANES` `f32` values.
 #[repr(simd)]
 pub struct SimdF32<const LANES: usize>([f32; LANES]);
 
 impl_float_vector! { SimdF32, f32, SimdU32 }
 
+/// Vector of two `f32` values
 pub type f32x2 = SimdF32<2>;
+
+/// Vector of four `f32` values
 pub type f32x4 = SimdF32<4>;
+
+/// Vector of eight `f32` values
 pub type f32x8 = SimdF32<8>;
+
+/// Vector of 16 `f32` values
 pub type f32x16 = SimdF32<16>;
 
 from_transmute_x86! { unsafe f32x4 => __m128 }
diff --git a/crates/core_simd/src/vectors_f64.rs b/crates/core_simd/src/vectors_f64.rs
index 4297c9d636c3..307f8a4acacd 100644
--- a/crates/core_simd/src/vectors_f64.rs
+++ b/crates/core_simd/src/vectors_f64.rs
@@ -1,13 +1,18 @@
 #![allow(non_camel_case_types)]
 
-/// A SIMD vector of containing `LANES` lanes of `f64`.
+/// A SIMD vector of containing `LANES` `f64` values.
 #[repr(simd)]
 pub struct SimdF64<const LANES: usize>([f64; LANES]);
 
 impl_float_vector! { SimdF64, f64, SimdU64 }
 
+/// Vector of two `f64` values
 pub type f64x2 = SimdF64<2>;
+
+/// Vector of four `f64` values
 pub type f64x4 = SimdF64<4>;
+
+/// Vector of eight `f64` values
 pub type f64x8 = SimdF64<8>;
 
 from_transmute_x86! { unsafe f64x2 => __m128d }
diff --git a/crates/core_simd/src/vectors_i128.rs b/crates/core_simd/src/vectors_i128.rs
index a48c823cbd66..16e6162be552 100644
--- a/crates/core_simd/src/vectors_i128.rs
+++ b/crates/core_simd/src/vectors_i128.rs
@@ -1,12 +1,15 @@
 #![allow(non_camel_case_types)]
 
-/// A SIMD vector of containing `LANES` lanes of `i128`.
+/// A SIMD vector of containing `LANES` `i128` values.
 #[repr(simd)]
 pub struct SimdI128<const LANES: usize>([i128; LANES]);
 
 impl_integer_vector! { SimdI128, i128 }
 
+/// Vector of two `i128` values
 pub type i128x2 = SimdI128<2>;
+
+/// Vector of four `i128` values
 pub type i128x4 = SimdI128<4>;
 
 from_transmute_x86! { unsafe i128x2 => __m256i }
diff --git a/crates/core_simd/src/vectors_i16.rs b/crates/core_simd/src/vectors_i16.rs
index 7bc522287a3b..08cc4af2a5ea 100644
--- a/crates/core_simd/src/vectors_i16.rs
+++ b/crates/core_simd/src/vectors_i16.rs
@@ -1,14 +1,21 @@
 #![allow(non_camel_case_types)]
 
-/// A SIMD vector of containing `LANES` lanes of `i16`.
+/// A SIMD vector of containing `LANES` `i16` values.
 #[repr(simd)]
 pub struct SimdI16<const LANES: usize>([i16; LANES]);
 
 impl_integer_vector! { SimdI16, i16 }
 
+/// Vector of four `i16` values
 pub type i16x4 = SimdI16<4>;
+
+/// Vector of eight `i16` values
 pub type i16x8 = SimdI16<8>;
+
+/// Vector of 16 `i16` values
 pub type i16x16 = SimdI16<16>;
+
+/// Vector of 32 `i16` values
 pub type i16x32 = SimdI16<32>;
 
 from_transmute_x86! { unsafe i16x8 => __m128i }
diff --git a/crates/core_simd/src/vectors_i32.rs b/crates/core_simd/src/vectors_i32.rs
index 05533bb0b6d7..116f2abaeeed 100644
--- a/crates/core_simd/src/vectors_i32.rs
+++ b/crates/core_simd/src/vectors_i32.rs
@@ -1,14 +1,21 @@
 #![allow(non_camel_case_types)]
 
-/// A SIMD vector of containing `LANES` lanes of `i32`.
+/// A SIMD vector of containing `LANES` `i32` values.
 #[repr(simd)]
 pub struct SimdI32<const LANES: usize>([i32; LANES]);
 
 impl_integer_vector! { SimdI32, i32 }
 
+/// Vector of two `i32` values
 pub type i32x2 = SimdI32<2>;
+
+/// Vector of four `i32` values
 pub type i32x4 = SimdI32<4>;
+
+/// Vector of eight `i32` values
 pub type i32x8 = SimdI32<8>;
+
+/// Vector of 16 `i32` values
 pub type i32x16 = SimdI32<16>;
 
 from_transmute_x86! { unsafe i32x4 => __m128i }
diff --git a/crates/core_simd/src/vectors_i64.rs b/crates/core_simd/src/vectors_i64.rs
index e669e8a367c0..6a1e2094179b 100644
--- a/crates/core_simd/src/vectors_i64.rs
+++ b/crates/core_simd/src/vectors_i64.rs
@@ -1,13 +1,18 @@
 #![allow(non_camel_case_types)]
 
-/// A SIMD vector of containing `LANES` lanes of `i64`.
+/// A SIMD vector of containing `LANES` `i64` values.
 #[repr(simd)]
 pub struct SimdI64<const LANES: usize>([i64; LANES]);
 
 impl_integer_vector! { SimdI64, i64 }
 
+/// Vector of two `i64` values
 pub type i64x2 = SimdI64<2>;
+
+/// Vector of four `i64` values
 pub type i64x4 = SimdI64<4>;
+
+/// Vector of eight `i64` values
 pub type i64x8 = SimdI64<8>;
 
 from_transmute_x86! { unsafe i64x2 => __m128i }
diff --git a/crates/core_simd/src/vectors_i8.rs b/crates/core_simd/src/vectors_i8.rs
index 55a440cc92fa..0ac5ba9efee5 100644
--- a/crates/core_simd/src/vectors_i8.rs
+++ b/crates/core_simd/src/vectors_i8.rs
@@ -1,14 +1,21 @@
 #![allow(non_camel_case_types)]
 
-/// A SIMD vector of containing `LANES` lanes of `i8`.
+/// A SIMD vector of containing `LANES` `i8` values.
 #[repr(simd)]
 pub struct SimdI8<const LANES: usize>([i8; LANES]);
 
 impl_integer_vector! { SimdI8, i8 }
 
+/// Vector of eight `i8` values
 pub type i8x8 = SimdI8<8>;
+
+/// Vector of 16 `i8` values
 pub type i8x16 = SimdI8<16>;
+
+/// Vector of 32 `i8` values
 pub type i8x32 = SimdI8<32>;
+
+/// Vector of 64 `i8` values
 pub type i8x64 = SimdI8<64>;
 
 from_transmute_x86! { unsafe i8x16 => __m128i }
diff --git a/crates/core_simd/src/vectors_isize.rs b/crates/core_simd/src/vectors_isize.rs
index 6d9b20615326..6856f305092d 100644
--- a/crates/core_simd/src/vectors_isize.rs
+++ b/crates/core_simd/src/vectors_isize.rs
@@ -1,13 +1,18 @@
 #![allow(non_camel_case_types)]
 
-/// A SIMD vector of containing `LANES` lanes of `isize`.
+/// A SIMD vector of containing `LANES` `isize` values.
 #[repr(simd)]
 pub struct SimdIsize<const LANES: usize>([isize; LANES]);
 
 impl_integer_vector! { SimdIsize, isize }
 
+/// Vector of two `isize` values
 pub type isizex2 = SimdIsize<2>;
+
+/// Vector of four `isize` values
 pub type isizex4 = SimdIsize<4>;
+
+/// Vector of eight `isize` values
 pub type isizex8 = SimdIsize<8>;
 
 #[cfg(target_pointer_width = "32")]
diff --git a/crates/core_simd/src/vectors_u128.rs b/crates/core_simd/src/vectors_u128.rs
index 54ad6e191f73..522404f133e5 100644
--- a/crates/core_simd/src/vectors_u128.rs
+++ b/crates/core_simd/src/vectors_u128.rs
@@ -1,12 +1,15 @@
 #![allow(non_camel_case_types)]
 
-/// A SIMD vector of containing `LANES` lanes of `u128`.
+/// A SIMD vector of containing `LANES` `u128` values.
 #[repr(simd)]
 pub struct SimdU128<const LANES: usize>([u128; LANES]);
 
 impl_integer_vector! { SimdU128, u128 }
 
+/// Vector of two `u128` values
 pub type u128x2 = SimdU128<2>;
+
+/// Vector of four `u128` values
 pub type u128x4 = SimdU128<4>;
 
 from_transmute_x86! { unsafe u128x2 => __m256i }
diff --git a/crates/core_simd/src/vectors_u16.rs b/crates/core_simd/src/vectors_u16.rs
index 7b0e345ef159..efe7dea58dc4 100644
--- a/crates/core_simd/src/vectors_u16.rs
+++ b/crates/core_simd/src/vectors_u16.rs
@@ -1,14 +1,21 @@
 #![allow(non_camel_case_types)]
 
-/// A SIMD vector of containing `LANES` lanes of `u16`.
+/// A SIMD vector of containing `LANES` `u16` values.
 #[repr(simd)]
 pub struct SimdU16<const LANES: usize>([u16; LANES]);
 
 impl_integer_vector! { SimdU16, u16 }
 
+/// Vector of four `u16` values
 pub type u16x4 = SimdU16<4>;
+
+/// Vector of eight `u16` values
 pub type u16x8 = SimdU16<8>;
+
+/// Vector of 16 `u16` values
 pub type u16x16 = SimdU16<16>;
+
+/// Vector of 32 `u16` values
 pub type u16x32 = SimdU16<32>;
 
 from_transmute_x86! { unsafe u16x8 => __m128i }
diff --git a/crates/core_simd/src/vectors_u32.rs b/crates/core_simd/src/vectors_u32.rs
index f80efbc59ebb..a6cef5baeb73 100644
--- a/crates/core_simd/src/vectors_u32.rs
+++ b/crates/core_simd/src/vectors_u32.rs
@@ -1,14 +1,21 @@
 #![allow(non_camel_case_types)]
 
-/// A SIMD vector of containing `LANES` lanes of `u32`.
+/// A SIMD vector of containing `LANES` `u32` values.
 #[repr(simd)]
 pub struct SimdU32<const LANES: usize>([u32; LANES]);
 
 impl_integer_vector! { SimdU32, u32 }
 
+/// Vector of two `u32` values
 pub type u32x2 = SimdU32<2>;
+
+/// Vector of four `u32` values
 pub type u32x4 = SimdU32<4>;
+
+/// Vector of eight `u32` values
 pub type u32x8 = SimdU32<8>;
+
+/// Vector of 16 `u32` values
 pub type u32x16 = SimdU32<16>;
 
 from_transmute_x86! { unsafe u32x4 => __m128i }
diff --git a/crates/core_simd/src/vectors_u64.rs b/crates/core_simd/src/vectors_u64.rs
index 848d90faaa73..3982e30f5705 100644
--- a/crates/core_simd/src/vectors_u64.rs
+++ b/crates/core_simd/src/vectors_u64.rs
@@ -1,13 +1,18 @@
 #![allow(non_camel_case_types)]
 
-/// A SIMD vector of containing `LANES` lanes of `u64`.
+/// A SIMD vector of containing `LANES` `u64` values.
 #[repr(simd)]
 pub struct SimdU64<const LANES: usize>([u64; LANES]);
 
 impl_integer_vector! { SimdU64, u64 }
 
+/// Vector of two `u64` values
 pub type u64x2 = SimdU64<2>;
+
+/// Vector of four `u64` values
 pub type u64x4 = SimdU64<4>;
+
+/// Vector of eight `u64` values
 pub type u64x8 = SimdU64<8>;
 
 from_transmute_x86! { unsafe u64x2 => __m128i }
diff --git a/crates/core_simd/src/vectors_u8.rs b/crates/core_simd/src/vectors_u8.rs
index b172801aa993..9cc4eaca47ad 100644
--- a/crates/core_simd/src/vectors_u8.rs
+++ b/crates/core_simd/src/vectors_u8.rs
@@ -1,14 +1,21 @@
 #![allow(non_camel_case_types)]
 
-/// A SIMD vector of containing `LANES` lanes of `u8`.
+/// A SIMD vector of containing `LANES` `u8` values.
 #[repr(simd)]
 pub struct SimdU8<const LANES: usize>([u8; LANES]);
 
 impl_integer_vector! { SimdU8, u8 }
 
+/// Vector of eight `u8` values
 pub type u8x8 = SimdU8<8>;
+
+/// Vector of 16 `u8` values
 pub type u8x16 = SimdU8<16>;
+
+/// Vector of 32 `u8` values
 pub type u8x32 = SimdU8<32>;
+
+/// Vector of 64 `u8` values
 pub type u8x64 = SimdU8<64>;
 
 from_transmute_x86! { unsafe u8x16 => __m128i }
diff --git a/crates/core_simd/src/vectors_usize.rs b/crates/core_simd/src/vectors_usize.rs
index b0655ab311b1..c882898f9fbe 100644
--- a/crates/core_simd/src/vectors_usize.rs
+++ b/crates/core_simd/src/vectors_usize.rs
@@ -1,13 +1,18 @@
 #![allow(non_camel_case_types)]
 
-/// A SIMD vector of containing `LANES` lanes of `usize`.
+/// A SIMD vector of containing `LANES` `usize` values.
 #[repr(simd)]
 pub struct SimdUsize<const LANES: usize>([usize; LANES]);
 
 impl_integer_vector! { SimdUsize, usize }
 
+/// Vector of two `usize` values
 pub type usizex2 = SimdUsize<2>;
+
+/// Vector of four `usize` values
 pub type usizex4 = SimdUsize<4>;
+
+/// Vector of eight `usize` values
 pub type usizex8 = SimdUsize<8>;
 
 #[cfg(target_pointer_width = "32")]
diff --git a/crates/core_simd/tests/ops_impl/mask_macros.rs b/crates/core_simd/tests/ops_impl/mask_macros.rs
index 3aaa036b9940..795f9e27c445 100644
--- a/crates/core_simd/tests/ops_impl/mask_macros.rs
+++ b/crates/core_simd/tests/ops_impl/mask_macros.rs
@@ -1,6 +1,5 @@
 macro_rules! mask_tests {
     { $vector:ident, $lanes:literal } => {
-        /*
         #[cfg(test)]
         mod $vector {
             use core_simd::$vector as Vector;
@@ -222,6 +221,5 @@ macro_rules! mask_tests {
                 assert_eq!(!v, expected);
             }
         }
-        */
     }
 }

From 0ddf7acc89d414d12c4fc04c90cf208c78fd8d5e Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Mon, 14 Dec 2020 00:07:36 -0500
Subject: [PATCH 063/249] Reenable rounding ops

---
 crates/core_simd/src/lib.rs                   |   2 +-
 crates/core_simd/src/round.rs                 | 111 ++++++------------
 .../core_simd/tests/ops_impl/float_macros.rs  |   3 -
 3 files changed, 35 insertions(+), 81 deletions(-)

diff --git a/crates/core_simd/src/lib.rs b/crates/core_simd/src/lib.rs
index 9d4ce683f22a..312a3237e232 100644
--- a/crates/core_simd/src/lib.rs
+++ b/crates/core_simd/src/lib.rs
@@ -9,7 +9,7 @@ mod macros;
 mod fmt;
 mod intrinsics;
 mod ops;
-//mod round;
+mod round;
 
 mod masks;
 pub use masks::*;
diff --git a/crates/core_simd/src/round.rs b/crates/core_simd/src/round.rs
index 0529bbe0080c..d77bc4e8fa7e 100644
--- a/crates/core_simd/src/round.rs
+++ b/crates/core_simd/src/round.rs
@@ -1,88 +1,45 @@
 macro_rules! implement {
     {
-        impl $type:ident {
-            int_type = $int_type:ident
-        }
+        $type:ident, $int_type:ident
     } => {
-        mod $type {
-            impl crate::$type {
-                /// Returns the largest integer less than or equal to each lane.
-                #[must_use = "method returns a new vector and does not mutate the original value"]
-                #[inline]
-                pub fn floor(self) -> Self {
-                    unsafe { crate::intrinsics::simd_floor(self) }
-                }
+        impl<const LANES: usize> crate::$type<LANES> {
+            /// Returns the largest integer less than or equal to each lane.
+            #[must_use = "method returns a new vector and does not mutate the original value"]
+            #[inline]
+            pub fn floor(self) -> Self {
+                unsafe { crate::intrinsics::simd_floor(self) }
+            }
 
-                /// Returns the smallest integer greater than or equal to each lane.
-                #[must_use = "method returns a new vector and does not mutate the original value"]
-                #[inline]
-                pub fn ceil(self) -> Self {
-                    unsafe { crate::intrinsics::simd_ceil(self) }
-                }
+            /// Returns the smallest integer greater than or equal to each lane.
+            #[must_use = "method returns a new vector and does not mutate the original value"]
+            #[inline]
+            pub fn ceil(self) -> Self {
+                unsafe { crate::intrinsics::simd_ceil(self) }
+            }
 
-                /// Rounds toward zero and converts to the same-width integer type, assuming that
-                /// the value is finite and fits in that type.
-                ///
-                /// # Safety
-                /// The value must:
-                ///
-                /// * Not be NaN
-                /// * Not be infinite
-                /// * Be representable in the return type, after truncating off its fractional part
-                #[inline]
-                pub unsafe fn to_int_unchecked(self) -> crate::$int_type {
-                    crate::intrinsics::simd_cast(self)
-                }
+            /// Rounds toward zero and converts to the same-width integer type, assuming that
+            /// the value is finite and fits in that type.
+            ///
+            /// # Safety
+            /// The value must:
+            ///
+            /// * Not be NaN
+            /// * Not be infinite
+            /// * Be representable in the return type, after truncating off its fractional part
+            #[inline]
+            pub unsafe fn to_int_unchecked(self) -> crate::$int_type<LANES> {
+                crate::intrinsics::simd_cast(self)
+            }
 
-                /// Creates a floating-point vector from an integer vector.  Rounds values that are
-                /// not exactly representable.
-                #[inline]
-                pub fn round_from_int(value: crate::$int_type) -> Self {
-                    unsafe { crate::intrinsics::simd_cast(value) }
-                }
+            /// Creates a floating-point vector from an integer vector.  Rounds values that are
+            /// not exactly representable.
+            #[inline]
+            pub fn round_from_int(value: crate::$int_type<LANES>) -> Self {
+                unsafe { crate::intrinsics::simd_cast(value) }
             }
         }
     }
 }
 
-implement! {
-    impl f32x2 {
-        int_type = i32x2
-    }
-}
-
-implement! {
-    impl f32x4 {
-        int_type = i32x4
-    }
-}
-
-implement! {
-    impl f32x8 {
-        int_type = i32x8
-    }
-}
-
-implement! {
-    impl f32x16 {
-        int_type = i32x16
-    }
-}
-
-implement! {
-    impl f64x2 {
-        int_type = i64x2
-    }
-}
-
-implement! {
-    impl f64x4 {
-        int_type = i64x4
-    }
-}
-
-implement! {
-    impl f64x8 {
-        int_type = i64x8
-    }
-}
+implement! { SimdF32, SimdI32 }
+implement! { SimdF64, SimdI64 }
diff --git a/crates/core_simd/tests/ops_impl/float_macros.rs b/crates/core_simd/tests/ops_impl/float_macros.rs
index a46367d0cc2f..fe347a5362da 100644
--- a/crates/core_simd/tests/ops_impl/float_macros.rs
+++ b/crates/core_simd/tests/ops_impl/float_macros.rs
@@ -335,8 +335,6 @@ macro_rules! float_tests {
                 }
             }
 
-            // TODO reenable after converting float ops to platform intrinsics
-            /*
             #[test]
             #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn ceil_odd_floats() {
@@ -415,7 +413,6 @@ macro_rules! float_tests {
                     assert_biteq!(core_simd::$vector::round_from_int(v), expected);
                 }
             }
-            */
         }
     }
 }

From 2720ccc5a78b4cbad1687aff59bb8631d59109c3 Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Mon, 14 Dec 2020 00:27:20 -0500
Subject: [PATCH 064/249] Fix masks

---
 crates/core_simd/src/masks/full_masks/mod.rs | 14 ++++++++++----
 1 file changed, 10 insertions(+), 4 deletions(-)

diff --git a/crates/core_simd/src/masks/full_masks/mod.rs b/crates/core_simd/src/masks/full_masks/mod.rs
index 829174669c2a..eac5178512e5 100644
--- a/crates/core_simd/src/masks/full_masks/mod.rs
+++ b/crates/core_simd/src/masks/full_masks/mod.rs
@@ -22,7 +22,13 @@ macro_rules! define_mask {
         impl<const $lanes: usize> $name<$lanes> {
             /// Construct a mask by setting all lanes to the given value.
             pub fn splat(value: bool) -> Self {
-                Self(<$type>::splat(value.into()))
+                Self(<$type>::splat(
+                    if value {
+                        -1
+                    } else {
+                        0
+                    }
+                ))
             }
 
             /// Tests the value of the specified lane.
@@ -31,7 +37,7 @@ macro_rules! define_mask {
             /// Panics if `lane` is greater than or equal to the number of lanes in the vector.
             #[inline]
             pub fn test(&self, lane: usize) -> bool {
-                self.0[lane] > 0
+                self.0[lane] == -1
             }
 
             /// Sets the value of the specified lane.
@@ -41,7 +47,7 @@ macro_rules! define_mask {
             #[inline]
             pub fn set(&mut self, lane: usize, value: bool) {
                 self.0[lane] = if value {
-                    !0
+                    -1
                 } else {
                     0
                 }
@@ -57,7 +63,7 @@ macro_rules! define_mask {
         impl<const $lanes: usize> core::convert::TryFrom<$type> for $name<$lanes> {
             type Error = TryFromMaskError;
             fn try_from(value: $type) -> Result<Self, Self::Error> {
-                if value.as_slice().iter().all(|x| *x == 0 || !*x == 0) {
+                if value.as_slice().iter().all(|x| *x == 0 || *x == -1) {
                     Ok(Self(value))
                 } else {
                     Err(TryFromMaskError(()))

From 62d98e3a11a289fc89da73f0c0601727de9cd28e Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Mon, 14 Dec 2020 00:34:32 -0500
Subject: [PATCH 065/249] Remove obsolete macros

---
 crates/core_simd/src/macros.rs    | 109 ------------------------------
 crates/core_simd/src/masks/mod.rs |   1 +
 crates/core_simd/src/ops.rs       |  15 ----
 3 files changed, 1 insertion(+), 124 deletions(-)

diff --git a/crates/core_simd/src/macros.rs b/crates/core_simd/src/macros.rs
index 99adb669bc51..c8f8e222da19 100644
--- a/crates/core_simd/src/macros.rs
+++ b/crates/core_simd/src/macros.rs
@@ -26,115 +26,6 @@ macro_rules! from_transmute_x86 {
     }
 }
 
-/// Calls a the macro `$mac` with the provided `$args` followed by `$repeat` repeated the specified
-/// number of times.
-macro_rules! call_repeat {
-    { 1 => $mac:path [$($repeat:tt)*] $($args:tt)* } => {
-        $mac! {
-            $($args)*
-            $($repeat)*
-        }
-    };
-    { 2 => $mac:path [$($repeat:tt)*] $($args:tt)* } => {
-        $mac! {
-            $($args)*
-            $($repeat)* $($repeat)*
-        }
-    };
-    { 4 => $mac:path [$($repeat:tt)*] $($args:tt)* } => {
-        $mac! {
-            $($args)*
-            $($repeat)* $($repeat)* $($repeat)* $($repeat)*
-        }
-    };
-    { 8 => $mac:path [$($repeat:tt)*] $($args:tt)* } => {
-        $mac! {
-            $($args)*
-            $($repeat)* $($repeat)* $($repeat)* $($repeat)* $($repeat)* $($repeat)* $($repeat)* $($repeat)*
-        }
-    };
-    { 16 => $mac:path [$($repeat:tt)*] $($args:tt)* } => {
-        $mac! {
-            $($args)*
-            $($repeat)* $($repeat)* $($repeat)* $($repeat)* $($repeat)* $($repeat)* $($repeat)* $($repeat)*
-            $($repeat)* $($repeat)* $($repeat)* $($repeat)* $($repeat)* $($repeat)* $($repeat)* $($repeat)*
-        }
-    };
-    { 32 => $mac:path [$($repeat:tt)*] $($args:tt)* } => {
-        $mac! {
-            $($args)*
-            $($repeat)* $($repeat)* $($repeat)* $($repeat)* $($repeat)* $($repeat)* $($repeat)* $($repeat)*
-            $($repeat)* $($repeat)* $($repeat)* $($repeat)* $($repeat)* $($repeat)* $($repeat)* $($repeat)*
-            $($repeat)* $($repeat)* $($repeat)* $($repeat)* $($repeat)* $($repeat)* $($repeat)* $($repeat)*
-            $($repeat)* $($repeat)* $($repeat)* $($repeat)* $($repeat)* $($repeat)* $($repeat)* $($repeat)*
-        }
-    };
-    { 64 => $mac:path [$($repeat:tt)*] $($args:tt)* } => {
-        $mac! {
-            $($args)*
-            $($repeat)* $($repeat)* $($repeat)* $($repeat)* $($repeat)* $($repeat)* $($repeat)* $($repeat)*
-            $($repeat)* $($repeat)* $($repeat)* $($repeat)* $($repeat)* $($repeat)* $($repeat)* $($repeat)*
-            $($repeat)* $($repeat)* $($repeat)* $($repeat)* $($repeat)* $($repeat)* $($repeat)* $($repeat)*
-            $($repeat)* $($repeat)* $($repeat)* $($repeat)* $($repeat)* $($repeat)* $($repeat)* $($repeat)*
-            $($repeat)* $($repeat)* $($repeat)* $($repeat)* $($repeat)* $($repeat)* $($repeat)* $($repeat)*
-            $($repeat)* $($repeat)* $($repeat)* $($repeat)* $($repeat)* $($repeat)* $($repeat)* $($repeat)*
-            $($repeat)* $($repeat)* $($repeat)* $($repeat)* $($repeat)* $($repeat)* $($repeat)* $($repeat)*
-            $($repeat)* $($repeat)* $($repeat)* $($repeat)* $($repeat)* $($repeat)* $($repeat)* $($repeat)*
-        }
-    };
-}
-
-/// Calls the macro `$mac` with the specified `$args` followed by the specified number of unique
-/// identifiers.
-macro_rules! call_counting_args {
-    { 1 => $mac:path => $($args:tt)* } => {
-        $mac! {
-            $($args)*
-            value
-        }
-    };
-    { 2 => $mac:path => $($args:tt)* } => {
-        $mac! {
-            $($args)*
-            v0 v1
-        }
-    };
-    { 4 => $mac:path => $($args:tt)* } => {
-        $mac! {
-            $($args)*
-            v0 v1 v2 v3
-        }
-    };
-    { 8 => $mac:path => $($args:tt)* } => {
-        $mac! {
-            $($args)*
-            v0 v1 v2 v3 v4 v5 v6 v7
-        }
-    };
-    { 16 => $mac:path => $($args:tt)* } => {
-        $mac! {
-            $($args)*
-            v0 v1 v2 v3 v4 v5 v6 v7 v8 v9 v10 v11 v12 v13 v14 v15
-        }
-    };
-    { 32 => $mac:path => $($args:tt)* } => {
-        $mac! {
-            $($args)*
-            v0  v1  v2  v3  v4  v5  v6  v7  v8  v9  v10 v11 v12 v13 v14 v15
-            v16 v17 v18 v19 v20 v21 v22 v23 v24 v25 v26 v27 v28 v29 v30 v31
-        }
-    };
-    { 64 => $mac:path => $($args:tt)* } => {
-        $mac! {
-            $($args)*
-            v0  v1  v2  v3  v4  v5  v6  v7  v8  v9  v10 v11 v12 v13 v14 v15
-            v16 v17 v18 v19 v20 v21 v22 v23 v24 v25 v26 v27 v28 v29 v30 v31
-            v32 v33 v34 v35 v36 v37 v38 v39 v40 v41 v42 v43 v44 v45 v46 v47
-            v48 v49 v50 v51 v52 v53 v54 v55 v56 v57 v58 v59 v60 v61 v62 v63
-        }
-    };
-}
-
 /// Implements common traits on the specified vector `$name`, holding multiple `$lanes` of `$type`.
 macro_rules! impl_vector {
     { $name:ident, $type:ty } => {
diff --git a/crates/core_simd/src/masks/mod.rs b/crates/core_simd/src/masks/mod.rs
index 9c90373fb472..a075040affe7 100644
--- a/crates/core_simd/src/masks/mod.rs
+++ b/crates/core_simd/src/masks/mod.rs
@@ -1,6 +1,7 @@
 //! Types and traits associated with masking lanes of vectors.
 #![allow(non_camel_case_types)]
 
+/// Implements bitwise ops on mask types by delegating the operators to the inner type.
 macro_rules! delegate_ops_to_inner {
     { $name:ident } => {
         impl<const LANES: usize> core::ops::BitAnd for $name<LANES> {
diff --git a/crates/core_simd/src/ops.rs b/crates/core_simd/src/ops.rs
index 942d071de44d..a21e9ab5bf1e 100644
--- a/crates/core_simd/src/ops.rs
+++ b/crates/core_simd/src/ops.rs
@@ -262,21 +262,6 @@ macro_rules! impl_float_ops {
     };
 }
 
-/// Implements mask operators for the provided types.
-macro_rules! impl_mask_ops {
-    { $($scalar:ty => $($vector:ident),*;)* } => {
-        $( // scalar
-            $( // vector
-                impl_op! { impl BitAnd for $vector, $scalar }
-                impl_op! { impl BitOr  for $vector, $scalar }
-                impl_op! { impl BitXor for $vector, $scalar }
-                impl_op! { impl Not for $vector, $scalar }
-                impl_op! { impl Index for $vector, $scalar }
-            )*
-        )*
-    };
-}
-
 /// Implements unsigned integer operators for the provided types.
 macro_rules! impl_unsigned_int_ops {
     { $($scalar:ty => $($vector:ident),*;)* } => {

From 9b8cb18c9f9efe95460a1d748894bc9b6a46a0f3 Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Mon, 14 Dec 2020 00:40:59 -0500
Subject: [PATCH 066/249] Remove obsolete files

---
 crates/core_simd/src/lib.rs                   | 13 ----------
 .../src/masks/full_masks/vectors_m128.rs      | 11 ---------
 .../src/masks/full_masks/vectors_m16.rs       | 21 ----------------
 .../src/masks/full_masks/vectors_m32.rs       | 21 ----------------
 .../src/masks/full_masks/vectors_m64.rs       | 16 -------------
 .../src/masks/full_masks/vectors_m8.rs        | 24 -------------------
 .../src/masks/full_masks/vectors_msize.rs     | 16 -------------
 7 files changed, 122 deletions(-)
 delete mode 100644 crates/core_simd/src/masks/full_masks/vectors_m128.rs
 delete mode 100644 crates/core_simd/src/masks/full_masks/vectors_m16.rs
 delete mode 100644 crates/core_simd/src/masks/full_masks/vectors_m32.rs
 delete mode 100644 crates/core_simd/src/masks/full_masks/vectors_m64.rs
 delete mode 100644 crates/core_simd/src/masks/full_masks/vectors_m8.rs
 delete mode 100644 crates/core_simd/src/masks/full_masks/vectors_msize.rs

diff --git a/crates/core_simd/src/lib.rs b/crates/core_simd/src/lib.rs
index 312a3237e232..3a3346975c15 100644
--- a/crates/core_simd/src/lib.rs
+++ b/crates/core_simd/src/lib.rs
@@ -44,16 +44,3 @@ mod vectors_f32;
 pub use vectors_f32::*;
 mod vectors_f64;
 pub use vectors_f64::*;
-
-//mod vectors_mask8;
-//pub use vectors_mask8::*;
-//mod vectors_mask16;
-//pub use vectors_mask16::*;
-//mod vectors_mask32;
-//pub use vectors_mask32::*;
-//mod vectors_mask64;
-//pub use vectors_mask64::*;
-//mod vectors_mask128;
-//pub use vectors_mask128::*;
-//mod vectors_masksize;
-//pub use vectors_masksize::*;
diff --git a/crates/core_simd/src/masks/full_masks/vectors_m128.rs b/crates/core_simd/src/masks/full_masks/vectors_m128.rs
deleted file mode 100644
index fddddac5fc4f..000000000000
--- a/crates/core_simd/src/masks/full_masks/vectors_m128.rs
+++ /dev/null
@@ -1,11 +0,0 @@
-use super::m128;
-
-define_mask_vector! {
-    /// Vector of two `m128` values
-    struct m128x2([i128 as m128; 2]);
-}
-
-define_mask_vector! {
-    /// Vector of four `m128` values
-    struct m128x4([i128 as m128; 4]);
-}
diff --git a/crates/core_simd/src/masks/full_masks/vectors_m16.rs b/crates/core_simd/src/masks/full_masks/vectors_m16.rs
deleted file mode 100644
index 3b05e83f673d..000000000000
--- a/crates/core_simd/src/masks/full_masks/vectors_m16.rs
+++ /dev/null
@@ -1,21 +0,0 @@
-use super::m16;
-
-define_mask_vector! {
-    /// Vector of four `m16` values
-    struct m16x4([i16 as m16; 4]);
-}
-
-define_mask_vector! {
-    /// Vector of eight `m16` values
-    struct m16x8([i16 as m16; 8]);
-}
-
-define_mask_vector! {
-    /// Vector of 16 `m16` values
-    struct m16x16([i16 as m16; 16]);
-}
-
-define_mask_vector! {
-    /// Vector of 32 `m16` values
-    struct m16x32([i16 as m16; 32]);
-}
diff --git a/crates/core_simd/src/masks/full_masks/vectors_m32.rs b/crates/core_simd/src/masks/full_masks/vectors_m32.rs
deleted file mode 100644
index de5745fb2833..000000000000
--- a/crates/core_simd/src/masks/full_masks/vectors_m32.rs
+++ /dev/null
@@ -1,21 +0,0 @@
-use super::m32;
-
-define_mask_vector! {
-    /// Vector of two `m32` values
-    struct m32x2([i32 as m32; 2]);
-}
-
-define_mask_vector! {
-    /// Vector of four `m32` values
-    struct m32x4([i32 as m32; 4]);
-}
-
-define_mask_vector! {
-    /// Vector of eight `m32` values
-    struct m32x8([i32 as m32; 8]);
-}
-
-define_mask_vector! {
-    /// Vector of 16 `m32` values
-    struct m32x16([i32 as m32; 16]);
-}
diff --git a/crates/core_simd/src/masks/full_masks/vectors_m64.rs b/crates/core_simd/src/masks/full_masks/vectors_m64.rs
deleted file mode 100644
index 55c8687fcfc4..000000000000
--- a/crates/core_simd/src/masks/full_masks/vectors_m64.rs
+++ /dev/null
@@ -1,16 +0,0 @@
-use super::m64;
-
-define_mask_vector! {
-    /// Vector of two `m64` values
-    struct m64x2([i64 as m64; 2]);
-}
-
-define_mask_vector! {
-    /// Vector of four `m64` values
-    struct m64x4([i64 as m64; 4]);
-}
-
-define_mask_vector! {
-    /// Vector of eight `m64` values
-    struct m64x8([i64 as m64; 8]);
-}
diff --git a/crates/core_simd/src/masks/full_masks/vectors_m8.rs b/crates/core_simd/src/masks/full_masks/vectors_m8.rs
deleted file mode 100644
index 85506dd93e13..000000000000
--- a/crates/core_simd/src/masks/full_masks/vectors_m8.rs
+++ /dev/null
@@ -1,24 +0,0 @@
-use super::m8;
-
-define_mask_vector! {
-    /// Vector of eight `m8` values
-    struct m8x8([i8 as m8; 8]);
-}
-
-define_mask_vector! {
-    /// Vector of 16 `m8` values
-    struct m8x16([i8 as m8; 16]);
-}
-
-define_mask_vector! {
-    /// Vector of 32 `m8` values
-    struct m8x32([i8 as m8; 32]);
-}
-
-define_mask_vector! {
-    /// Vector of 64 `m8` values
-    struct m8x64([i8 as m8; 64]);
-}
-
-#[repr(transparent)]
-struct VectorMask8<const LANES: usize>(crate::SimdI8<LANES>);
diff --git a/crates/core_simd/src/masks/full_masks/vectors_msize.rs b/crates/core_simd/src/masks/full_masks/vectors_msize.rs
deleted file mode 100644
index 497aba8ddbbf..000000000000
--- a/crates/core_simd/src/masks/full_masks/vectors_msize.rs
+++ /dev/null
@@ -1,16 +0,0 @@
-use super::msize;
-
-define_mask_vector! {
-    /// Vector of two `msize` values
-    struct msizex2([isize as msize; 2]);
-}
-
-define_mask_vector! {
-    /// Vector of four `msize` values
-    struct msizex4([isize as msize; 4]);
-}
-
-define_mask_vector! {
-    /// Vector of eight `msize` values
-    struct msizex8([isize as msize; 8]);
-}

From 59947717c5644a110caea924114b8f64fcb8234c Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Thu, 17 Dec 2020 01:19:39 -0500
Subject: [PATCH 067/249] Add workaround for rust-lang/rust#80108

---
 crates/core_simd/src/macros.rs | 18 +++++++++++++++++-
 1 file changed, 17 insertions(+), 1 deletion(-)

diff --git a/crates/core_simd/src/macros.rs b/crates/core_simd/src/macros.rs
index c8f8e222da19..7e4d85142490 100644
--- a/crates/core_simd/src/macros.rs
+++ b/crates/core_simd/src/macros.rs
@@ -52,7 +52,23 @@ macro_rules! impl_vector {
 
             /// Converts a SIMD vector to an array.
             pub const fn to_array(self) -> [$type; LANES] {
-                self.0
+                // workaround for rust-lang/rust#80108
+                // TODO fix this
+                #[cfg(target_arch = "wasm32")]
+                {
+                    let mut arr = [self.0[0]; LANES];
+                    let mut i = 0;
+                    while i < LANES {
+                        arr[i] = self.0[i];
+                        i += 1;
+                    }
+                    arr
+                }
+
+                #[cfg(not(target_arch = "wasm32"))]
+                {
+                    self.0
+                }
             }
         }
 

From cd36c983e7f4cf4a6bcab1430ae1d2060ec6136c Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Sun, 27 Dec 2020 19:36:12 -0500
Subject: [PATCH 068/249] Deploy documentation to GitHub Pages

---
 .github/workflows/doc.yml | 30 ++++++++++++++++++++++++++++++
 1 file changed, 30 insertions(+)
 create mode 100644 .github/workflows/doc.yml

diff --git a/.github/workflows/doc.yml b/.github/workflows/doc.yml
new file mode 100644
index 000000000000..4631bc65b388
--- /dev/null
+++ b/.github/workflows/doc.yml
@@ -0,0 +1,30 @@
+name: Documentation
+
+on:
+  push:
+    branches:
+      - master
+
+jobs:
+  release:
+    name: Deploy Documentation
+    runs-on: ubuntu-latest
+
+    steps:
+      - name: Checkout Repository
+        uses: actions/checkout@v1
+
+      - name: Setup Rust
+        run: |
+          rustup update nightly --no-self-update
+          rustup default nightly
+
+      - name: Build Documentation
+        run: cargo doc --no-deps
+      
+      - name: Deploy Documentation
+        uses: peaceiris/actions-gh-pages@v3
+        with:
+          deploy_key: ${{ secrets.ACTIONS_DEPLOY_KEY }}
+          publish_branch: gh-pages
+          publish_dir: ./target/doc

From d72927c85b1ae34e1115a19897d93b8dcf59e44d Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Mon, 28 Dec 2020 23:48:18 -0500
Subject: [PATCH 069/249] Switch docs deploy to GITHUB_TOKEN

---
 .github/workflows/doc.yml | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/.github/workflows/doc.yml b/.github/workflows/doc.yml
index 4631bc65b388..9d1fa66ccb59 100644
--- a/.github/workflows/doc.yml
+++ b/.github/workflows/doc.yml
@@ -25,6 +25,6 @@ jobs:
       - name: Deploy Documentation
         uses: peaceiris/actions-gh-pages@v3
         with:
-          deploy_key: ${{ secrets.ACTIONS_DEPLOY_KEY }}
+          github_token: ${{ secrets.GITHUB_TOKEN }}
           publish_branch: gh-pages
           publish_dir: ./target/doc

From cb036b534df1edde8a023bee3d820c7b73cbc863 Mon Sep 17 00:00:00 2001
From: Jubilee Young <workingjubilee@gmail.com>
Date: Tue, 26 Jan 2021 21:44:07 -0800
Subject: [PATCH 070/249] min_const_generics ride the train to stable

---
 crates/core_simd/src/lib.rs | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/crates/core_simd/src/lib.rs b/crates/core_simd/src/lib.rs
index 3a3346975c15..6c8da3a4e7f6 100644
--- a/crates/core_simd/src/lib.rs
+++ b/crates/core_simd/src/lib.rs
@@ -1,5 +1,5 @@
 #![no_std]
-#![feature(repr_simd, platform_intrinsics, link_llvm_intrinsics, simd_ffi, min_const_generics)]
+#![feature(repr_simd, platform_intrinsics, link_llvm_intrinsics, simd_ffi)]
 #![warn(missing_docs)]
 //! Portable SIMD module.
 

From c67fc2e4c5cf062725458108042dd89d618bd8de Mon Sep 17 00:00:00 2001
From: miguel raz <miguelraz@gmail.com>
Date: Mon, 25 Jan 2021 17:58:29 -0600
Subject: [PATCH 071/249] Add guards/tests for div,rem overflow cases

---
 crates/core_simd/src/ops.rs                   | 29 +++++++++-
 crates/core_simd/tests/ops_impl/int_macros.rs | 57 +++++++++++++++++++
 2 files changed, 84 insertions(+), 2 deletions(-)

diff --git a/crates/core_simd/src/ops.rs b/crates/core_simd/src/ops.rs
index a21e9ab5bf1e..51e97757c54e 100644
--- a/crates/core_simd/src/ops.rs
+++ b/crates/core_simd/src/ops.rs
@@ -283,13 +283,20 @@ macro_rules! impl_unsigned_int_ops {
 
                         #[inline]
                         fn div(self, rhs: Self) -> Self::Output {
-                            // TODO there is probably a better way of doing this
-                            if AsRef::<[$scalar]>::as_ref(&rhs)
+                            if rhs.as_slice()
                                 .iter()
                                 .any(|x| *x == 0)
                             {
                                 panic!("attempt to divide by zero");
                             }
+
+                            // Guards for div(MIN, -1),
+                            // this check only applies to signed ints
+                            if <$scalar>::MIN != 0 && self.as_slice().iter()
+                                    .zip(rhs.as_slice().iter())
+                                    .any(|(x,y)| *x == <$scalar>::MIN && *y == -1 as _) {
+                                panic!("attempt to divide with overflow");
+                            }
                             unsafe { crate::intrinsics::simd_div(self, rhs) }
                         }
                     }
@@ -304,6 +311,11 @@ macro_rules! impl_unsigned_int_ops {
                             if rhs == 0 {
                                 panic!("attempt to divide by zero");
                             }
+                            if <$scalar>::MIN != 0 &&
+                                self.as_slice().iter().any(|x| *x == <$scalar>::MIN) &&
+                                rhs == -1 as _ {
+                                    panic!("attempt to divide with overflow");
+                            }
                             let rhs = Self::splat(rhs);
                             unsafe { crate::intrinsics::simd_div(self, rhs) }
                         }
@@ -353,6 +365,14 @@ macro_rules! impl_unsigned_int_ops {
                             {
                                 panic!("attempt to calculate the remainder with a divisor of zero");
                             }
+
+                            // Guards for rem(MIN, -1)
+                            // this branch applies the check only to signed ints
+                            if <$scalar>::MIN != 0 && self.as_slice().iter()
+                                    .zip(rhs.as_slice().iter())
+                                    .any(|(x,y)| *x == <$scalar>::MIN && *y == -1 as _) {
+                                panic!("attempt to calculate the remainder with overflow");
+                            }
                             unsafe { crate::intrinsics::simd_rem(self, rhs) }
                         }
                     }
@@ -367,6 +387,11 @@ macro_rules! impl_unsigned_int_ops {
                             if rhs == 0 {
                                 panic!("attempt to calculate the remainder with a divisor of zero");
                             }
+                            if <$scalar>::MIN != 0 &&
+                                self.as_slice().iter().any(|x| *x == <$scalar>::MIN) &&
+                                rhs == -1 as _ {
+                                    panic!("attempt to calculate the remainder with overflow");
+                            }
                             let rhs = Self::splat(rhs);
                             unsafe { crate::intrinsics::simd_rem(self, rhs) }
                         }
diff --git a/crates/core_simd/tests/ops_impl/int_macros.rs b/crates/core_simd/tests/ops_impl/int_macros.rs
index af956aa3e521..43c75c6a9c49 100644
--- a/crates/core_simd/tests/ops_impl/int_macros.rs
+++ b/crates/core_simd/tests/ops_impl/int_macros.rs
@@ -228,6 +228,39 @@ macro_rules! int_tests {
                 assert_biteq!(a, expected);
             }
 
+            #[test]
+            #[should_panic]
+            fn div_min_panics() {
+                let a = from_slice(&vec![$scalar::MIN; 64]);
+                let b = from_slice(&vec![-1; 64]);
+                let _ = a / b;
+            }
+
+            #[test]
+            #[should_panic]
+            fn div_by_all_zeros_panics() {
+                let a = from_slice(&A);
+                let b = from_slice(&vec![0 ; 64]);
+                let _ = a / b;
+            }
+
+            #[test]
+            #[should_panic]
+            fn div_by_one_zero_panics() {
+                let a = from_slice(&A);
+                let mut b = from_slice(&B);
+                b[0] = 0 as _;
+                let _ = a / b;
+            }
+
+            #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
+            fn div_min_neg_one_no_panic() {
+                let a = from_slice(&A);
+                let b = from_slice(&vec![-1; 64]);
+                let _ = a / b;
+            }
+
             #[test]
             #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn rem() {
@@ -275,6 +308,30 @@ macro_rules! int_tests {
                 assert_biteq!(a, expected);
             }
 
+            #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
+            fn rem_min_neg_one_no_panic() {
+                let a = from_slice(&A);
+                let b = from_slice(&vec![-1; 64]);
+                let _ = a % b;
+            }
+
+            #[test]
+            #[should_panic]
+            fn rem_min_panic() {
+                let a = from_slice(&vec![$scalar::MIN; 64]);
+                let b = from_slice(&vec![-1 ; 64]);
+                let _ = a % b;
+            }
+
+            #[test]
+            #[should_panic]
+            fn rem_min_zero_panic() {
+                let a = from_slice(&A);
+                let b = from_slice(&vec![0 ; 64]);
+                let _ = a % b;
+            }
+
             #[test]
             #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn bitand() {

From e4cdd15b2182936378a7faeb765b6d435c986b04 Mon Sep 17 00:00:00 2001
From: Jubilee Young <workingjubilee@gmail.com>
Date: Thu, 28 Jan 2021 14:33:50 -0800
Subject: [PATCH 072/249] Add issue templates

The main purpose of these is to offer rerouting where rerouting is
appropriate. Mostly derived from existing examples in rust-lang/rust.
---
 .github/ISSUE_TEMPLATE/bug_report.md      | 50 +++++++++++++++++++++++
 .github/ISSUE_TEMPLATE/config.yml         |  8 ++++
 .github/ISSUE_TEMPLATE/feature_request.md | 14 +++++++
 3 files changed, 72 insertions(+)
 create mode 100644 .github/ISSUE_TEMPLATE/bug_report.md
 create mode 100644 .github/ISSUE_TEMPLATE/config.yml
 create mode 100644 .github/ISSUE_TEMPLATE/feature_request.md

diff --git a/.github/ISSUE_TEMPLATE/bug_report.md b/.github/ISSUE_TEMPLATE/bug_report.md
new file mode 100644
index 000000000000..16a8251d52ba
--- /dev/null
+++ b/.github/ISSUE_TEMPLATE/bug_report.md
@@ -0,0 +1,50 @@
+---
+name: Bug Report
+about: Create a bug report for Rust.
+labels: C-bug
+---
+<!--
+Thank you for filing a bug report! 🐛 Please provide a short summary of the bug,
+along with any information you feel relevant to replicating the bug.
+-->
+
+I tried this code:
+
+```rust
+<code>
+```
+
+I expected to see this happen: *explanation*
+
+Instead, this happened: *explanation*
+
+### Meta
+
+`rustc --version --verbose`:
+```
+<version>
+```
+
+
+`crate version in Cargo.toml`:
+```toml
+[dependencies]
+stdsimd = 
+```
+<!-- If this specifies the repo at HEAD, please include the latest commit. -->
+
+
+<!--
+If a backtrace is available, please include a backtrace in the code block by
+setting `RUST_BACKTRACE=1` in your environment. e.g.
+`RUST_BACKTRACE=1 cargo build`.
+-->
+<details><summary>Backtrace</summary>
+<p>
+
+```
+<backtrace>
+```
+
+</p>
+</details>
diff --git a/.github/ISSUE_TEMPLATE/config.yml b/.github/ISSUE_TEMPLATE/config.yml
new file mode 100644
index 000000000000..a31dab6a0921
--- /dev/null
+++ b/.github/ISSUE_TEMPLATE/config.yml
@@ -0,0 +1,8 @@
+blank_issues_enabled: true
+contact_links:
+  - name: Intrinsic Support
+    url: https://github.com/rust-lang/stdarch/issues
+    about: Please direct issues about Rust's support for vendor intrinsics to core::arch
+  - name: Internal Compiler Error
+    url: https://github.com/rust-lang/rust/issues
+    about: Please report ICEs to the rustc repository
diff --git a/.github/ISSUE_TEMPLATE/feature_request.md b/.github/ISSUE_TEMPLATE/feature_request.md
new file mode 100644
index 000000000000..be6c9e3d17e2
--- /dev/null
+++ b/.github/ISSUE_TEMPLATE/feature_request.md
@@ -0,0 +1,14 @@
+---
+name: Feature Request
+about: Request an addition to the core::simd API
+labels: C-feature-request
+---
+<!--
+  Hello!
+
+  We are very interested in any feature requests you may have.
+
+  However, please be aware that core::simd exists to address concerns with creating a portable SIMD API for Rust.
+  Requests for extensions to compiler features, such as `target_feature`, binary versioning for SIMD APIs, or
+  improving specific compilation issues in general should be discussed at https://internals.rust-lang.org/
+-->

From fd6179b4cd14d54a1bf8cd747ede76e1e3a1ba40 Mon Sep 17 00:00:00 2001
From: Jubilee Young <workingjubilee@gmail.com>
Date: Tue, 2 Feb 2021 17:43:05 -0800
Subject: [PATCH 073/249] Add a blank issue template

Setting this in config.yml only enables the appearance of a tiny link
on GitHub that is hard to see and find. This template adds a "blank issue"
link that is as equally visible as the rest of the template options.
---
 .github/ISSUE_TEMPLATE/blank_issue.md | 4 ++++
 .github/ISSUE_TEMPLATE/config.yml     | 2 ++
 2 files changed, 6 insertions(+)
 create mode 100644 .github/ISSUE_TEMPLATE/blank_issue.md

diff --git a/.github/ISSUE_TEMPLATE/blank_issue.md b/.github/ISSUE_TEMPLATE/blank_issue.md
new file mode 100644
index 000000000000..9aef3ebe637a
--- /dev/null
+++ b/.github/ISSUE_TEMPLATE/blank_issue.md
@@ -0,0 +1,4 @@
+---
+name: Blank Issue
+about: Create a blank issue.
+---
diff --git a/.github/ISSUE_TEMPLATE/config.yml b/.github/ISSUE_TEMPLATE/config.yml
index a31dab6a0921..1567542c01a4 100644
--- a/.github/ISSUE_TEMPLATE/config.yml
+++ b/.github/ISSUE_TEMPLATE/config.yml
@@ -1,3 +1,5 @@
+# This only controls whether a tiny, hard-to-find "open a blank issue" link appears at the end of
+# the template list.
 blank_issues_enabled: true
 contact_links:
   - name: Intrinsic Support

From 8bea63425ea5d18f69802d9a1475ea1e2f417595 Mon Sep 17 00:00:00 2001
From: miguel raz <miguelraz@gmail.com>
Date: Wed, 3 Feb 2021 17:46:08 -0600
Subject: [PATCH 074/249] AsRef -> as_slices()

---
 crates/core_simd/src/ops.rs | 5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)

diff --git a/crates/core_simd/src/ops.rs b/crates/core_simd/src/ops.rs
index 51e97757c54e..ae7c86c81c50 100644
--- a/crates/core_simd/src/ops.rs
+++ b/crates/core_simd/src/ops.rs
@@ -358,8 +358,7 @@ macro_rules! impl_unsigned_int_ops {
 
                         #[inline]
                         fn rem(self, rhs: Self) -> Self::Output {
-                            // TODO there is probably a better way of doing this
-                            if AsRef::<[$scalar]>::as_ref(&rhs)
+                            if rhs.as_slice()
                                 .iter()
                                 .any(|x| *x == 0)
                             {
@@ -435,7 +434,7 @@ macro_rules! impl_unsigned_int_ops {
                         #[inline]
                         fn shl(self, rhs: Self) -> Self::Output {
                             // TODO there is probably a better way of doing this
-                            if AsRef::<[$scalar]>::as_ref(&rhs)
+                            if rhs.as_slice()
                                 .iter()
                                 .copied()
                                 .any(invalid_shift_rhs)

From 5424140b66bf839d20cbf32bf4604e23b30074c5 Mon Sep 17 00:00:00 2001
From: Jubilee Young <workingjubilee@gmail.com>
Date: Thu, 4 Feb 2021 14:39:15 -0800
Subject: [PATCH 075/249] Add SIMD shuffles for SimdType{2,4,8,16,32,64}

This const generic implementation for certain lane sizes represents
a more limited interface than what LLVM's shufflevector instruction
can handle, as normally the length of U can be different from the
length of T, but offers an interface that it is expected to be able
to expand the capabilities of in the future.
---
 crates/core_simd/src/intrinsics.rs |  8 ++++++++
 crates/core_simd/src/lib.rs        | 11 ++++++++++-
 crates/core_simd/src/macros.rs     |  2 ++
 crates/core_simd/src/permute.rs    | 29 +++++++++++++++++++++++++++++
 crates/core_simd/tests/permute.rs  | 15 +++++++++++++++
 5 files changed, 64 insertions(+), 1 deletion(-)
 create mode 100644 crates/core_simd/src/permute.rs
 create mode 100644 crates/core_simd/tests/permute.rs

diff --git a/crates/core_simd/src/intrinsics.rs b/crates/core_simd/src/intrinsics.rs
index 51689cd97bea..4ea383763487 100644
--- a/crates/core_simd/src/intrinsics.rs
+++ b/crates/core_simd/src/intrinsics.rs
@@ -52,4 +52,12 @@ extern "platform-intrinsic" {
     pub(crate) fn simd_le<T, U>(x: T, y: T) -> U;
     pub(crate) fn simd_gt<T, U>(x: T, y: T) -> U;
     pub(crate) fn simd_ge<T, U>(x: T, y: T) -> U;
+
+    // shufflevector
+    pub(crate) fn simd_shuffle2<T, U>(x: T, y: T, idx: [u32; 2]) -> U;
+    pub(crate) fn simd_shuffle4<T, U>(x: T, y: T, idx: [u32; 4]) -> U;
+    pub(crate) fn simd_shuffle8<T, U>(x: T, y: T, idx: [u32; 8]) -> U;
+    pub(crate) fn simd_shuffle16<T, U>(x: T, y: T, idx: [u32; 16]) -> U;
+    pub(crate) fn simd_shuffle32<T, U>(x: T, y: T, idx: [u32; 32]) -> U;
+    pub(crate) fn simd_shuffle64<T, U>(x: T, y: T, idx: [u32; 64]) -> U;
 }
diff --git a/crates/core_simd/src/lib.rs b/crates/core_simd/src/lib.rs
index 6c8da3a4e7f6..62489e4fbd50 100644
--- a/crates/core_simd/src/lib.rs
+++ b/crates/core_simd/src/lib.rs
@@ -1,10 +1,19 @@
 #![no_std]
-#![feature(repr_simd, platform_intrinsics, link_llvm_intrinsics, simd_ffi)]
+#![allow(incomplete_features)]
+#![feature(
+    repr_simd,
+    platform_intrinsics,
+    link_llvm_intrinsics,
+    simd_ffi,
+    const_generics
+)]
 #![warn(missing_docs)]
 //! Portable SIMD module.
 
 #[macro_use]
 mod macros;
+#[macro_use]
+mod permute;
 
 mod fmt;
 mod intrinsics;
diff --git a/crates/core_simd/src/macros.rs b/crates/core_simd/src/macros.rs
index 7e4d85142490..0abafe71f50d 100644
--- a/crates/core_simd/src/macros.rs
+++ b/crates/core_simd/src/macros.rs
@@ -148,6 +148,8 @@ macro_rules! impl_vector {
                 Self::splat(value)
             }
         }
+
+        impl_shuffle_2pow_lanes!{ $name }
     }
 }
 
diff --git a/crates/core_simd/src/permute.rs b/crates/core_simd/src/permute.rs
new file mode 100644
index 000000000000..05a78c3764b8
--- /dev/null
+++ b/crates/core_simd/src/permute.rs
@@ -0,0 +1,29 @@
+macro_rules! impl_shuffle_lane {
+    { $name:ident, $fn:ident, $n:literal } => {
+        impl $name<$n> {
+            /// A const SIMD shuffle that takes 2 SIMD vectors and produces another vector, using
+            /// the indices in the const parameter. The first or "self" vector will have its lanes
+            /// indexed from 0, and the second vector will have its first lane indexed at $n.
+            /// Indices must be in-bounds of either vector at compile time.
+            ///
+            /// Some SIMD shuffle instructions can be quite slow, so avoiding them by loading data
+            /// into the desired patterns in advance is preferred, but shuffles are still faster
+            /// than storing and reloading from memory.
+            #[inline]
+            pub fn shuffle<const IDX: [u32; $n]>(self, second: Self) -> Self {
+                unsafe { crate::intrinsics::$fn(self, second, IDX) }
+            }
+        }
+    }
+}
+
+macro_rules! impl_shuffle_2pow_lanes {
+    { $name:ident } => {
+        impl_shuffle_lane!{ $name, simd_shuffle2, 2 }
+        impl_shuffle_lane!{ $name, simd_shuffle4, 4 }
+        impl_shuffle_lane!{ $name, simd_shuffle8, 8 }
+        impl_shuffle_lane!{ $name, simd_shuffle16, 16 }
+        impl_shuffle_lane!{ $name, simd_shuffle32, 32 }
+        impl_shuffle_lane!{ $name, simd_shuffle64, 64 }
+    }
+}
diff --git a/crates/core_simd/tests/permute.rs b/crates/core_simd/tests/permute.rs
new file mode 100644
index 000000000000..1c6c391d8d06
--- /dev/null
+++ b/crates/core_simd/tests/permute.rs
@@ -0,0 +1,15 @@
+use core_simd::SimdU32;
+
+#[cfg(target_arch = "wasm32")]
+use wasm_bindgen_test::*;
+
+#[cfg(target_arch = "wasm32")]
+wasm_bindgen_test_configure!(run_in_browser);
+
+#[test]
+#[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
+fn simple_shuffle() {
+    let a = SimdU32::from_array([2, 4, 1, 9]);
+    let b = a;
+    assert_eq!(a.shuffle::<{ [3, 1, 4, 6] }>(b).to_array(), [9, 4, 2, 1]);
+}

From 55f0efcbb4f8e3291ea8cedb97ee94c8db5652bb Mon Sep 17 00:00:00 2001
From: Ashley Mannix <kodraus@hey.com>
Date: Tue, 9 Feb 2021 12:52:27 +1000
Subject: [PATCH 076/249] add a link to docs to the readme

---
 README.md | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/README.md b/README.md
index 049c8b097e7e..fe17108c843f 100644
--- a/README.md
+++ b/README.md
@@ -4,6 +4,9 @@
 Code repository for the [Portable SIMD Project Group](https://github.com/rust-lang/project-portable-simd).
 Please refer to [CONTRIBUTING.md](./CONTRIBUTING.md) for our contributing guidelines.
 
+The docs for this crate are published from the main branch.
+You can [read them here][docs].
+
 If you have questions about SIMD, we have begun writing a [guide][simd-guide].
 We can also be found on [Zulip][zulip-project-portable-simd].
 
@@ -29,3 +32,4 @@ The `mask` types are "truthy" values, but they use the number of bits in their n
 [simd-guide]: ./beginners-guide.md
 [zulip-project-portable-simd]: https://rust-lang.zulipchat.com/#narrow/stream/257879-project-portable-simd
 [stdarch]: https://github.com/rust-lang/stdarch
+[docs]: https://rust-lang.github.io/stdsimd/core_simd

From 92293aff9d44c74a57f85edf9af0f1b449f20ff0 Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Thu, 4 Feb 2021 00:31:57 -0500
Subject: [PATCH 077/249] Add bitmask that supports up to 64 lanes. Simplify
 mask op API.

---
 crates/core_simd/src/lib.rs                  |   8 +-
 crates/core_simd/src/masks/bitmask.rs        | 220 ++++++++++++
 crates/core_simd/src/masks/full_masks.rs     | 301 +++++++++++++++++
 crates/core_simd/src/masks/full_masks/mod.rs | 205 -----------
 crates/core_simd/src/masks/mod.rs            | 336 +++++++++----------
 5 files changed, 687 insertions(+), 383 deletions(-)
 create mode 100644 crates/core_simd/src/masks/bitmask.rs
 create mode 100644 crates/core_simd/src/masks/full_masks.rs
 delete mode 100644 crates/core_simd/src/masks/full_masks/mod.rs

diff --git a/crates/core_simd/src/lib.rs b/crates/core_simd/src/lib.rs
index 62489e4fbd50..de9cb566022e 100644
--- a/crates/core_simd/src/lib.rs
+++ b/crates/core_simd/src/lib.rs
@@ -1,12 +1,6 @@
 #![no_std]
 #![allow(incomplete_features)]
-#![feature(
-    repr_simd,
-    platform_intrinsics,
-    link_llvm_intrinsics,
-    simd_ffi,
-    const_generics
-)]
+#![feature(repr_simd, platform_intrinsics, simd_ffi, const_generics)]
 #![warn(missing_docs)]
 //! Portable SIMD module.
 
diff --git a/crates/core_simd/src/masks/bitmask.rs b/crates/core_simd/src/masks/bitmask.rs
new file mode 100644
index 000000000000..51ed8037043d
--- /dev/null
+++ b/crates/core_simd/src/masks/bitmask.rs
@@ -0,0 +1,220 @@
+/// Implemented for bitmask sizes that are supported by the implementation.
+pub trait LanesAtMost64 {}
+impl LanesAtMost64 for BitMask<1> {}
+impl LanesAtMost64 for BitMask<2> {}
+impl LanesAtMost64 for BitMask<4> {}
+impl LanesAtMost64 for BitMask<8> {}
+impl LanesAtMost64 for BitMask<16> {}
+impl LanesAtMost64 for BitMask<32> {}
+impl LanesAtMost64 for BitMask<64> {}
+
+/// A mask where each lane is represented by a single bit.
+#[derive(Copy, Clone, Debug)]
+#[repr(transparent)]
+pub struct BitMask<const LANES: usize>(u64)
+where
+    BitMask<LANES>: LanesAtMost64;
+
+impl<const LANES: usize> BitMask<LANES>
+where
+    Self: LanesAtMost64,
+{
+    /// Construct a mask by setting all lanes to the given value.
+    pub fn splat(value: bool) -> Self {
+        if value {
+            Self(u64::MAX)
+        } else {
+            Self(u64::MIN)
+        }
+    }
+
+    /// Tests the value of the specified lane.
+    ///
+    /// # Panics
+    /// Panics if `lane` is greater than or equal to the number of lanes in the vector.
+    #[inline]
+    pub fn test(&self, lane: usize) -> bool {
+        assert!(lane < LANES, "lane index out of range");
+        (self.0 >> lane) & 0x1 > 0
+    }
+
+    /// Sets the value of the specified lane.
+    ///
+    /// # Panics
+    /// Panics if `lane` is greater than or equal to the number of lanes in the vector.
+    #[inline]
+    pub fn set(&mut self, lane: usize, value: bool) {
+        assert!(lane < LANES, "lane index out of range");
+        self.0 ^= ((value ^ self.test(lane)) as u64) << lane
+    }
+}
+
+impl<const LANES: usize> core::ops::BitAnd for BitMask<LANES>
+where
+    Self: LanesAtMost64,
+{
+    type Output = Self;
+    #[inline]
+    fn bitand(self, rhs: Self) -> Self {
+        Self(self.0 & rhs.0)
+    }
+}
+
+impl<const LANES: usize> core::ops::BitAnd<bool> for BitMask<LANES>
+where
+    Self: LanesAtMost64,
+{
+    type Output = Self;
+    #[inline]
+    fn bitand(self, rhs: bool) -> Self {
+        self & Self::splat(rhs)
+    }
+}
+
+impl<const LANES: usize> core::ops::BitAnd<BitMask<LANES>> for bool
+where
+    BitMask<LANES>: LanesAtMost64,
+{
+    type Output = BitMask<LANES>;
+    #[inline]
+    fn bitand(self, rhs: BitMask<LANES>) -> BitMask<LANES> {
+        BitMask::<LANES>::splat(self) & rhs
+    }
+}
+
+impl<const LANES: usize> core::ops::BitOr for BitMask<LANES>
+where
+    Self: LanesAtMost64,
+{
+    type Output = Self;
+    #[inline]
+    fn bitor(self, rhs: Self) -> Self {
+        Self(self.0 | rhs.0)
+    }
+}
+
+impl<const LANES: usize> core::ops::BitOr<bool> for BitMask<LANES>
+where
+    Self: LanesAtMost64,
+{
+    type Output = Self;
+    #[inline]
+    fn bitor(self, rhs: bool) -> Self {
+        self | Self::splat(rhs)
+    }
+}
+
+impl<const LANES: usize> core::ops::BitOr<BitMask<LANES>> for bool
+where
+    BitMask<LANES>: LanesAtMost64,
+{
+    type Output = BitMask<LANES>;
+    #[inline]
+    fn bitor(self, rhs: BitMask<LANES>) -> BitMask<LANES> {
+        BitMask::<LANES>::splat(self) | rhs
+    }
+}
+
+impl<const LANES: usize> core::ops::BitXor for BitMask<LANES>
+where
+    Self: LanesAtMost64,
+{
+    type Output = Self;
+    #[inline]
+    fn bitxor(self, rhs: Self) -> Self::Output {
+        Self(self.0 ^ rhs.0)
+    }
+}
+
+impl<const LANES: usize> core::ops::BitXor<bool> for BitMask<LANES>
+where
+    Self: LanesAtMost64,
+{
+    type Output = Self;
+    #[inline]
+    fn bitxor(self, rhs: bool) -> Self::Output {
+        self ^ Self::splat(rhs)
+    }
+}
+
+impl<const LANES: usize> core::ops::BitXor<BitMask<LANES>> for bool
+where
+    BitMask<LANES>: LanesAtMost64,
+{
+    type Output = BitMask<LANES>;
+    #[inline]
+    fn bitxor(self, rhs: BitMask<LANES>) -> Self::Output {
+        BitMask::<LANES>::splat(self) ^ rhs
+    }
+}
+
+impl<const LANES: usize> core::ops::Not for BitMask<LANES>
+where
+    Self: LanesAtMost64,
+{
+    type Output = BitMask<LANES>;
+    #[inline]
+    fn not(self) -> Self::Output {
+        Self(!self.0)
+    }
+}
+
+impl<const LANES: usize> core::ops::BitAndAssign for BitMask<LANES>
+where
+    Self: LanesAtMost64,
+{
+    #[inline]
+    fn bitand_assign(&mut self, rhs: Self) {
+        self.0 &= rhs.0;
+    }
+}
+
+impl<const LANES: usize> core::ops::BitAndAssign<bool> for BitMask<LANES>
+where
+    Self: LanesAtMost64,
+{
+    #[inline]
+    fn bitand_assign(&mut self, rhs: bool) {
+        *self &= Self::splat(rhs);
+    }
+}
+
+impl<const LANES: usize> core::ops::BitOrAssign for BitMask<LANES>
+where
+    Self: LanesAtMost64,
+{
+    #[inline]
+    fn bitor_assign(&mut self, rhs: Self) {
+        self.0 |= rhs.0;
+    }
+}
+
+impl<const LANES: usize> core::ops::BitOrAssign<bool> for BitMask<LANES>
+where
+    Self: LanesAtMost64,
+{
+    #[inline]
+    fn bitor_assign(&mut self, rhs: bool) {
+        *self |= Self::splat(rhs);
+    }
+}
+
+impl<const LANES: usize> core::ops::BitXorAssign for BitMask<LANES>
+where
+    Self: LanesAtMost64,
+{
+    #[inline]
+    fn bitxor_assign(&mut self, rhs: Self) {
+        self.0 ^= rhs.0;
+    }
+}
+
+impl<const LANES: usize> core::ops::BitXorAssign<bool> for BitMask<LANES>
+where
+    Self: LanesAtMost64,
+{
+    #[inline]
+    fn bitxor_assign(&mut self, rhs: bool) {
+        *self ^= Self::splat(rhs);
+    }
+}
diff --git a/crates/core_simd/src/masks/full_masks.rs b/crates/core_simd/src/masks/full_masks.rs
new file mode 100644
index 000000000000..d7c4af47727c
--- /dev/null
+++ b/crates/core_simd/src/masks/full_masks.rs
@@ -0,0 +1,301 @@
+//! Masks that take up full SIMD vector registers.
+
+/// The error type returned when converting an integer to a mask fails.
+#[derive(Debug, Copy, Clone, PartialEq, Eq)]
+pub struct TryFromMaskError(());
+
+impl core::fmt::Display for TryFromMaskError {
+    fn fmt(&self, f: &mut core::fmt::Formatter) -> core::fmt::Result {
+        write!(
+            f,
+            "mask vector must have all bits set or unset in each lane"
+        )
+    }
+}
+
+macro_rules! define_mask {
+    { $(#[$attr:meta])* struct $name:ident<const $lanes:ident: usize>($type:ty); } => {
+        $(#[$attr])*
+        #[derive(Copy, Clone, Default, PartialEq, PartialOrd, Eq, Ord, Hash)]
+        #[repr(transparent)]
+        pub struct $name<const $lanes: usize>($type);
+
+        impl<const $lanes: usize> $name<$lanes> {
+            /// Construct a mask by setting all lanes to the given value.
+            pub fn splat(value: bool) -> Self {
+                Self(<$type>::splat(
+                    if value {
+                        -1
+                    } else {
+                        0
+                    }
+                ))
+            }
+
+            /// Tests the value of the specified lane.
+            ///
+            /// # Panics
+            /// Panics if `lane` is greater than or equal to the number of lanes in the vector.
+            #[inline]
+            pub fn test(&self, lane: usize) -> bool {
+                assert!(lane < LANES, "lane index out of range");
+                self.0[lane] == -1
+            }
+
+            /// Sets the value of the specified lane.
+            ///
+            /// # Panics
+            /// Panics if `lane` is greater than or equal to the number of lanes in the vector.
+            #[inline]
+            pub fn set(&mut self, lane: usize, value: bool) {
+                assert!(lane < LANES, "lane index out of range");
+                self.0[lane] = if value {
+                    -1
+                } else {
+                    0
+                }
+            }
+        }
+
+        impl<const $lanes: usize> core::convert::From<bool> for $name<$lanes> {
+            fn from(value: bool) -> Self {
+                Self::splat(value)
+            }
+        }
+
+        impl<const $lanes: usize> core::convert::TryFrom<$type> for $name<$lanes> {
+            type Error = TryFromMaskError;
+            fn try_from(value: $type) -> Result<Self, Self::Error> {
+                if value.as_slice().iter().all(|x| *x == 0 || *x == -1) {
+                    Ok(Self(value))
+                } else {
+                    Err(TryFromMaskError(()))
+                }
+            }
+        }
+
+        impl<const $lanes: usize> core::convert::From<$name<$lanes>> for $type {
+            fn from(value: $name<$lanes>) -> Self {
+                value.0
+            }
+        }
+
+        impl<const $lanes: usize> core::convert::From<crate::BitMask<$lanes>> for $name<$lanes>
+        where
+            crate::BitMask<$lanes>: crate::LanesAtMost64,
+        {
+            fn from(value: crate::BitMask<$lanes>) -> Self {
+                // TODO use an intrinsic to do this efficiently (with LLVM's sext instruction)
+                let mut mask = Self::splat(false);
+                for lane in 0..LANES {
+                    mask.set(lane, value.test(lane));
+                }
+                mask
+            }
+        }
+
+        impl<const $lanes: usize> core::convert::From<$name<$lanes>> for crate::BitMask<$lanes>
+        where
+            crate::BitMask<$lanes>: crate::LanesAtMost64,
+        {
+            fn from(value: $name<$lanes>) -> Self {
+                // TODO use an intrinsic to do this efficiently (with LLVM's trunc instruction)
+                let mut mask = Self::splat(false);
+                for lane in 0..LANES {
+                    mask.set(lane, value.test(lane));
+                }
+                mask
+            }
+        }
+
+        impl<const $lanes: usize> core::fmt::Debug for $name<$lanes> {
+            fn fmt(&self, f: &mut core::fmt::Formatter) -> core::fmt::Result {
+                f.debug_list()
+                    .entries((0..LANES).map(|lane| self.test(lane)))
+                    .finish()
+            }
+        }
+
+        impl<const $lanes: usize> core::fmt::Binary for $name<$lanes> {
+            fn fmt(&self, f: &mut core::fmt::Formatter) -> core::fmt::Result {
+                core::fmt::Binary::fmt(&self.0, f)
+            }
+        }
+
+        impl<const $lanes: usize> core::fmt::Octal for $name<$lanes> {
+            fn fmt(&self, f: &mut core::fmt::Formatter) -> core::fmt::Result {
+                core::fmt::Octal::fmt(&self.0, f)
+            }
+        }
+
+        impl<const $lanes: usize> core::fmt::LowerHex for $name<$lanes> {
+            fn fmt(&self, f: &mut core::fmt::Formatter) -> core::fmt::Result {
+                core::fmt::LowerHex::fmt(&self.0, f)
+            }
+        }
+
+        impl<const $lanes: usize> core::fmt::UpperHex for $name<$lanes> {
+            fn fmt(&self, f: &mut core::fmt::Formatter) -> core::fmt::Result {
+                core::fmt::UpperHex::fmt(&self.0, f)
+            }
+        }
+
+        impl<const LANES: usize> core::ops::BitAnd for $name<LANES> {
+            type Output = Self;
+            #[inline]
+            fn bitand(self, rhs: Self) -> Self {
+                Self(self.0 & rhs.0)
+            }
+        }
+
+        impl<const LANES: usize> core::ops::BitAnd<bool> for $name<LANES> {
+            type Output = Self;
+            #[inline]
+            fn bitand(self, rhs: bool) -> Self {
+                self & Self::splat(rhs)
+            }
+        }
+
+        impl<const LANES: usize> core::ops::BitAnd<$name<LANES>> for bool {
+            type Output = $name<LANES>;
+            #[inline]
+            fn bitand(self, rhs: $name<LANES>) -> $name<LANES> {
+                $name::<LANES>::splat(self) & rhs
+            }
+        }
+
+        impl<const LANES: usize> core::ops::BitOr for $name<LANES> {
+            type Output = Self;
+            #[inline]
+            fn bitor(self, rhs: Self) -> Self {
+                Self(self.0 | rhs.0)
+            }
+        }
+
+        impl<const LANES: usize> core::ops::BitOr<bool> for $name<LANES> {
+            type Output = Self;
+            #[inline]
+            fn bitor(self, rhs: bool) -> Self {
+                self | Self::splat(rhs)
+            }
+        }
+
+        impl<const LANES: usize> core::ops::BitOr<$name<LANES>> for bool {
+            type Output = $name<LANES>;
+            #[inline]
+            fn bitor(self, rhs: $name<LANES>) -> $name<LANES> {
+                $name::<LANES>::splat(self) | rhs
+            }
+        }
+
+        impl<const LANES: usize> core::ops::BitXor for $name<LANES> {
+            type Output = Self;
+            #[inline]
+            fn bitxor(self, rhs: Self) -> Self::Output {
+                Self(self.0 ^ rhs.0)
+            }
+        }
+
+        impl<const LANES: usize> core::ops::BitXor<bool> for $name<LANES> {
+            type Output = Self;
+            #[inline]
+            fn bitxor(self, rhs: bool) -> Self::Output {
+                self ^ Self::splat(rhs)
+            }
+        }
+
+        impl<const LANES: usize> core::ops::BitXor<$name<LANES>> for bool {
+            type Output = $name<LANES>;
+            #[inline]
+            fn bitxor(self, rhs: $name<LANES>) -> Self::Output {
+                $name::<LANES>::splat(self) ^ rhs
+            }
+        }
+
+        impl<const LANES: usize> core::ops::Not for $name<LANES> {
+            type Output = $name<LANES>;
+            #[inline]
+            fn not(self) -> Self::Output {
+                Self(!self.0)
+            }
+        }
+
+        impl<const LANES: usize> core::ops::BitAndAssign for $name<LANES> {
+            #[inline]
+            fn bitand_assign(&mut self, rhs: Self) {
+                self.0 &= rhs.0;
+            }
+        }
+
+        impl<const LANES: usize> core::ops::BitAndAssign<bool> for $name<LANES> {
+            #[inline]
+            fn bitand_assign(&mut self, rhs: bool) {
+                *self &= Self::splat(rhs);
+            }
+        }
+
+        impl<const LANES: usize> core::ops::BitOrAssign for $name<LANES> {
+            #[inline]
+            fn bitor_assign(&mut self, rhs: Self) {
+                self.0 |= rhs.0;
+            }
+        }
+
+        impl<const LANES: usize> core::ops::BitOrAssign<bool> for $name<LANES> {
+            #[inline]
+            fn bitor_assign(&mut self, rhs: bool) {
+                *self |= Self::splat(rhs);
+            }
+        }
+
+        impl<const LANES: usize> core::ops::BitXorAssign for $name<LANES> {
+            #[inline]
+            fn bitxor_assign(&mut self, rhs: Self) {
+                self.0 ^= rhs.0;
+            }
+        }
+
+        impl<const LANES: usize> core::ops::BitXorAssign<bool> for $name<LANES> {
+            #[inline]
+            fn bitxor_assign(&mut self, rhs: bool) {
+                *self ^= Self::splat(rhs);
+            }
+        }
+    }
+}
+
+define_mask! {
+    /// A mask equivalent to [SimdI8](crate::SimdI8), where all bits in the lane must be either set
+    /// or unset.
+    struct SimdMask8<const LANES: usize>(crate::SimdI8<LANES>);
+}
+
+define_mask! {
+    /// A mask equivalent to [SimdI16](crate::SimdI16), where all bits in the lane must be either set
+    /// or unset.
+    struct SimdMask16<const LANES: usize>(crate::SimdI16<LANES>);
+}
+
+define_mask! {
+    /// A mask equivalent to [SimdI32](crate::SimdI32), where all bits in the lane must be either set
+    /// or unset.
+    struct SimdMask32<const LANES: usize>(crate::SimdI32<LANES>);
+}
+
+define_mask! {
+    /// A mask equivalent to [SimdI64](crate::SimdI64), where all bits in the lane must be either set
+    /// or unset.
+    struct SimdMask64<const LANES: usize>(crate::SimdI64<LANES>);
+}
+
+define_mask! {
+    /// A mask equivalent to [SimdI128](crate::SimdI128), where all bits in the lane must be either set
+    /// or unset.
+    struct SimdMask128<const LANES: usize>(crate::SimdI64<LANES>);
+}
+
+define_mask! {
+    /// A mask equivalent to [SimdIsize](crate::SimdIsize), where all bits in the lane must be either set
+    /// or unset.
+    struct SimdMaskSize<const LANES: usize>(crate::SimdI64<LANES>);
+}
diff --git a/crates/core_simd/src/masks/full_masks/mod.rs b/crates/core_simd/src/masks/full_masks/mod.rs
deleted file mode 100644
index eac5178512e5..000000000000
--- a/crates/core_simd/src/masks/full_masks/mod.rs
+++ /dev/null
@@ -1,205 +0,0 @@
-//! Masks that take up full SIMD vector registers.
-
-/// The error type returned when converting an integer to a mask fails.
-#[derive(Debug, Copy, Clone, PartialEq, Eq)]
-pub struct TryFromMaskError(());
-
-impl core::fmt::Display for TryFromMaskError {
-    fn fmt(&self, f: &mut core::fmt::Formatter) -> core::fmt::Result {
-        write!(f, "mask vector must have all bits set or unset in each lane")
-    }
-}
-
-macro_rules! define_mask {
-    { $(#[$attr:meta])* struct $name:ident<const $lanes:ident: usize>($type:ty); } => {
-        $(#[$attr])*
-        #[derive(Copy, Clone, Default, PartialEq, PartialOrd, Eq, Ord, Hash)]
-        #[repr(transparent)]
-        pub struct $name<const $lanes: usize>($type);
-
-        delegate_ops_to_inner! { $name }
-
-        impl<const $lanes: usize> $name<$lanes> {
-            /// Construct a mask by setting all lanes to the given value.
-            pub fn splat(value: bool) -> Self {
-                Self(<$type>::splat(
-                    if value {
-                        -1
-                    } else {
-                        0
-                    }
-                ))
-            }
-
-            /// Tests the value of the specified lane.
-            ///
-            /// # Panics
-            /// Panics if `lane` is greater than or equal to the number of lanes in the vector.
-            #[inline]
-            pub fn test(&self, lane: usize) -> bool {
-                self.0[lane] == -1
-            }
-
-            /// Sets the value of the specified lane.
-            ///
-            /// # Panics
-            /// Panics if `lane` is greater than or equal to the number of lanes in the vector.
-            #[inline]
-            pub fn set(&mut self, lane: usize, value: bool) {
-                self.0[lane] = if value {
-                    -1
-                } else {
-                    0
-                }
-            }
-        }
-
-        impl<const $lanes: usize> core::convert::From<bool> for $name<$lanes> {
-            fn from(value: bool) -> Self {
-                Self::splat(value)
-            }
-        }
-
-        impl<const $lanes: usize> core::convert::TryFrom<$type> for $name<$lanes> {
-            type Error = TryFromMaskError;
-            fn try_from(value: $type) -> Result<Self, Self::Error> {
-                if value.as_slice().iter().all(|x| *x == 0 || *x == -1) {
-                    Ok(Self(value))
-                } else {
-                    Err(TryFromMaskError(()))
-                }
-            }
-        }
-
-        impl<const $lanes: usize> core::convert::From<$name<$lanes>> for $type {
-            fn from(value: $name<$lanes>) -> Self {
-                value.0
-            }
-        }
-
-        impl<const $lanes: usize> core::fmt::Debug for $name<$lanes> {
-            fn fmt(&self, f: &mut core::fmt::Formatter) -> core::fmt::Result {
-                f.debug_list()
-                    .entries((0..LANES).map(|lane| self.test(lane)))
-                    .finish()
-            }
-        }
-
-        impl<const $lanes: usize> core::fmt::Binary for $name<$lanes> {
-            fn fmt(&self, f: &mut core::fmt::Formatter) -> core::fmt::Result {
-                core::fmt::Binary::fmt(&self.0, f)
-            }
-        }
-
-        impl<const $lanes: usize> core::fmt::Octal for $name<$lanes> {
-            fn fmt(&self, f: &mut core::fmt::Formatter) -> core::fmt::Result {
-                core::fmt::Octal::fmt(&self.0, f)
-            }
-        }
-
-        impl<const $lanes: usize> core::fmt::LowerHex for $name<$lanes> {
-            fn fmt(&self, f: &mut core::fmt::Formatter) -> core::fmt::Result {
-                core::fmt::LowerHex::fmt(&self.0, f)
-            }
-        }
-
-        impl<const $lanes: usize> core::fmt::UpperHex for $name<$lanes> {
-            fn fmt(&self, f: &mut core::fmt::Formatter) -> core::fmt::Result {
-                core::fmt::UpperHex::fmt(&self.0, f)
-            }
-        }
-    }
-}
-
-define_mask! {
-    /// A mask equivalent to [SimdI8](crate::SimdI8), where all bits in the lane must be either set
-    /// or unset.
-    struct SimdI8Mask<const LANES: usize>(crate::SimdI8<LANES>);
-}
-
-define_mask! {
-    /// A mask equivalent to [SimdI16](crate::SimdI16), where all bits in the lane must be either set
-    /// or unset.
-    struct SimdI16Mask<const LANES: usize>(crate::SimdI16<LANES>);
-}
-
-define_mask! {
-    /// A mask equivalent to [SimdI32](crate::SimdI32), where all bits in the lane must be either set
-    /// or unset.
-    struct SimdI32Mask<const LANES: usize>(crate::SimdI32<LANES>);
-}
-
-define_mask! {
-    /// A mask equivalent to [SimdI64](crate::SimdI64), where all bits in the lane must be either set
-    /// or unset.
-    struct SimdI64Mask<const LANES: usize>(crate::SimdI64<LANES>);
-}
-
-define_mask! {
-    /// A mask equivalent to [SimdI128](crate::SimdI128), where all bits in the lane must be either set
-    /// or unset.
-    struct SimdI128Mask<const LANES: usize>(crate::SimdI64<LANES>);
-}
-
-define_mask! {
-    /// A mask equivalent to [SimdIsize](crate::SimdIsize), where all bits in the lane must be either set
-    /// or unset.
-    struct SimdIsizeMask<const LANES: usize>(crate::SimdI64<LANES>);
-}
-
-macro_rules! implement_mask_ext {
-    { $($vector:ident => $mask:ident,)* } => {
-        $(
-            impl<const LANES: usize> crate::masks::MaskExt<$mask<LANES>> for crate::$vector<LANES> {
-                #[inline]
-                fn lanes_eq(&self, other: &Self) -> $mask<LANES> {
-                    unsafe { crate::intrinsics::simd_eq(self, other) }
-                }
-
-                #[inline]
-                fn lanes_ne(&self, other: &Self) -> $mask<LANES> {
-                    unsafe { crate::intrinsics::simd_ne(self, other) }
-                }
-
-                #[inline]
-                fn lanes_lt(&self, other: &Self) -> $mask<LANES> {
-                    unsafe { crate::intrinsics::simd_lt(self, other) }
-                }
-
-                #[inline]
-                fn lanes_gt(&self, other: &Self) -> $mask<LANES> {
-                    unsafe { crate::intrinsics::simd_gt(self, other) }
-                }
-
-                #[inline]
-                fn lanes_le(&self, other: &Self) -> $mask<LANES> {
-                    unsafe { crate::intrinsics::simd_le(self, other) }
-                }
-
-                #[inline]
-                fn lanes_ge(&self, other: &Self) -> $mask<LANES> {
-                    unsafe { crate::intrinsics::simd_ge(self, other) }
-                }
-            }
-        )*
-    }
-}
-
-implement_mask_ext! {
-    SimdI8 => SimdI8Mask,
-    SimdI16 => SimdI16Mask,
-    SimdI32 => SimdI32Mask,
-    SimdI64 => SimdI64Mask,
-    SimdI128 => SimdI128Mask,
-    SimdIsize => SimdIsizeMask,
-
-    SimdU8 => SimdI8Mask,
-    SimdU16 => SimdI16Mask,
-    SimdU32 => SimdI32Mask,
-    SimdU64 => SimdI64Mask,
-    SimdU128 => SimdI128Mask,
-    SimdUsize => SimdIsizeMask,
-
-    SimdF32 => SimdI32Mask,
-    SimdF64 => SimdI64Mask,
-}
diff --git a/crates/core_simd/src/masks/mod.rs b/crates/core_simd/src/masks/mod.rs
index a075040affe7..61e1199095fb 100644
--- a/crates/core_simd/src/masks/mod.rs
+++ b/crates/core_simd/src/masks/mod.rs
@@ -1,134 +1,11 @@
 //! Types and traits associated with masking lanes of vectors.
 #![allow(non_camel_case_types)]
 
-/// Implements bitwise ops on mask types by delegating the operators to the inner type.
-macro_rules! delegate_ops_to_inner {
-    { $name:ident } => {
-        impl<const LANES: usize> core::ops::BitAnd for $name<LANES> {
-            type Output = Self;
-            #[inline]
-            fn bitand(self, rhs: Self) -> Self {
-                Self(self.0 & rhs.0)
-            }
-        }
+mod full_masks;
+pub use full_masks::*;
 
-        impl<const LANES: usize> core::ops::BitAnd<bool> for $name<LANES> {
-            type Output = Self;
-            #[inline]
-            fn bitand(self, rhs: bool) -> Self {
-                self & Self::splat(rhs)
-            }
-        }
-
-        impl<const LANES: usize> core::ops::BitAnd<$name<LANES>> for bool {
-            type Output = $name<LANES>;
-            #[inline]
-            fn bitand(self, rhs: $name<LANES>) -> $name<LANES> {
-                $name::<LANES>::splat(self) & rhs
-            }
-        }
-
-        impl<const LANES: usize> core::ops::BitOr for $name<LANES> {
-            type Output = Self;
-            #[inline]
-            fn bitor(self, rhs: Self) -> Self {
-                Self(self.0 | rhs.0)
-            }
-        }
-
-        impl<const LANES: usize> core::ops::BitOr<bool> for $name<LANES> {
-            type Output = Self;
-            #[inline]
-            fn bitor(self, rhs: bool) -> Self {
-                self | Self::splat(rhs)
-            }
-        }
-
-        impl<const LANES: usize> core::ops::BitOr<$name<LANES>> for bool {
-            type Output = $name<LANES>;
-            #[inline]
-            fn bitor(self, rhs: $name<LANES>) -> $name<LANES> {
-                $name::<LANES>::splat(self) | rhs
-            }
-        }
-
-        impl<const LANES: usize> core::ops::BitXor for $name<LANES> {
-            type Output = Self;
-            #[inline]
-            fn bitxor(self, rhs: Self) -> Self::Output {
-                Self(self.0 ^ rhs.0)
-            }
-        }
-
-        impl<const LANES: usize> core::ops::BitXor<bool> for $name<LANES> {
-            type Output = Self;
-            #[inline]
-            fn bitxor(self, rhs: bool) -> Self::Output {
-                self ^ Self::splat(rhs)
-            }
-        }
-
-        impl<const LANES: usize> core::ops::BitXor<$name<LANES>> for bool {
-            type Output = $name<LANES>;
-            #[inline]
-            fn bitxor(self, rhs: $name<LANES>) -> Self::Output {
-                $name::<LANES>::splat(self) ^ rhs
-            }
-        }
-
-        impl<const LANES: usize> core::ops::Not for $name<LANES> {
-            type Output = $name<LANES>;
-            #[inline]
-            fn not(self) -> Self::Output {
-                Self(!self.0)
-            }
-        }
-
-        impl<const LANES: usize> core::ops::BitAndAssign for $name<LANES> {
-            #[inline]
-            fn bitand_assign(&mut self, rhs: Self) {
-                self.0 &= rhs.0;
-            }
-        }
-
-        impl<const LANES: usize> core::ops::BitAndAssign<bool> for $name<LANES> {
-            #[inline]
-            fn bitand_assign(&mut self, rhs: bool) {
-                *self &= Self::splat(rhs);
-            }
-        }
-
-        impl<const LANES: usize> core::ops::BitOrAssign for $name<LANES> {
-            #[inline]
-            fn bitor_assign(&mut self, rhs: Self) {
-                self.0 |= rhs.0;
-            }
-        }
-
-        impl<const LANES: usize> core::ops::BitOrAssign<bool> for $name<LANES> {
-            #[inline]
-            fn bitor_assign(&mut self, rhs: bool) {
-                *self |= Self::splat(rhs);
-            }
-        }
-
-        impl<const LANES: usize> core::ops::BitXorAssign for $name<LANES> {
-            #[inline]
-            fn bitxor_assign(&mut self, rhs: Self) {
-                self.0 ^= rhs.0;
-            }
-        }
-
-        impl<const LANES: usize> core::ops::BitXorAssign<bool> for $name<LANES> {
-            #[inline]
-            fn bitxor_assign(&mut self, rhs: bool) {
-                *self ^= Self::splat(rhs);
-            }
-        }
-    }
-}
-
-pub mod full_masks;
+mod bitmask;
+pub use bitmask::*;
 
 macro_rules! define_opaque_mask {
     {
@@ -137,11 +14,9 @@ macro_rules! define_opaque_mask {
     } => {
         $(#[$attr])*
         #[allow(non_camel_case_types)]
-        pub struct $name<const $lanes: usize>($inner_ty);
+        pub struct $name<const $lanes: usize>($inner_ty) where BitMask<LANES>: LanesAtMost64;
 
-        delegate_ops_to_inner! { $name }
-
-        impl<const $lanes: usize> $name<$lanes> {
+        impl<const $lanes: usize> $name<$lanes> where BitMask<$lanes>: LanesAtMost64 {
             /// Construct a mask by setting all lanes to the given value.
             pub fn splat(value: bool) -> Self {
                 Self(<$inner_ty>::splat(value))
@@ -166,41 +41,181 @@ macro_rules! define_opaque_mask {
             }
         }
 
-        impl<const $lanes: usize> Copy for $name<$lanes> {}
+        impl<const $lanes: usize> From<BitMask<$lanes>> for $name<$lanes>
+        where
+            BitMask<$lanes>: LanesAtMost64,
+        {
+            fn from(value: BitMask<$lanes>) -> Self {
+                Self(value.into())
+            }
+        }
 
-        impl<const $lanes: usize> Clone for $name<$lanes> {
+        impl<const $lanes: usize> From<$name<$lanes>> for crate::BitMask<$lanes>
+        where
+            BitMask<$lanes>: LanesAtMost64,
+        {
+            fn from(value: $name<$lanes>) -> Self {
+                value.0.into()
+            }
+        }
+
+        impl<const $lanes: usize> Copy for $name<$lanes> where BitMask<$lanes>: LanesAtMost64 {}
+
+        impl<const $lanes: usize> Clone for $name<$lanes> where BitMask<$lanes>: LanesAtMost64 {
             #[inline]
             fn clone(&self) -> Self {
                 *self
             }
         }
 
-        impl<const $lanes: usize> Default for $name<$lanes> {
+        impl<const $lanes: usize> Default for $name<$lanes> where BitMask<$lanes>: LanesAtMost64 {
             #[inline]
             fn default() -> Self {
                 Self::splat(false)
             }
         }
 
-        impl<const $lanes: usize> PartialEq for $name<$lanes> {
+        impl<const $lanes: usize> PartialEq for $name<$lanes> where BitMask<$lanes>: LanesAtMost64 {
             #[inline]
             fn eq(&self, other: &Self) -> bool {
                 self.0 == other.0
             }
         }
 
-        impl<const $lanes: usize> PartialOrd for $name<$lanes> {
+        impl<const $lanes: usize> PartialOrd for $name<$lanes> where BitMask<$lanes>: LanesAtMost64 {
             #[inline]
             fn partial_cmp(&self, other: &Self) -> Option<core::cmp::Ordering> {
                 self.0.partial_cmp(&other.0)
             }
         }
 
-        impl<const $lanes: usize> core::fmt::Debug for $name<$lanes> {
+        impl<const $lanes: usize> core::fmt::Debug for $name<$lanes> where BitMask<$lanes>: LanesAtMost64 {
             fn fmt(&self, f: &mut core::fmt::Formatter) -> core::fmt::Result {
                 core::fmt::Debug::fmt(&self.0, f)
             }
         }
+
+        impl<const LANES: usize> core::ops::BitAnd for $name<LANES> where BitMask<LANES>: LanesAtMost64 {
+            type Output = Self;
+            #[inline]
+            fn bitand(self, rhs: Self) -> Self {
+                Self(self.0 & rhs.0)
+            }
+        }
+
+        impl<const LANES: usize> core::ops::BitAnd<bool> for $name<LANES> where BitMask<LANES>: LanesAtMost64 {
+            type Output = Self;
+            #[inline]
+            fn bitand(self, rhs: bool) -> Self {
+                self & Self::splat(rhs)
+            }
+        }
+
+        impl<const LANES: usize> core::ops::BitAnd<$name<LANES>> for bool where BitMask<LANES>: LanesAtMost64 {
+            type Output = $name<LANES>;
+            #[inline]
+            fn bitand(self, rhs: $name<LANES>) -> $name<LANES> {
+                $name::<LANES>::splat(self) & rhs
+            }
+        }
+
+        impl<const LANES: usize> core::ops::BitOr for $name<LANES> where BitMask<LANES>: LanesAtMost64 {
+            type Output = Self;
+            #[inline]
+            fn bitor(self, rhs: Self) -> Self {
+                Self(self.0 | rhs.0)
+            }
+        }
+
+        impl<const LANES: usize> core::ops::BitOr<bool> for $name<LANES> where BitMask<LANES>: LanesAtMost64 {
+            type Output = Self;
+            #[inline]
+            fn bitor(self, rhs: bool) -> Self {
+                self | Self::splat(rhs)
+            }
+        }
+
+        impl<const LANES: usize> core::ops::BitOr<$name<LANES>> for bool where BitMask<LANES>: LanesAtMost64 {
+            type Output = $name<LANES>;
+            #[inline]
+            fn bitor(self, rhs: $name<LANES>) -> $name<LANES> {
+                $name::<LANES>::splat(self) | rhs
+            }
+        }
+
+        impl<const LANES: usize> core::ops::BitXor for $name<LANES> where BitMask<LANES>: LanesAtMost64 {
+            type Output = Self;
+            #[inline]
+            fn bitxor(self, rhs: Self) -> Self::Output {
+                Self(self.0 ^ rhs.0)
+            }
+        }
+
+        impl<const LANES: usize> core::ops::BitXor<bool> for $name<LANES> where BitMask<LANES>: LanesAtMost64 {
+            type Output = Self;
+            #[inline]
+            fn bitxor(self, rhs: bool) -> Self::Output {
+                self ^ Self::splat(rhs)
+            }
+        }
+
+        impl<const LANES: usize> core::ops::BitXor<$name<LANES>> for bool where BitMask<LANES>: LanesAtMost64 {
+            type Output = $name<LANES>;
+            #[inline]
+            fn bitxor(self, rhs: $name<LANES>) -> Self::Output {
+                $name::<LANES>::splat(self) ^ rhs
+            }
+        }
+
+        impl<const LANES: usize> core::ops::Not for $name<LANES> where BitMask<LANES>: LanesAtMost64 {
+            type Output = $name<LANES>;
+            #[inline]
+            fn not(self) -> Self::Output {
+                Self(!self.0)
+            }
+        }
+
+        impl<const LANES: usize> core::ops::BitAndAssign for $name<LANES> where BitMask<LANES>: LanesAtMost64 {
+            #[inline]
+            fn bitand_assign(&mut self, rhs: Self) {
+                self.0 &= rhs.0;
+            }
+        }
+
+        impl<const LANES: usize> core::ops::BitAndAssign<bool> for $name<LANES> where BitMask<LANES>: LanesAtMost64 {
+            #[inline]
+            fn bitand_assign(&mut self, rhs: bool) {
+                *self &= Self::splat(rhs);
+            }
+        }
+
+        impl<const LANES: usize> core::ops::BitOrAssign for $name<LANES> where BitMask<LANES>: LanesAtMost64 {
+            #[inline]
+            fn bitor_assign(&mut self, rhs: Self) {
+                self.0 |= rhs.0;
+            }
+        }
+
+        impl<const LANES: usize> core::ops::BitOrAssign<bool> for $name<LANES> where BitMask<LANES>: LanesAtMost64 {
+            #[inline]
+            fn bitor_assign(&mut self, rhs: bool) {
+                *self |= Self::splat(rhs);
+            }
+        }
+
+        impl<const LANES: usize> core::ops::BitXorAssign for $name<LANES> where BitMask<LANES>: LanesAtMost64 {
+            #[inline]
+            fn bitxor_assign(&mut self, rhs: Self) {
+                self.0 ^= rhs.0;
+            }
+        }
+
+        impl<const LANES: usize> core::ops::BitXorAssign<bool> for $name<LANES> where BitMask<LANES>: LanesAtMost64 {
+            #[inline]
+            fn bitxor_assign(&mut self, rhs: bool) {
+                *self ^= Self::splat(rhs);
+            }
+        }
     };
 }
 
@@ -208,103 +223,82 @@ define_opaque_mask! {
     /// Mask for vectors with `LANES` 8-bit elements.
     ///
     /// The layout of this type is unspecified.
-    struct Mask8<const LANES: usize>(full_masks::SimdI8Mask<LANES>);
+    struct Mask8<const LANES: usize>(SimdMask8<LANES>);
 }
 
 define_opaque_mask! {
     /// Mask for vectors with `LANES` 16-bit elements.
     ///
     /// The layout of this type is unspecified.
-    struct Mask16<const LANES: usize>(full_masks::SimdI16Mask<LANES>);
+    struct Mask16<const LANES: usize>(SimdMask16<LANES>);
 }
 
 define_opaque_mask! {
     /// Mask for vectors with `LANES` 32-bit elements.
     ///
     /// The layout of this type is unspecified.
-    struct Mask32<const LANES: usize>(full_masks::SimdI32Mask<LANES>);
+    struct Mask32<const LANES: usize>(SimdMask32<LANES>);
 }
 
 define_opaque_mask! {
     /// Mask for vectors with `LANES` 64-bit elements.
     ///
     /// The layout of this type is unspecified.
-    struct Mask64<const LANES: usize>(full_masks::SimdI64Mask<LANES>);
+    struct Mask64<const LANES: usize>(SimdMask64<LANES>);
 }
 
 define_opaque_mask! {
     /// Mask for vectors with `LANES` 128-bit elements.
     ///
     /// The layout of this type is unspecified.
-    struct Mask128<const LANES: usize>(full_masks::SimdI128Mask<LANES>);
+    struct Mask128<const LANES: usize>(SimdMask128<LANES>);
 }
 
 define_opaque_mask! {
     /// Mask for vectors with `LANES` pointer-width elements.
     ///
     /// The layout of this type is unspecified.
-    struct MaskSize<const LANES: usize>(full_masks::SimdIsizeMask<LANES>);
-}
-
-/// Mask-related operations using a particular mask layout.
-pub trait MaskExt<Mask> {
-    /// Test if each lane is equal to the corresponding lane in `other`.
-    fn lanes_eq(&self, other: &Self) -> Mask;
-
-    /// Test if each lane is not equal to the corresponding lane in `other`.
-    fn lanes_ne(&self, other: &Self) -> Mask;
-
-    /// Test if each lane is less than the corresponding lane in `other`.
-    fn lanes_lt(&self, other: &Self) -> Mask;
-
-    /// Test if each lane is greater than the corresponding lane in `other`.
-    fn lanes_gt(&self, other: &Self) -> Mask;
-
-    /// Test if each lane is less than or equal to the corresponding lane in `other`.
-    fn lanes_le(&self, other: &Self) -> Mask;
-
-    /// Test if each lane is greater than or equal to the corresponding lane in `other`.
-    fn lanes_ge(&self, other: &Self) -> Mask;
+    struct MaskSize<const LANES: usize>(SimdMaskSize<LANES>);
 }
 
 macro_rules! implement_mask_ops {
     { $($vector:ident => $mask:ident,)* } => {
         $(
-            impl<const LANES: usize> crate::$vector<LANES> {
+            impl<const LANES: usize> crate::$vector<LANES> where BitMask<LANES>: LanesAtMost64 {
                 /// Test if each lane is equal to the corresponding lane in `other`.
                 #[inline]
                 pub fn lanes_eq(&self, other: &Self) -> $mask<LANES> {
-                    $mask(MaskExt::lanes_eq(self, other))
+                    unsafe { $mask(crate::intrinsics::simd_eq(self, other)) }
                 }
 
                 /// Test if each lane is not equal to the corresponding lane in `other`.
                 #[inline]
                 pub fn lanes_ne(&self, other: &Self) -> $mask<LANES> {
-                    $mask(MaskExt::lanes_ne(self, other))
+                    unsafe { $mask(crate::intrinsics::simd_ne(self, other)) }
                 }
 
                 /// Test if each lane is less than the corresponding lane in `other`.
                 #[inline]
                 pub fn lanes_lt(&self, other: &Self) -> $mask<LANES> {
-                    $mask(MaskExt::lanes_lt(self, other))
+                    unsafe { $mask(crate::intrinsics::simd_lt(self, other)) }
                 }
 
                 /// Test if each lane is greater than the corresponding lane in `other`.
                 #[inline]
                 pub fn lanes_gt(&self, other: &Self) -> $mask<LANES> {
-                    $mask(MaskExt::lanes_gt(self, other))
+                    unsafe { $mask(crate::intrinsics::simd_gt(self, other)) }
                 }
 
                 /// Test if each lane is less than or equal to the corresponding lane in `other`.
                 #[inline]
                 pub fn lanes_le(&self, other: &Self) -> $mask<LANES> {
-                    $mask(MaskExt::lanes_le(self, other))
+                    unsafe { $mask(crate::intrinsics::simd_le(self, other)) }
                 }
 
                 /// Test if each lane is greater than or equal to the corresponding lane in `other`.
                 #[inline]
                 pub fn lanes_ge(&self, other: &Self) -> $mask<LANES> {
-                    $mask(MaskExt::lanes_ge(self, other))
+                    unsafe { $mask(crate::intrinsics::simd_ge(self, other)) }
                 }
             }
         )*

From 9e96c8a2a8336428aa1995e73901815af02e79cb Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Thu, 4 Feb 2021 20:47:13 -0500
Subject: [PATCH 078/249] Add missing From implementation, add simple mask API
 tests

---
 crates/core_simd/src/masks/mod.rs | 18 +++++++++
 crates/core_simd/tests/masks.rs   | 67 +++++++++++++++++++++++++++++++
 2 files changed, 85 insertions(+)
 create mode 100644 crates/core_simd/tests/masks.rs

diff --git a/crates/core_simd/src/masks/mod.rs b/crates/core_simd/src/masks/mod.rs
index 61e1199095fb..e51052f53f2f 100644
--- a/crates/core_simd/src/masks/mod.rs
+++ b/crates/core_simd/src/masks/mod.rs
@@ -59,6 +59,24 @@ macro_rules! define_opaque_mask {
             }
         }
 
+        impl<const $lanes: usize> From<$inner_ty> for $name<$lanes>
+        where
+            BitMask<$lanes>: LanesAtMost64,
+        {
+            fn from(value: $inner_ty) -> Self {
+                Self(value)
+            }
+        }
+
+        impl<const $lanes: usize> From<$name<$lanes>> for $inner_ty
+        where
+            BitMask<$lanes>: LanesAtMost64,
+        {
+            fn from(value: $name<$lanes>) -> Self {
+                value.0
+            }
+        }
+
         impl<const $lanes: usize> Copy for $name<$lanes> where BitMask<$lanes>: LanesAtMost64 {}
 
         impl<const $lanes: usize> Clone for $name<$lanes> where BitMask<$lanes>: LanesAtMost64 {
diff --git a/crates/core_simd/tests/masks.rs b/crates/core_simd/tests/masks.rs
new file mode 100644
index 000000000000..1ae003fc27fc
--- /dev/null
+++ b/crates/core_simd/tests/masks.rs
@@ -0,0 +1,67 @@
+use core_simd::{BitMask, Mask8, SimdMask8, SimdI8};
+use core::convert::TryFrom;
+
+#[cfg(target_arch = "wasm32")]
+use wasm_bindgen_test::*;
+
+#[cfg(target_arch = "wasm32")]
+wasm_bindgen_test_configure!(run_in_browser);
+
+#[test]
+#[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
+fn mask_format_round_trip() {
+    let ints = SimdI8::from_array([-1, 0, 0, -1]);
+
+    let simd_mask = SimdMask8::try_from(ints).unwrap();
+
+    let bitmask = BitMask::from(simd_mask);
+
+    let opaque_mask = Mask8::from(bitmask);
+
+    let simd_mask_returned = SimdMask8::from(opaque_mask);
+
+    let ints_returned = SimdI8::from(simd_mask_returned);
+
+    assert_eq!(ints_returned, ints);
+}
+
+macro_rules! test_mask_api {
+    { $name:ident } => {
+        #[allow(non_snake_case)]
+        mod $name {
+            #[test]
+            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
+            fn set_and_test() {
+                let values = [true, false, false, true, false, false, true, false];
+                let mut mask = core_simd::$name::<8>::splat(false);
+                for (lane, value) in values.iter().copied().enumerate() {
+                    mask.set(lane, value);
+                }
+                for (lane, value) in values.iter().copied().enumerate() {
+                    assert_eq!(mask.test(lane), value);
+                }
+            }
+
+            #[test]
+            #[should_panic]
+            fn set_invalid_lane() {
+                let mut mask = core_simd::$name::<8>::splat(false);
+                mask.set(8, true);
+                let _ = mask;
+            }
+
+            #[test]
+            #[should_panic]
+            fn test_invalid_lane() {
+                let mask = core_simd::$name::<8>::splat(false);
+                let _ = mask.test(8);
+            }
+        }
+    }
+}
+
+mod mask_api {
+    test_mask_api!{ Mask8 }
+    test_mask_api!{ SimdMask8 }
+    test_mask_api!{ BitMask }
+}

From 26061b4e84f833a100b4cd0978940997ecd72be3 Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Tue, 9 Feb 2021 00:09:27 -0500
Subject: [PATCH 079/249] Fix wasm tests

---
 crates/core_simd/tests/masks.rs | 11 +++++++----
 1 file changed, 7 insertions(+), 4 deletions(-)

diff --git a/crates/core_simd/tests/masks.rs b/crates/core_simd/tests/masks.rs
index 1ae003fc27fc..03a835b9c66f 100644
--- a/crates/core_simd/tests/masks.rs
+++ b/crates/core_simd/tests/masks.rs
@@ -1,5 +1,5 @@
-use core_simd::{BitMask, Mask8, SimdMask8, SimdI8};
 use core::convert::TryFrom;
+use core_simd::{BitMask, Mask8, SimdI8, SimdMask8};
 
 #[cfg(target_arch = "wasm32")]
 use wasm_bindgen_test::*;
@@ -29,6 +29,9 @@ macro_rules! test_mask_api {
     { $name:ident } => {
         #[allow(non_snake_case)]
         mod $name {
+            #[cfg(target_arch = "wasm32")]
+            use wasm_bindgen_test::*;
+
             #[test]
             #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn set_and_test() {
@@ -61,7 +64,7 @@ macro_rules! test_mask_api {
 }
 
 mod mask_api {
-    test_mask_api!{ Mask8 }
-    test_mask_api!{ SimdMask8 }
-    test_mask_api!{ BitMask }
+    test_mask_api! { Mask8 }
+    test_mask_api! { SimdMask8 }
+    test_mask_api! { BitMask }
 }

From 6362540f11aaa2f133c12cbb0b3ee903ea6f7f0e Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Tue, 9 Feb 2021 22:13:27 -0500
Subject: [PATCH 080/249] Limit all types to 64 lanes

---
 crates/core_simd/src/fmt.rs              |   5 +-
 crates/core_simd/src/lanes_at_most_64.rs |  35 +++++
 crates/core_simd/src/lib.rs              |   3 +
 crates/core_simd/src/macros.rs           |  36 +++--
 crates/core_simd/src/masks/bitmask.rs    |  10 +-
 crates/core_simd/src/masks/full_masks.rs | 152 ++++++++++++++----
 crates/core_simd/src/masks/mod.rs        | 169 +++++++++++++++-----
 crates/core_simd/src/ops.rs              | 191 ++++++++++++++++++-----
 crates/core_simd/src/vectors_u8.rs       |   2 +-
 9 files changed, 468 insertions(+), 135 deletions(-)
 create mode 100644 crates/core_simd/src/lanes_at_most_64.rs

diff --git a/crates/core_simd/src/fmt.rs b/crates/core_simd/src/fmt.rs
index 16b8f3b95d9a..6fa238cfda6a 100644
--- a/crates/core_simd/src/fmt.rs
+++ b/crates/core_simd/src/fmt.rs
@@ -33,7 +33,10 @@ macro_rules! impl_fmt_trait {
     { $($type:ident => $(($trait:ident, $format:ident)),*;)* } => {
         $( // repeat type
             $( // repeat trait
-                impl<const LANES: usize> core::fmt::$trait for crate::$type<LANES> {
+                impl<const LANES: usize> core::fmt::$trait for crate::$type<LANES>
+                where
+                    Self: crate::LanesAtMost64,
+                {
                     fn fmt(&self, f: &mut core::fmt::Formatter) -> core::fmt::Result {
                         $format(self.as_ref(), f)
                     }
diff --git a/crates/core_simd/src/lanes_at_most_64.rs b/crates/core_simd/src/lanes_at_most_64.rs
new file mode 100644
index 000000000000..63882152b6d9
--- /dev/null
+++ b/crates/core_simd/src/lanes_at_most_64.rs
@@ -0,0 +1,35 @@
+/// Implemented for bitmask sizes that are supported by the implementation.
+pub trait LanesAtMost64 {}
+
+macro_rules! impl_for {
+    { $name:ident } => {
+        impl LanesAtMost64 for $name<1> {}
+        impl LanesAtMost64 for $name<2> {}
+        impl LanesAtMost64 for $name<4> {}
+        impl LanesAtMost64 for $name<8> {}
+        impl LanesAtMost64 for $name<16> {}
+        impl LanesAtMost64 for $name<32> {}
+        impl LanesAtMost64 for $name<64> {}
+    }
+}
+
+use crate::*;
+
+impl_for! { SimdU8 }
+impl_for! { SimdU16 }
+impl_for! { SimdU32 }
+impl_for! { SimdU64 }
+impl_for! { SimdU128 }
+impl_for! { SimdUsize }
+
+impl_for! { SimdI8 }
+impl_for! { SimdI16 }
+impl_for! { SimdI32 }
+impl_for! { SimdI64 }
+impl_for! { SimdI128 }
+impl_for! { SimdIsize }
+
+impl_for! { SimdF32 }
+impl_for! { SimdF64 }
+
+impl_for! { BitMask }
diff --git a/crates/core_simd/src/lib.rs b/crates/core_simd/src/lib.rs
index de9cb566022e..e2e4864cc758 100644
--- a/crates/core_simd/src/lib.rs
+++ b/crates/core_simd/src/lib.rs
@@ -14,6 +14,9 @@ mod intrinsics;
 mod ops;
 mod round;
 
+mod lanes_at_most_64;
+pub use lanes_at_most_64::*;
+
 mod masks;
 pub use masks::*;
 
diff --git a/crates/core_simd/src/macros.rs b/crates/core_simd/src/macros.rs
index 0abafe71f50d..5328f22b42ab 100644
--- a/crates/core_simd/src/macros.rs
+++ b/crates/core_simd/src/macros.rs
@@ -29,7 +29,7 @@ macro_rules! from_transmute_x86 {
 /// Implements common traits on the specified vector `$name`, holding multiple `$lanes` of `$type`.
 macro_rules! impl_vector {
     { $name:ident, $type:ty } => {
-        impl<const LANES: usize> $name<LANES> {
+        impl<const LANES: usize> $name<LANES> where Self: crate::LanesAtMost64 {
             /// Construct a SIMD vector by setting all lanes to the given value.
             pub const fn splat(value: $type) -> Self {
                 Self([value; LANES])
@@ -72,23 +72,23 @@ macro_rules! impl_vector {
             }
         }
 
-        impl<const LANES: usize> Copy for $name<LANES> {}
+        impl<const LANES: usize> Copy for $name<LANES> where Self: crate::LanesAtMost64 {}
 
-        impl<const LANES: usize> Clone for $name<LANES> {
+        impl<const LANES: usize> Clone for $name<LANES> where Self: crate::LanesAtMost64 {
             #[inline]
             fn clone(&self) -> Self {
                 *self
             }
         }
 
-        impl<const LANES: usize> Default for $name<LANES> {
+        impl<const LANES: usize> Default for $name<LANES> where Self: crate::LanesAtMost64 {
             #[inline]
             fn default() -> Self {
                 Self::splat(<$type>::default())
             }
         }
 
-        impl<const LANES: usize> PartialEq for $name<LANES> {
+        impl<const LANES: usize> PartialEq for $name<LANES> where Self: crate::LanesAtMost64 {
             #[inline]
             fn eq(&self, other: &Self) -> bool {
                 // TODO use SIMD equality
@@ -96,7 +96,7 @@ macro_rules! impl_vector {
             }
         }
 
-        impl<const LANES: usize> PartialOrd for $name<LANES> {
+        impl<const LANES: usize> PartialOrd for $name<LANES> where Self: crate::LanesAtMost64 {
             #[inline]
             fn partial_cmp(&self, other: &Self) -> Option<core::cmp::Ordering> {
                 // TODO use SIMD equalitya
@@ -105,14 +105,14 @@ macro_rules! impl_vector {
         }
 
         // array references
-        impl<const LANES: usize> AsRef<[$type; LANES]> for $name<LANES> {
+        impl<const LANES: usize> AsRef<[$type; LANES]> for $name<LANES> where Self: crate::LanesAtMost64 {
             #[inline]
             fn as_ref(&self) -> &[$type; LANES] {
                 &self.0
             }
         }
 
-        impl<const LANES: usize> AsMut<[$type; LANES]> for $name<LANES> {
+        impl<const LANES: usize> AsMut<[$type; LANES]> for $name<LANES> where Self: crate::LanesAtMost64 {
             #[inline]
             fn as_mut(&mut self) -> &mut [$type; LANES] {
                 &mut self.0
@@ -120,14 +120,14 @@ macro_rules! impl_vector {
         }
 
         // slice references
-        impl<const LANES: usize> AsRef<[$type]> for $name<LANES> {
+        impl<const LANES: usize> AsRef<[$type]> for $name<LANES> where Self: crate::LanesAtMost64 {
             #[inline]
             fn as_ref(&self) -> &[$type] {
                 &self.0
             }
         }
 
-        impl<const LANES: usize> AsMut<[$type]> for $name<LANES> {
+        impl<const LANES: usize> AsMut<[$type]> for $name<LANES> where Self: crate::LanesAtMost64 {
             #[inline]
             fn as_mut(&mut self) -> &mut [$type] {
                 &mut self.0
@@ -135,14 +135,14 @@ macro_rules! impl_vector {
         }
 
         // vector/array conversion
-        impl<const LANES: usize> From<[$type; LANES]> for $name<LANES> {
+        impl<const LANES: usize> From<[$type; LANES]> for $name<LANES> where Self: crate::LanesAtMost64 {
             fn from(array: [$type; LANES]) -> Self {
                 Self(array)
             }
         }
 
         // splat
-        impl<const LANES: usize> From<$type> for $name<LANES> {
+        impl<const LANES: usize> From<$type> for $name<LANES> where Self: crate::LanesAtMost64 {
             #[inline]
             fn from(value: $type) -> Self {
                 Self::splat(value)
@@ -158,9 +158,9 @@ macro_rules! impl_integer_vector {
     { $name:ident, $type:ty } => {
         impl_vector! { $name, $type }
 
-        impl<const LANES: usize> Eq for $name<LANES> {}
+        impl<const LANES: usize> Eq for $name<LANES> where Self: crate::LanesAtMost64 {}
 
-        impl<const LANES: usize> Ord for $name<LANES> {
+        impl<const LANES: usize> Ord for $name<LANES> where Self: crate::LanesAtMost64 {
             #[inline]
             fn cmp(&self, other: &Self) -> core::cmp::Ordering {
                 // TODO use SIMD cmp
@@ -168,7 +168,7 @@ macro_rules! impl_integer_vector {
             }
         }
 
-        impl<const LANES: usize> core::hash::Hash for $name<LANES> {
+        impl<const LANES: usize> core::hash::Hash for $name<LANES> where Self: crate::LanesAtMost64 {
             #[inline]
             fn hash<H>(&self, state: &mut H)
             where
@@ -187,7 +187,11 @@ macro_rules! impl_float_vector {
     { $name:ident, $type:ty, $bits_ty:ident } => {
         impl_vector! { $name, $type }
 
-        impl<const LANES: usize> $name<LANES> {
+        impl<const LANES: usize> $name<LANES>
+        where
+            Self: crate::LanesAtMost64,
+            crate::$bits_ty<LANES>: crate::LanesAtMost64,
+        {
             /// Raw transmutation to an unsigned integer vector type with the
             /// same size and number of lanes.
             #[inline]
diff --git a/crates/core_simd/src/masks/bitmask.rs b/crates/core_simd/src/masks/bitmask.rs
index 51ed8037043d..b9b1160a3f38 100644
--- a/crates/core_simd/src/masks/bitmask.rs
+++ b/crates/core_simd/src/masks/bitmask.rs
@@ -1,12 +1,4 @@
-/// Implemented for bitmask sizes that are supported by the implementation.
-pub trait LanesAtMost64 {}
-impl LanesAtMost64 for BitMask<1> {}
-impl LanesAtMost64 for BitMask<2> {}
-impl LanesAtMost64 for BitMask<4> {}
-impl LanesAtMost64 for BitMask<8> {}
-impl LanesAtMost64 for BitMask<16> {}
-impl LanesAtMost64 for BitMask<32> {}
-impl LanesAtMost64 for BitMask<64> {}
+use crate::LanesAtMost64;
 
 /// A mask where each lane is represented by a single bit.
 #[derive(Copy, Clone, Debug)]
diff --git a/crates/core_simd/src/masks/full_masks.rs b/crates/core_simd/src/masks/full_masks.rs
index d7c4af47727c..fa93d252df46 100644
--- a/crates/core_simd/src/masks/full_masks.rs
+++ b/crates/core_simd/src/masks/full_masks.rs
@@ -16,11 +16,31 @@ impl core::fmt::Display for TryFromMaskError {
 macro_rules! define_mask {
     { $(#[$attr:meta])* struct $name:ident<const $lanes:ident: usize>($type:ty); } => {
         $(#[$attr])*
-        #[derive(Copy, Clone, Default, PartialEq, PartialOrd, Eq, Ord, Hash)]
+        #[derive(Default, PartialEq, PartialOrd, Eq, Ord, Hash)]
         #[repr(transparent)]
-        pub struct $name<const $lanes: usize>($type);
+        pub struct $name<const $lanes: usize>($type)
+        where
+            $type: crate::LanesAtMost64;
 
-        impl<const $lanes: usize> $name<$lanes> {
+        impl<const LANES: usize> Copy for $name<LANES>
+        where
+            $type: crate::LanesAtMost64,
+        {}
+
+        impl<const LANES: usize> Clone for $name<LANES>
+        where
+            $type: crate::LanesAtMost64,
+        {
+            #[inline]
+            fn clone(&self) -> Self {
+                *self
+            }
+        }
+
+        impl<const $lanes: usize> $name<$lanes>
+        where
+            $type: crate::LanesAtMost64,
+        {
             /// Construct a mask by setting all lanes to the given value.
             pub fn splat(value: bool) -> Self {
                 Self(<$type>::splat(
@@ -57,13 +77,19 @@ macro_rules! define_mask {
             }
         }
 
-        impl<const $lanes: usize> core::convert::From<bool> for $name<$lanes> {
+        impl<const $lanes: usize> core::convert::From<bool> for $name<$lanes>
+        where
+            $type: crate::LanesAtMost64,
+        {
             fn from(value: bool) -> Self {
                 Self::splat(value)
             }
         }
 
-        impl<const $lanes: usize> core::convert::TryFrom<$type> for $name<$lanes> {
+        impl<const $lanes: usize> core::convert::TryFrom<$type> for $name<$lanes>
+        where
+            $type: crate::LanesAtMost64,
+        {
             type Error = TryFromMaskError;
             fn try_from(value: $type) -> Result<Self, Self::Error> {
                 if value.as_slice().iter().all(|x| *x == 0 || *x == -1) {
@@ -74,7 +100,10 @@ macro_rules! define_mask {
             }
         }
 
-        impl<const $lanes: usize> core::convert::From<$name<$lanes>> for $type {
+        impl<const $lanes: usize> core::convert::From<$name<$lanes>> for $type
+        where
+            $type: crate::LanesAtMost64,
+        {
             fn from(value: $name<$lanes>) -> Self {
                 value.0
             }
@@ -82,6 +111,7 @@ macro_rules! define_mask {
 
         impl<const $lanes: usize> core::convert::From<crate::BitMask<$lanes>> for $name<$lanes>
         where
+            $type: crate::LanesAtMost64,
             crate::BitMask<$lanes>: crate::LanesAtMost64,
         {
             fn from(value: crate::BitMask<$lanes>) -> Self {
@@ -96,6 +126,7 @@ macro_rules! define_mask {
 
         impl<const $lanes: usize> core::convert::From<$name<$lanes>> for crate::BitMask<$lanes>
         where
+            $type: crate::LanesAtMost64,
             crate::BitMask<$lanes>: crate::LanesAtMost64,
         {
             fn from(value: $name<$lanes>) -> Self {
@@ -108,7 +139,10 @@ macro_rules! define_mask {
             }
         }
 
-        impl<const $lanes: usize> core::fmt::Debug for $name<$lanes> {
+        impl<const $lanes: usize> core::fmt::Debug for $name<$lanes>
+        where
+            $type: crate::LanesAtMost64,
+        {
             fn fmt(&self, f: &mut core::fmt::Formatter) -> core::fmt::Result {
                 f.debug_list()
                     .entries((0..LANES).map(|lane| self.test(lane)))
@@ -116,31 +150,46 @@ macro_rules! define_mask {
             }
         }
 
-        impl<const $lanes: usize> core::fmt::Binary for $name<$lanes> {
+        impl<const $lanes: usize> core::fmt::Binary for $name<$lanes>
+        where
+            $type: crate::LanesAtMost64,
+        {
             fn fmt(&self, f: &mut core::fmt::Formatter) -> core::fmt::Result {
                 core::fmt::Binary::fmt(&self.0, f)
             }
         }
 
-        impl<const $lanes: usize> core::fmt::Octal for $name<$lanes> {
+        impl<const $lanes: usize> core::fmt::Octal for $name<$lanes>
+        where
+            $type: crate::LanesAtMost64,
+        {
             fn fmt(&self, f: &mut core::fmt::Formatter) -> core::fmt::Result {
                 core::fmt::Octal::fmt(&self.0, f)
             }
         }
 
-        impl<const $lanes: usize> core::fmt::LowerHex for $name<$lanes> {
+        impl<const $lanes: usize> core::fmt::LowerHex for $name<$lanes>
+        where
+            $type: crate::LanesAtMost64,
+        {
             fn fmt(&self, f: &mut core::fmt::Formatter) -> core::fmt::Result {
                 core::fmt::LowerHex::fmt(&self.0, f)
             }
         }
 
-        impl<const $lanes: usize> core::fmt::UpperHex for $name<$lanes> {
+        impl<const $lanes: usize> core::fmt::UpperHex for $name<$lanes>
+        where
+            $type: crate::LanesAtMost64,
+        {
             fn fmt(&self, f: &mut core::fmt::Formatter) -> core::fmt::Result {
                 core::fmt::UpperHex::fmt(&self.0, f)
             }
         }
 
-        impl<const LANES: usize> core::ops::BitAnd for $name<LANES> {
+        impl<const LANES: usize> core::ops::BitAnd for $name<LANES>
+        where
+            $type: crate::LanesAtMost64,
+        {
             type Output = Self;
             #[inline]
             fn bitand(self, rhs: Self) -> Self {
@@ -148,7 +197,10 @@ macro_rules! define_mask {
             }
         }
 
-        impl<const LANES: usize> core::ops::BitAnd<bool> for $name<LANES> {
+        impl<const LANES: usize> core::ops::BitAnd<bool> for $name<LANES>
+        where
+            $type: crate::LanesAtMost64,
+        {
             type Output = Self;
             #[inline]
             fn bitand(self, rhs: bool) -> Self {
@@ -156,7 +208,10 @@ macro_rules! define_mask {
             }
         }
 
-        impl<const LANES: usize> core::ops::BitAnd<$name<LANES>> for bool {
+        impl<const LANES: usize> core::ops::BitAnd<$name<LANES>> for bool
+        where
+            $type: crate::LanesAtMost64,
+        {
             type Output = $name<LANES>;
             #[inline]
             fn bitand(self, rhs: $name<LANES>) -> $name<LANES> {
@@ -164,7 +219,10 @@ macro_rules! define_mask {
             }
         }
 
-        impl<const LANES: usize> core::ops::BitOr for $name<LANES> {
+        impl<const LANES: usize> core::ops::BitOr for $name<LANES>
+        where
+            $type: crate::LanesAtMost64,
+        {
             type Output = Self;
             #[inline]
             fn bitor(self, rhs: Self) -> Self {
@@ -172,7 +230,10 @@ macro_rules! define_mask {
             }
         }
 
-        impl<const LANES: usize> core::ops::BitOr<bool> for $name<LANES> {
+        impl<const LANES: usize> core::ops::BitOr<bool> for $name<LANES>
+        where
+            $type: crate::LanesAtMost64,
+        {
             type Output = Self;
             #[inline]
             fn bitor(self, rhs: bool) -> Self {
@@ -180,7 +241,10 @@ macro_rules! define_mask {
             }
         }
 
-        impl<const LANES: usize> core::ops::BitOr<$name<LANES>> for bool {
+        impl<const LANES: usize> core::ops::BitOr<$name<LANES>> for bool
+        where
+            $type: crate::LanesAtMost64,
+        {
             type Output = $name<LANES>;
             #[inline]
             fn bitor(self, rhs: $name<LANES>) -> $name<LANES> {
@@ -188,7 +252,10 @@ macro_rules! define_mask {
             }
         }
 
-        impl<const LANES: usize> core::ops::BitXor for $name<LANES> {
+        impl<const LANES: usize> core::ops::BitXor for $name<LANES>
+        where
+            $type: crate::LanesAtMost64,
+        {
             type Output = Self;
             #[inline]
             fn bitxor(self, rhs: Self) -> Self::Output {
@@ -196,7 +263,10 @@ macro_rules! define_mask {
             }
         }
 
-        impl<const LANES: usize> core::ops::BitXor<bool> for $name<LANES> {
+        impl<const LANES: usize> core::ops::BitXor<bool> for $name<LANES>
+        where
+            $type: crate::LanesAtMost64,
+        {
             type Output = Self;
             #[inline]
             fn bitxor(self, rhs: bool) -> Self::Output {
@@ -204,7 +274,10 @@ macro_rules! define_mask {
             }
         }
 
-        impl<const LANES: usize> core::ops::BitXor<$name<LANES>> for bool {
+        impl<const LANES: usize> core::ops::BitXor<$name<LANES>> for bool
+        where
+            $type: crate::LanesAtMost64,
+        {
             type Output = $name<LANES>;
             #[inline]
             fn bitxor(self, rhs: $name<LANES>) -> Self::Output {
@@ -212,7 +285,10 @@ macro_rules! define_mask {
             }
         }
 
-        impl<const LANES: usize> core::ops::Not for $name<LANES> {
+        impl<const LANES: usize> core::ops::Not for $name<LANES>
+        where
+            $type: crate::LanesAtMost64,
+        {
             type Output = $name<LANES>;
             #[inline]
             fn not(self) -> Self::Output {
@@ -220,42 +296,60 @@ macro_rules! define_mask {
             }
         }
 
-        impl<const LANES: usize> core::ops::BitAndAssign for $name<LANES> {
+        impl<const LANES: usize> core::ops::BitAndAssign for $name<LANES>
+        where
+            $type: crate::LanesAtMost64,
+        {
             #[inline]
             fn bitand_assign(&mut self, rhs: Self) {
                 self.0 &= rhs.0;
             }
         }
 
-        impl<const LANES: usize> core::ops::BitAndAssign<bool> for $name<LANES> {
+        impl<const LANES: usize> core::ops::BitAndAssign<bool> for $name<LANES>
+        where
+            $type: crate::LanesAtMost64,
+        {
             #[inline]
             fn bitand_assign(&mut self, rhs: bool) {
                 *self &= Self::splat(rhs);
             }
         }
 
-        impl<const LANES: usize> core::ops::BitOrAssign for $name<LANES> {
+        impl<const LANES: usize> core::ops::BitOrAssign for $name<LANES>
+        where
+            $type: crate::LanesAtMost64,
+        {
             #[inline]
             fn bitor_assign(&mut self, rhs: Self) {
                 self.0 |= rhs.0;
             }
         }
 
-        impl<const LANES: usize> core::ops::BitOrAssign<bool> for $name<LANES> {
+        impl<const LANES: usize> core::ops::BitOrAssign<bool> for $name<LANES>
+        where
+            $type: crate::LanesAtMost64,
+        {
             #[inline]
             fn bitor_assign(&mut self, rhs: bool) {
                 *self |= Self::splat(rhs);
             }
         }
 
-        impl<const LANES: usize> core::ops::BitXorAssign for $name<LANES> {
+        impl<const LANES: usize> core::ops::BitXorAssign for $name<LANES>
+        where
+            $type: crate::LanesAtMost64,
+        {
             #[inline]
             fn bitxor_assign(&mut self, rhs: Self) {
                 self.0 ^= rhs.0;
             }
         }
 
-        impl<const LANES: usize> core::ops::BitXorAssign<bool> for $name<LANES> {
+        impl<const LANES: usize> core::ops::BitXorAssign<bool> for $name<LANES>
+        where
+            $type: crate::LanesAtMost64,
+        {
             #[inline]
             fn bitxor_assign(&mut self, rhs: bool) {
                 *self ^= Self::splat(rhs);
@@ -291,11 +385,11 @@ define_mask! {
 define_mask! {
     /// A mask equivalent to [SimdI128](crate::SimdI128), where all bits in the lane must be either set
     /// or unset.
-    struct SimdMask128<const LANES: usize>(crate::SimdI64<LANES>);
+    struct SimdMask128<const LANES: usize>(crate::SimdI128<LANES>);
 }
 
 define_mask! {
     /// A mask equivalent to [SimdIsize](crate::SimdIsize), where all bits in the lane must be either set
     /// or unset.
-    struct SimdMaskSize<const LANES: usize>(crate::SimdI64<LANES>);
+    struct SimdMaskSize<const LANES: usize>(crate::SimdIsize<LANES>);
 }
diff --git a/crates/core_simd/src/masks/mod.rs b/crates/core_simd/src/masks/mod.rs
index e51052f53f2f..7d7f7af627df 100644
--- a/crates/core_simd/src/masks/mod.rs
+++ b/crates/core_simd/src/masks/mod.rs
@@ -7,16 +7,22 @@ pub use full_masks::*;
 mod bitmask;
 pub use bitmask::*;
 
+use crate::LanesAtMost64;
+
 macro_rules! define_opaque_mask {
     {
         $(#[$attr:meta])*
         struct $name:ident<const $lanes:ident: usize>($inner_ty:ty);
+        @bits $bits_ty:ty
     } => {
         $(#[$attr])*
         #[allow(non_camel_case_types)]
-        pub struct $name<const $lanes: usize>($inner_ty) where BitMask<LANES>: LanesAtMost64;
+        pub struct $name<const $lanes: usize>($inner_ty) where $bits_ty: LanesAtMost64;
 
-        impl<const $lanes: usize> $name<$lanes> where BitMask<$lanes>: LanesAtMost64 {
+        impl<const $lanes: usize> $name<$lanes>
+        where
+            $bits_ty: LanesAtMost64
+        {
             /// Construct a mask by setting all lanes to the given value.
             pub fn splat(value: bool) -> Self {
                 Self(<$inner_ty>::splat(value))
@@ -43,6 +49,7 @@ macro_rules! define_opaque_mask {
 
         impl<const $lanes: usize> From<BitMask<$lanes>> for $name<$lanes>
         where
+            $bits_ty: LanesAtMost64,
             BitMask<$lanes>: LanesAtMost64,
         {
             fn from(value: BitMask<$lanes>) -> Self {
@@ -52,6 +59,7 @@ macro_rules! define_opaque_mask {
 
         impl<const $lanes: usize> From<$name<$lanes>> for crate::BitMask<$lanes>
         where
+            $bits_ty: LanesAtMost64,
             BitMask<$lanes>: LanesAtMost64,
         {
             fn from(value: $name<$lanes>) -> Self {
@@ -61,7 +69,7 @@ macro_rules! define_opaque_mask {
 
         impl<const $lanes: usize> From<$inner_ty> for $name<$lanes>
         where
-            BitMask<$lanes>: LanesAtMost64,
+            $bits_ty: LanesAtMost64,
         {
             fn from(value: $inner_ty) -> Self {
                 Self(value)
@@ -70,50 +78,72 @@ macro_rules! define_opaque_mask {
 
         impl<const $lanes: usize> From<$name<$lanes>> for $inner_ty
         where
-            BitMask<$lanes>: LanesAtMost64,
+            $bits_ty: LanesAtMost64,
         {
             fn from(value: $name<$lanes>) -> Self {
                 value.0
             }
         }
 
-        impl<const $lanes: usize> Copy for $name<$lanes> where BitMask<$lanes>: LanesAtMost64 {}
+        impl<const $lanes: usize> Copy for $name<$lanes>
+        where
+            $inner_ty: Copy,
+            $bits_ty: LanesAtMost64,
+        {}
 
-        impl<const $lanes: usize> Clone for $name<$lanes> where BitMask<$lanes>: LanesAtMost64 {
+        impl<const $lanes: usize> Clone for $name<$lanes>
+        where
+            $bits_ty: LanesAtMost64,
+        {
             #[inline]
             fn clone(&self) -> Self {
                 *self
             }
         }
 
-        impl<const $lanes: usize> Default for $name<$lanes> where BitMask<$lanes>: LanesAtMost64 {
+        impl<const $lanes: usize> Default for $name<$lanes>
+        where
+            $bits_ty: LanesAtMost64,
+        {
             #[inline]
             fn default() -> Self {
                 Self::splat(false)
             }
         }
 
-        impl<const $lanes: usize> PartialEq for $name<$lanes> where BitMask<$lanes>: LanesAtMost64 {
+        impl<const $lanes: usize> PartialEq for $name<$lanes>
+        where
+            $bits_ty: LanesAtMost64,
+        {
             #[inline]
             fn eq(&self, other: &Self) -> bool {
                 self.0 == other.0
             }
         }
 
-        impl<const $lanes: usize> PartialOrd for $name<$lanes> where BitMask<$lanes>: LanesAtMost64 {
+        impl<const $lanes: usize> PartialOrd for $name<$lanes>
+        where
+            $bits_ty: LanesAtMost64,
+        {
             #[inline]
             fn partial_cmp(&self, other: &Self) -> Option<core::cmp::Ordering> {
                 self.0.partial_cmp(&other.0)
             }
         }
 
-        impl<const $lanes: usize> core::fmt::Debug for $name<$lanes> where BitMask<$lanes>: LanesAtMost64 {
+        impl<const $lanes: usize> core::fmt::Debug for $name<$lanes>
+        where
+            $bits_ty: LanesAtMost64,
+        {
             fn fmt(&self, f: &mut core::fmt::Formatter) -> core::fmt::Result {
                 core::fmt::Debug::fmt(&self.0, f)
             }
         }
 
-        impl<const LANES: usize> core::ops::BitAnd for $name<LANES> where BitMask<LANES>: LanesAtMost64 {
+        impl<const LANES: usize> core::ops::BitAnd for $name<LANES>
+        where
+            $bits_ty: LanesAtMost64,
+        {
             type Output = Self;
             #[inline]
             fn bitand(self, rhs: Self) -> Self {
@@ -121,7 +151,10 @@ macro_rules! define_opaque_mask {
             }
         }
 
-        impl<const LANES: usize> core::ops::BitAnd<bool> for $name<LANES> where BitMask<LANES>: LanesAtMost64 {
+        impl<const LANES: usize> core::ops::BitAnd<bool> for $name<LANES>
+        where
+            $bits_ty: LanesAtMost64,
+        {
             type Output = Self;
             #[inline]
             fn bitand(self, rhs: bool) -> Self {
@@ -129,7 +162,10 @@ macro_rules! define_opaque_mask {
             }
         }
 
-        impl<const LANES: usize> core::ops::BitAnd<$name<LANES>> for bool where BitMask<LANES>: LanesAtMost64 {
+        impl<const LANES: usize> core::ops::BitAnd<$name<LANES>> for bool
+        where
+            $bits_ty: LanesAtMost64,
+        {
             type Output = $name<LANES>;
             #[inline]
             fn bitand(self, rhs: $name<LANES>) -> $name<LANES> {
@@ -137,7 +173,10 @@ macro_rules! define_opaque_mask {
             }
         }
 
-        impl<const LANES: usize> core::ops::BitOr for $name<LANES> where BitMask<LANES>: LanesAtMost64 {
+        impl<const LANES: usize> core::ops::BitOr for $name<LANES>
+        where
+            $bits_ty: LanesAtMost64,
+        {
             type Output = Self;
             #[inline]
             fn bitor(self, rhs: Self) -> Self {
@@ -145,7 +184,10 @@ macro_rules! define_opaque_mask {
             }
         }
 
-        impl<const LANES: usize> core::ops::BitOr<bool> for $name<LANES> where BitMask<LANES>: LanesAtMost64 {
+        impl<const LANES: usize> core::ops::BitOr<bool> for $name<LANES>
+        where
+            $bits_ty: LanesAtMost64,
+        {
             type Output = Self;
             #[inline]
             fn bitor(self, rhs: bool) -> Self {
@@ -153,7 +195,10 @@ macro_rules! define_opaque_mask {
             }
         }
 
-        impl<const LANES: usize> core::ops::BitOr<$name<LANES>> for bool where BitMask<LANES>: LanesAtMost64 {
+        impl<const LANES: usize> core::ops::BitOr<$name<LANES>> for bool
+        where
+            $bits_ty: LanesAtMost64,
+        {
             type Output = $name<LANES>;
             #[inline]
             fn bitor(self, rhs: $name<LANES>) -> $name<LANES> {
@@ -161,7 +206,10 @@ macro_rules! define_opaque_mask {
             }
         }
 
-        impl<const LANES: usize> core::ops::BitXor for $name<LANES> where BitMask<LANES>: LanesAtMost64 {
+        impl<const LANES: usize> core::ops::BitXor for $name<LANES>
+        where
+            $bits_ty: LanesAtMost64,
+        {
             type Output = Self;
             #[inline]
             fn bitxor(self, rhs: Self) -> Self::Output {
@@ -169,7 +217,10 @@ macro_rules! define_opaque_mask {
             }
         }
 
-        impl<const LANES: usize> core::ops::BitXor<bool> for $name<LANES> where BitMask<LANES>: LanesAtMost64 {
+        impl<const LANES: usize> core::ops::BitXor<bool> for $name<LANES>
+        where
+            $bits_ty: LanesAtMost64,
+        {
             type Output = Self;
             #[inline]
             fn bitxor(self, rhs: bool) -> Self::Output {
@@ -177,7 +228,10 @@ macro_rules! define_opaque_mask {
             }
         }
 
-        impl<const LANES: usize> core::ops::BitXor<$name<LANES>> for bool where BitMask<LANES>: LanesAtMost64 {
+        impl<const LANES: usize> core::ops::BitXor<$name<LANES>> for bool
+        where
+            $bits_ty: LanesAtMost64,
+        {
             type Output = $name<LANES>;
             #[inline]
             fn bitxor(self, rhs: $name<LANES>) -> Self::Output {
@@ -185,7 +239,10 @@ macro_rules! define_opaque_mask {
             }
         }
 
-        impl<const LANES: usize> core::ops::Not for $name<LANES> where BitMask<LANES>: LanesAtMost64 {
+        impl<const LANES: usize> core::ops::Not for $name<LANES>
+        where
+            $bits_ty: LanesAtMost64,
+        {
             type Output = $name<LANES>;
             #[inline]
             fn not(self) -> Self::Output {
@@ -193,42 +250,60 @@ macro_rules! define_opaque_mask {
             }
         }
 
-        impl<const LANES: usize> core::ops::BitAndAssign for $name<LANES> where BitMask<LANES>: LanesAtMost64 {
+        impl<const LANES: usize> core::ops::BitAndAssign for $name<LANES>
+        where
+            $bits_ty: LanesAtMost64,
+        {
             #[inline]
             fn bitand_assign(&mut self, rhs: Self) {
                 self.0 &= rhs.0;
             }
         }
 
-        impl<const LANES: usize> core::ops::BitAndAssign<bool> for $name<LANES> where BitMask<LANES>: LanesAtMost64 {
+        impl<const LANES: usize> core::ops::BitAndAssign<bool> for $name<LANES>
+        where
+            $bits_ty: LanesAtMost64,
+        {
             #[inline]
             fn bitand_assign(&mut self, rhs: bool) {
                 *self &= Self::splat(rhs);
             }
         }
 
-        impl<const LANES: usize> core::ops::BitOrAssign for $name<LANES> where BitMask<LANES>: LanesAtMost64 {
+        impl<const LANES: usize> core::ops::BitOrAssign for $name<LANES>
+        where
+            $bits_ty: LanesAtMost64,
+        {
             #[inline]
             fn bitor_assign(&mut self, rhs: Self) {
                 self.0 |= rhs.0;
             }
         }
 
-        impl<const LANES: usize> core::ops::BitOrAssign<bool> for $name<LANES> where BitMask<LANES>: LanesAtMost64 {
+        impl<const LANES: usize> core::ops::BitOrAssign<bool> for $name<LANES>
+        where
+            $bits_ty: LanesAtMost64,
+        {
             #[inline]
             fn bitor_assign(&mut self, rhs: bool) {
                 *self |= Self::splat(rhs);
             }
         }
 
-        impl<const LANES: usize> core::ops::BitXorAssign for $name<LANES> where BitMask<LANES>: LanesAtMost64 {
+        impl<const LANES: usize> core::ops::BitXorAssign for $name<LANES>
+        where
+            $bits_ty: LanesAtMost64,
+        {
             #[inline]
             fn bitxor_assign(&mut self, rhs: Self) {
                 self.0 ^= rhs.0;
             }
         }
 
-        impl<const LANES: usize> core::ops::BitXorAssign<bool> for $name<LANES> where BitMask<LANES>: LanesAtMost64 {
+        impl<const LANES: usize> core::ops::BitXorAssign<bool> for $name<LANES>
+        where
+            $bits_ty: LanesAtMost64,
+        {
             #[inline]
             fn bitxor_assign(&mut self, rhs: bool) {
                 *self ^= Self::splat(rhs);
@@ -242,6 +317,7 @@ define_opaque_mask! {
     ///
     /// The layout of this type is unspecified.
     struct Mask8<const LANES: usize>(SimdMask8<LANES>);
+    @bits crate::SimdI8<LANES>
 }
 
 define_opaque_mask! {
@@ -249,6 +325,7 @@ define_opaque_mask! {
     ///
     /// The layout of this type is unspecified.
     struct Mask16<const LANES: usize>(SimdMask16<LANES>);
+    @bits crate::SimdI16<LANES>
 }
 
 define_opaque_mask! {
@@ -256,6 +333,7 @@ define_opaque_mask! {
     ///
     /// The layout of this type is unspecified.
     struct Mask32<const LANES: usize>(SimdMask32<LANES>);
+    @bits crate::SimdI32<LANES>
 }
 
 define_opaque_mask! {
@@ -263,6 +341,7 @@ define_opaque_mask! {
     ///
     /// The layout of this type is unspecified.
     struct Mask64<const LANES: usize>(SimdMask64<LANES>);
+    @bits crate::SimdI64<LANES>
 }
 
 define_opaque_mask! {
@@ -270,6 +349,7 @@ define_opaque_mask! {
     ///
     /// The layout of this type is unspecified.
     struct Mask128<const LANES: usize>(SimdMask128<LANES>);
+    @bits crate::SimdI128<LANES>
 }
 
 define_opaque_mask! {
@@ -277,12 +357,17 @@ define_opaque_mask! {
     ///
     /// The layout of this type is unspecified.
     struct MaskSize<const LANES: usize>(SimdMaskSize<LANES>);
+    @bits crate::SimdIsize<LANES>
 }
 
 macro_rules! implement_mask_ops {
-    { $($vector:ident => $mask:ident,)* } => {
+    { $($vector:ident => $mask:ident ($inner_ty:ident),)* } => {
         $(
-            impl<const LANES: usize> crate::$vector<LANES> where BitMask<LANES>: LanesAtMost64 {
+            impl<const LANES: usize> crate::$vector<LANES>
+            where
+                crate::$vector<LANES>: LanesAtMost64,
+                crate::$inner_ty<LANES>: LanesAtMost64,
+            {
                 /// Test if each lane is equal to the corresponding lane in `other`.
                 #[inline]
                 pub fn lanes_eq(&self, other: &Self) -> $mask<LANES> {
@@ -324,22 +409,22 @@ macro_rules! implement_mask_ops {
 }
 
 implement_mask_ops! {
-    SimdI8 => Mask8,
-    SimdI16 => Mask16,
-    SimdI32 => Mask32,
-    SimdI64 => Mask64,
-    SimdI128 => Mask128,
-    SimdIsize => MaskSize,
+    SimdI8 => Mask8 (SimdI8),
+    SimdI16 => Mask16 (SimdI16),
+    SimdI32 => Mask32 (SimdI32),
+    SimdI64 => Mask64 (SimdI64),
+    SimdI128 => Mask128 (SimdI128),
+    SimdIsize => MaskSize (SimdIsize),
 
-    SimdU8 => Mask8,
-    SimdU16 => Mask16,
-    SimdU32 => Mask32,
-    SimdU64 => Mask64,
-    SimdU128 => Mask128,
-    SimdUsize => MaskSize,
+    SimdU8 => Mask8 (SimdI8),
+    SimdU16 => Mask16 (SimdI16),
+    SimdU32 => Mask32 (SimdI32),
+    SimdU64 => Mask64 (SimdI64),
+    SimdU128 => Mask128 (SimdI128),
+    SimdUsize => MaskSize (SimdIsize),
 
-    SimdF32 => Mask32,
-    SimdF64 => Mask64,
+    SimdF32 => Mask32 (SimdI32),
+    SimdF64 => Mask64 (SimdI64),
 }
 
 /// Vector of eight 8-bit masks
diff --git a/crates/core_simd/src/ops.rs b/crates/core_simd/src/ops.rs
index ae7c86c81c50..1d9e1eeb92ce 100644
--- a/crates/core_simd/src/ops.rs
+++ b/crates/core_simd/src/ops.rs
@@ -1,3 +1,5 @@
+use crate::LanesAtMost64;
+
 /// Checks if the right-hand side argument of a left- or right-shift would cause overflow.
 fn invalid_shift_rhs<T>(rhs: T) -> bool
 where
@@ -12,21 +14,30 @@ where
 macro_rules! impl_ref_ops {
     // binary op
     {
-        impl<const $lanes:ident: usize> core::ops::$trait:ident<$rhs:ty> for $type:ty {
+        impl<const $lanes:ident: usize> core::ops::$trait:ident<$rhs:ty> for $type:ty
+        where
+            $($bound:path: LanesAtMost64,)*
+        {
             type Output = $output:ty;
 
             $(#[$attrs:meta])*
             fn $fn:ident($self_tok:ident, $rhs_arg:ident: $rhs_arg_ty:ty) -> Self::Output $body:tt
         }
     } => {
-        impl<const $lanes: usize> core::ops::$trait<$rhs> for $type {
+        impl<const $lanes: usize> core::ops::$trait<$rhs> for $type
+        where
+            $($bound: LanesAtMost64,)*
+        {
             type Output = $output;
 
             $(#[$attrs])*
             fn $fn($self_tok, $rhs_arg: $rhs_arg_ty) -> Self::Output $body
         }
 
-        impl<const $lanes: usize> core::ops::$trait<&'_ $rhs> for $type {
+        impl<const $lanes: usize> core::ops::$trait<&'_ $rhs> for $type
+        where
+            $($bound: LanesAtMost64,)*
+        {
             type Output = <$type as core::ops::$trait<$rhs>>::Output;
 
             $(#[$attrs])*
@@ -35,7 +46,10 @@ macro_rules! impl_ref_ops {
             }
         }
 
-        impl<const $lanes: usize> core::ops::$trait<$rhs> for &'_ $type {
+        impl<const $lanes: usize> core::ops::$trait<$rhs> for &'_ $type
+        where
+            $($bound: LanesAtMost64,)*
+        {
             type Output = <$type as core::ops::$trait<$rhs>>::Output;
 
             $(#[$attrs])*
@@ -44,7 +58,10 @@ macro_rules! impl_ref_ops {
             }
         }
 
-        impl<const $lanes: usize> core::ops::$trait<&'_ $rhs> for &'_ $type {
+        impl<const $lanes: usize> core::ops::$trait<&'_ $rhs> for &'_ $type
+        where
+            $($bound: LanesAtMost64,)*
+        {
             type Output = <$type as core::ops::$trait<$rhs>>::Output;
 
             $(#[$attrs])*
@@ -56,17 +73,26 @@ macro_rules! impl_ref_ops {
 
     // binary assignment op
     {
-        impl<const $lanes:ident: usize> core::ops::$trait:ident<$rhs:ty> for $type:ty {
+        impl<const $lanes:ident: usize> core::ops::$trait:ident<$rhs:ty> for $type:ty
+        where
+            $($bound:path: LanesAtMost64,)*
+        {
             $(#[$attrs:meta])*
             fn $fn:ident(&mut $self_tok:ident, $rhs_arg:ident: $rhs_arg_ty:ty) $body:tt
         }
     } => {
-        impl<const $lanes: usize> core::ops::$trait<$rhs> for $type {
+        impl<const $lanes: usize> core::ops::$trait<$rhs> for $type
+        where
+            $($bound: LanesAtMost64,)*
+        {
             $(#[$attrs])*
             fn $fn(&mut $self_tok, $rhs_arg: $rhs_arg_ty) $body
         }
 
-        impl<const $lanes: usize> core::ops::$trait<&'_ $rhs> for $type {
+        impl<const $lanes: usize> core::ops::$trait<&'_ $rhs> for $type
+        where
+            $($bound: LanesAtMost64,)*
+        {
             $(#[$attrs])*
             fn $fn(&mut $self_tok, $rhs_arg: &$rhs_arg_ty) {
                 core::ops::$trait::$fn($self_tok, *$rhs_arg)
@@ -76,17 +102,26 @@ macro_rules! impl_ref_ops {
 
     // unary op
     {
-        impl<const $lanes:ident: usize> core::ops::$trait:ident for $type:ty {
+        impl<const $lanes:ident: usize> core::ops::$trait:ident for $type:ty
+        where
+            $($bound:path: LanesAtMost64,)*
+        {
             type Output = $output:ty;
             fn $fn:ident($self_tok:ident) -> Self::Output $body:tt
         }
     } => {
-        impl<const $lanes: usize> core::ops::$trait for $type {
+        impl<const $lanes: usize> core::ops::$trait for $type
+        where
+            $($bound: LanesAtMost64,)*
+        {
             type Output = $output;
             fn $fn($self_tok) -> Self::Output $body
         }
 
-        impl<const $lanes: usize> core::ops::$trait for &'_ $type {
+        impl<const $lanes: usize> core::ops::$trait for &'_ $type
+        where
+            $($bound: LanesAtMost64,)*
+        {
             type Output = <$type as core::ops::$trait>::Output;
             fn $fn($self_tok) -> Self::Output {
                 core::ops::$trait::$fn(*$self_tok)
@@ -130,7 +165,10 @@ macro_rules! impl_op {
 
     { impl Not for $type:ident, $scalar:ty } => {
         impl_ref_ops! {
-            impl<const LANES: usize> core::ops::Not for crate::$type<LANES> {
+            impl<const LANES: usize> core::ops::Not for crate::$type<LANES>
+            where
+                crate::$type<LANES>: LanesAtMost64,
+            {
                 type Output = Self;
                 fn not(self) -> Self::Output {
                     self ^ Self::splat(!<$scalar>::default())
@@ -141,7 +179,10 @@ macro_rules! impl_op {
 
     { impl Neg for $type:ident, $scalar:ty } => {
         impl_ref_ops! {
-            impl<const LANES: usize> core::ops::Neg for crate::$type<LANES> {
+            impl<const LANES: usize> core::ops::Neg for crate::$type<LANES>
+            where
+                crate::$type<LANES>: LanesAtMost64,
+            {
                 type Output = Self;
                 fn neg(self) -> Self::Output {
                     Self::splat(0) - self
@@ -152,7 +193,12 @@ macro_rules! impl_op {
 
     { impl Neg for $type:ident, $scalar:ty, @float } => {
         impl_ref_ops! {
-            impl<const LANES: usize> core::ops::Neg for crate::$type<LANES> {
+            impl<const LANES: usize> core::ops::Neg for crate::$type<LANES>
+            where
+                crate::$type<LANES>: LanesAtMost64,
+                crate::SimdU32<LANES>: LanesAtMost64,
+                crate::SimdU64<LANES>: LanesAtMost64,
+            {
                 type Output = Self;
                 fn neg(self) -> Self::Output {
                     // FIXME: Replace this with fneg intrinsic once available.
@@ -166,6 +212,7 @@ macro_rules! impl_op {
     { impl Index for $type:ident, $scalar:ty } => {
         impl<I, const LANES: usize> core::ops::Index<I> for crate::$type<LANES>
         where
+            Self: LanesAtMost64,
             I: core::slice::SliceIndex<[$scalar]>,
         {
             type Output = I::Output;
@@ -177,6 +224,7 @@ macro_rules! impl_op {
 
         impl<I, const LANES: usize> core::ops::IndexMut<I> for crate::$type<LANES>
         where
+            Self: LanesAtMost64,
             I: core::slice::SliceIndex<[$scalar]>,
         {
             fn index_mut(&mut self, index: I) -> &mut Self::Output {
@@ -189,7 +237,10 @@ macro_rules! impl_op {
     // generic binary op with assignment when output is `Self`
     { @binary $type:ident, $scalar:ty, $trait:ident :: $trait_fn:ident, $assign_trait:ident :: $assign_trait_fn:ident, $intrinsic:ident } => {
         impl_ref_ops! {
-            impl<const LANES: usize> core::ops::$trait<Self> for crate::$type<LANES> {
+            impl<const LANES: usize> core::ops::$trait<Self> for crate::$type<LANES>
+            where
+                crate::$type<LANES>: LanesAtMost64,
+            {
                 type Output = Self;
 
                 #[inline]
@@ -202,7 +253,10 @@ macro_rules! impl_op {
         }
 
         impl_ref_ops! {
-            impl<const LANES: usize> core::ops::$trait<$scalar> for crate::$type<LANES> {
+            impl<const LANES: usize> core::ops::$trait<$scalar> for crate::$type<LANES>
+            where
+                crate::$type<LANES>: LanesAtMost64,
+            {
                 type Output = Self;
 
                 #[inline]
@@ -213,7 +267,10 @@ macro_rules! impl_op {
         }
 
         impl_ref_ops! {
-            impl<const LANES: usize> core::ops::$trait<crate::$type<LANES>> for $scalar {
+            impl<const LANES: usize> core::ops::$trait<crate::$type<LANES>> for $scalar
+            where
+                crate::$type<LANES>: LanesAtMost64,
+            {
                 type Output = crate::$type<LANES>;
 
                 #[inline]
@@ -224,7 +281,10 @@ macro_rules! impl_op {
         }
 
         impl_ref_ops! {
-            impl<const LANES: usize> core::ops::$assign_trait<Self> for crate::$type<LANES> {
+            impl<const LANES: usize> core::ops::$assign_trait<Self> for crate::$type<LANES>
+            where
+                crate::$type<LANES>: LanesAtMost64,
+            {
                 #[inline]
                 fn $assign_trait_fn(&mut self, rhs: Self) {
                     unsafe {
@@ -235,7 +295,10 @@ macro_rules! impl_op {
         }
 
         impl_ref_ops! {
-            impl<const LANES: usize> core::ops::$assign_trait<$scalar> for crate::$type<LANES> {
+            impl<const LANES: usize> core::ops::$assign_trait<$scalar> for crate::$type<LANES>
+            where
+                crate::$type<LANES>: LanesAtMost64,
+            {
                 #[inline]
                 fn $assign_trait_fn(&mut self, rhs: $scalar) {
                     core::ops::$assign_trait::$assign_trait_fn(self, Self::splat(rhs));
@@ -278,7 +341,10 @@ macro_rules! impl_unsigned_int_ops {
 
                 // Integers panic on divide by 0
                 impl_ref_ops! {
-                    impl<const LANES: usize> core::ops::Div<Self> for crate::$vector<LANES> {
+                    impl<const LANES: usize> core::ops::Div<Self> for crate::$vector<LANES>
+                    where
+                        crate::$vector<LANES>: LanesAtMost64,
+                    {
                         type Output = Self;
 
                         #[inline]
@@ -303,7 +369,10 @@ macro_rules! impl_unsigned_int_ops {
                 }
 
                 impl_ref_ops! {
-                    impl<const LANES: usize> core::ops::Div<$scalar> for crate::$vector<LANES> {
+                    impl<const LANES: usize> core::ops::Div<$scalar> for crate::$vector<LANES>
+                    where
+                        crate::$vector<LANES>: LanesAtMost64,
+                    {
                         type Output = Self;
 
                         #[inline]
@@ -323,7 +392,10 @@ macro_rules! impl_unsigned_int_ops {
                 }
 
                 impl_ref_ops! {
-                    impl<const LANES: usize> core::ops::Div<crate::$vector<LANES>> for $scalar {
+                    impl<const LANES: usize> core::ops::Div<crate::$vector<LANES>> for $scalar
+                    where
+                        crate::$vector<LANES>: LanesAtMost64,
+                    {
                         type Output = crate::$vector<LANES>;
 
                         #[inline]
@@ -334,7 +406,10 @@ macro_rules! impl_unsigned_int_ops {
                 }
 
                 impl_ref_ops! {
-                    impl<const LANES: usize> core::ops::DivAssign<Self> for crate::$vector<LANES> {
+                    impl<const LANES: usize> core::ops::DivAssign<Self> for crate::$vector<LANES>
+                    where
+                        crate::$vector<LANES>: LanesAtMost64,
+                    {
                         #[inline]
                         fn div_assign(&mut self, rhs: Self) {
                             *self = *self / rhs;
@@ -343,7 +418,10 @@ macro_rules! impl_unsigned_int_ops {
                 }
 
                 impl_ref_ops! {
-                    impl<const LANES: usize> core::ops::DivAssign<$scalar> for crate::$vector<LANES> {
+                    impl<const LANES: usize> core::ops::DivAssign<$scalar> for crate::$vector<LANES>
+                    where
+                        crate::$vector<LANES>: LanesAtMost64,
+                    {
                         #[inline]
                         fn div_assign(&mut self, rhs: $scalar) {
                             *self = *self / rhs;
@@ -353,7 +431,10 @@ macro_rules! impl_unsigned_int_ops {
 
                 // remainder panics on zero divisor
                 impl_ref_ops! {
-                    impl<const LANES: usize> core::ops::Rem<Self> for crate::$vector<LANES> {
+                    impl<const LANES: usize> core::ops::Rem<Self> for crate::$vector<LANES>
+                    where
+                        crate::$vector<LANES>: LanesAtMost64,
+                    {
                         type Output = Self;
 
                         #[inline]
@@ -378,7 +459,10 @@ macro_rules! impl_unsigned_int_ops {
                 }
 
                 impl_ref_ops! {
-                    impl<const LANES: usize> core::ops::Rem<$scalar> for crate::$vector<LANES> {
+                    impl<const LANES: usize> core::ops::Rem<$scalar> for crate::$vector<LANES>
+                    where
+                        crate::$vector<LANES>: LanesAtMost64,
+                    {
                         type Output = Self;
 
                         #[inline]
@@ -398,7 +482,10 @@ macro_rules! impl_unsigned_int_ops {
                 }
 
                 impl_ref_ops! {
-                    impl<const LANES: usize> core::ops::Rem<crate::$vector<LANES>> for $scalar {
+                    impl<const LANES: usize> core::ops::Rem<crate::$vector<LANES>> for $scalar
+                    where
+                        crate::$vector<LANES>: LanesAtMost64,
+                    {
                         type Output = crate::$vector<LANES>;
 
                         #[inline]
@@ -409,7 +496,10 @@ macro_rules! impl_unsigned_int_ops {
                 }
 
                 impl_ref_ops! {
-                    impl<const LANES: usize> core::ops::RemAssign<Self> for crate::$vector<LANES> {
+                    impl<const LANES: usize> core::ops::RemAssign<Self> for crate::$vector<LANES>
+                    where
+                        crate::$vector<LANES>: LanesAtMost64,
+                    {
                         #[inline]
                         fn rem_assign(&mut self, rhs: Self) {
                             *self = *self % rhs;
@@ -418,7 +508,10 @@ macro_rules! impl_unsigned_int_ops {
                 }
 
                 impl_ref_ops! {
-                    impl<const LANES: usize> core::ops::RemAssign<$scalar> for crate::$vector<LANES> {
+                    impl<const LANES: usize> core::ops::RemAssign<$scalar> for crate::$vector<LANES>
+                    where
+                        crate::$vector<LANES>: LanesAtMost64,
+                    {
                         #[inline]
                         fn rem_assign(&mut self, rhs: $scalar) {
                             *self = *self % rhs;
@@ -428,7 +521,10 @@ macro_rules! impl_unsigned_int_ops {
 
                 // shifts panic on overflow
                 impl_ref_ops! {
-                    impl<const LANES: usize> core::ops::Shl<Self> for crate::$vector<LANES> {
+                    impl<const LANES: usize> core::ops::Shl<Self> for crate::$vector<LANES>
+                    where
+                        crate::$vector<LANES>: LanesAtMost64,
+                    {
                         type Output = Self;
 
                         #[inline]
@@ -447,7 +543,10 @@ macro_rules! impl_unsigned_int_ops {
                 }
 
                 impl_ref_ops! {
-                    impl<const LANES: usize> core::ops::Shl<$scalar> for crate::$vector<LANES> {
+                    impl<const LANES: usize> core::ops::Shl<$scalar> for crate::$vector<LANES>
+                    where
+                        crate::$vector<LANES>: LanesAtMost64,
+                    {
                         type Output = Self;
 
                         #[inline]
@@ -463,7 +562,10 @@ macro_rules! impl_unsigned_int_ops {
 
 
                 impl_ref_ops! {
-                    impl<const LANES: usize> core::ops::ShlAssign<Self> for crate::$vector<LANES> {
+                    impl<const LANES: usize> core::ops::ShlAssign<Self> for crate::$vector<LANES>
+                    where
+                        crate::$vector<LANES>: LanesAtMost64,
+                    {
                         #[inline]
                         fn shl_assign(&mut self, rhs: Self) {
                             *self = *self << rhs;
@@ -472,7 +574,10 @@ macro_rules! impl_unsigned_int_ops {
                 }
 
                 impl_ref_ops! {
-                    impl<const LANES: usize> core::ops::ShlAssign<$scalar> for crate::$vector<LANES> {
+                    impl<const LANES: usize> core::ops::ShlAssign<$scalar> for crate::$vector<LANES>
+                    where
+                        crate::$vector<LANES>: LanesAtMost64,
+                    {
                         #[inline]
                         fn shl_assign(&mut self, rhs: $scalar) {
                             *self = *self << rhs;
@@ -481,7 +586,10 @@ macro_rules! impl_unsigned_int_ops {
                 }
 
                 impl_ref_ops! {
-                    impl<const LANES: usize> core::ops::Shr<Self> for crate::$vector<LANES> {
+                    impl<const LANES: usize> core::ops::Shr<Self> for crate::$vector<LANES>
+                    where
+                        crate::$vector<LANES>: LanesAtMost64,
+                    {
                         type Output = Self;
 
                         #[inline]
@@ -500,7 +608,10 @@ macro_rules! impl_unsigned_int_ops {
                 }
 
                 impl_ref_ops! {
-                    impl<const LANES: usize> core::ops::Shr<$scalar> for crate::$vector<LANES> {
+                    impl<const LANES: usize> core::ops::Shr<$scalar> for crate::$vector<LANES>
+                    where
+                        crate::$vector<LANES>: LanesAtMost64,
+                    {
                         type Output = Self;
 
                         #[inline]
@@ -516,7 +627,10 @@ macro_rules! impl_unsigned_int_ops {
 
 
                 impl_ref_ops! {
-                    impl<const LANES: usize> core::ops::ShrAssign<Self> for crate::$vector<LANES> {
+                    impl<const LANES: usize> core::ops::ShrAssign<Self> for crate::$vector<LANES>
+                    where
+                        crate::$vector<LANES>: LanesAtMost64,
+                    {
                         #[inline]
                         fn shr_assign(&mut self, rhs: Self) {
                             *self = *self >> rhs;
@@ -525,7 +639,10 @@ macro_rules! impl_unsigned_int_ops {
                 }
 
                 impl_ref_ops! {
-                    impl<const LANES: usize> core::ops::ShrAssign<$scalar> for crate::$vector<LANES> {
+                    impl<const LANES: usize> core::ops::ShrAssign<$scalar> for crate::$vector<LANES>
+                    where
+                        crate::$vector<LANES>: LanesAtMost64,
+                    {
                         #[inline]
                         fn shr_assign(&mut self, rhs: $scalar) {
                             *self = *self >> rhs;
diff --git a/crates/core_simd/src/vectors_u8.rs b/crates/core_simd/src/vectors_u8.rs
index 9cc4eaca47ad..6cf623f68013 100644
--- a/crates/core_simd/src/vectors_u8.rs
+++ b/crates/core_simd/src/vectors_u8.rs
@@ -2,7 +2,7 @@
 
 /// A SIMD vector of containing `LANES` `u8` values.
 #[repr(simd)]
-pub struct SimdU8<const LANES: usize>([u8; LANES]);
+pub struct SimdU8<const LANES: usize>([u8; LANES]) where Self: crate::LanesAtMost64;
 
 impl_integer_vector! { SimdU8, u8 }
 

From 16904ebfc752af858141c7b72df00f6bb3ddc303 Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Sat, 13 Feb 2021 00:49:51 -0500
Subject: [PATCH 081/249] Add missing type bounds

---
 crates/core_simd/src/round.rs         | 11 ++++++++++-
 crates/core_simd/src/vectors_f32.rs   |  4 +++-
 crates/core_simd/src/vectors_f64.rs   |  4 +++-
 crates/core_simd/src/vectors_i128.rs  |  4 +++-
 crates/core_simd/src/vectors_i16.rs   |  4 +++-
 crates/core_simd/src/vectors_i32.rs   |  4 +++-
 crates/core_simd/src/vectors_i64.rs   |  4 +++-
 crates/core_simd/src/vectors_i8.rs    |  4 +++-
 crates/core_simd/src/vectors_isize.rs |  4 +++-
 crates/core_simd/src/vectors_u128.rs  |  4 +++-
 crates/core_simd/src/vectors_u16.rs   |  4 +++-
 crates/core_simd/src/vectors_u32.rs   |  4 +++-
 crates/core_simd/src/vectors_u64.rs   |  4 +++-
 crates/core_simd/src/vectors_usize.rs |  4 +++-
 14 files changed, 49 insertions(+), 14 deletions(-)

diff --git a/crates/core_simd/src/round.rs b/crates/core_simd/src/round.rs
index d77bc4e8fa7e..ee232e2b2225 100644
--- a/crates/core_simd/src/round.rs
+++ b/crates/core_simd/src/round.rs
@@ -2,7 +2,10 @@ macro_rules! implement {
     {
         $type:ident, $int_type:ident
     } => {
-        impl<const LANES: usize> crate::$type<LANES> {
+        impl<const LANES: usize> crate::$type<LANES>
+        where
+            Self: crate::LanesAtMost64,
+        {
             /// Returns the largest integer less than or equal to each lane.
             #[must_use = "method returns a new vector and does not mutate the original value"]
             #[inline]
@@ -16,7 +19,13 @@ macro_rules! implement {
             pub fn ceil(self) -> Self {
                 unsafe { crate::intrinsics::simd_ceil(self) }
             }
+        }
 
+        impl<const LANES: usize> crate::$type<LANES>
+        where
+            Self: crate::LanesAtMost64,
+            crate::$int_type<LANES>: crate::LanesAtMost64,
+        {
             /// Rounds toward zero and converts to the same-width integer type, assuming that
             /// the value is finite and fits in that type.
             ///
diff --git a/crates/core_simd/src/vectors_f32.rs b/crates/core_simd/src/vectors_f32.rs
index 0b5d8c6ec49b..5bb8f3a1c34d 100644
--- a/crates/core_simd/src/vectors_f32.rs
+++ b/crates/core_simd/src/vectors_f32.rs
@@ -2,7 +2,9 @@
 
 /// A SIMD vector of containing `LANES` `f32` values.
 #[repr(simd)]
-pub struct SimdF32<const LANES: usize>([f32; LANES]);
+pub struct SimdF32<const LANES: usize>([f32; LANES])
+where
+    Self: crate::LanesAtMost64;
 
 impl_float_vector! { SimdF32, f32, SimdU32 }
 
diff --git a/crates/core_simd/src/vectors_f64.rs b/crates/core_simd/src/vectors_f64.rs
index 307f8a4acacd..c0dca6a52ac6 100644
--- a/crates/core_simd/src/vectors_f64.rs
+++ b/crates/core_simd/src/vectors_f64.rs
@@ -2,7 +2,9 @@
 
 /// A SIMD vector of containing `LANES` `f64` values.
 #[repr(simd)]
-pub struct SimdF64<const LANES: usize>([f64; LANES]);
+pub struct SimdF64<const LANES: usize>([f64; LANES])
+where
+    Self: crate::LanesAtMost64;
 
 impl_float_vector! { SimdF64, f64, SimdU64 }
 
diff --git a/crates/core_simd/src/vectors_i128.rs b/crates/core_simd/src/vectors_i128.rs
index 16e6162be552..568fa81da80e 100644
--- a/crates/core_simd/src/vectors_i128.rs
+++ b/crates/core_simd/src/vectors_i128.rs
@@ -2,7 +2,9 @@
 
 /// A SIMD vector of containing `LANES` `i128` values.
 #[repr(simd)]
-pub struct SimdI128<const LANES: usize>([i128; LANES]);
+pub struct SimdI128<const LANES: usize>([i128; LANES])
+where
+    Self: crate::LanesAtMost64;
 
 impl_integer_vector! { SimdI128, i128 }
 
diff --git a/crates/core_simd/src/vectors_i16.rs b/crates/core_simd/src/vectors_i16.rs
index 08cc4af2a5ea..d77e593a2edc 100644
--- a/crates/core_simd/src/vectors_i16.rs
+++ b/crates/core_simd/src/vectors_i16.rs
@@ -2,7 +2,9 @@
 
 /// A SIMD vector of containing `LANES` `i16` values.
 #[repr(simd)]
-pub struct SimdI16<const LANES: usize>([i16; LANES]);
+pub struct SimdI16<const LANES: usize>([i16; LANES])
+where
+    Self: crate::LanesAtMost64;
 
 impl_integer_vector! { SimdI16, i16 }
 
diff --git a/crates/core_simd/src/vectors_i32.rs b/crates/core_simd/src/vectors_i32.rs
index 116f2abaeeed..0a89eeda3b2f 100644
--- a/crates/core_simd/src/vectors_i32.rs
+++ b/crates/core_simd/src/vectors_i32.rs
@@ -2,7 +2,9 @@
 
 /// A SIMD vector of containing `LANES` `i32` values.
 #[repr(simd)]
-pub struct SimdI32<const LANES: usize>([i32; LANES]);
+pub struct SimdI32<const LANES: usize>([i32; LANES])
+where
+    Self: crate::LanesAtMost64;
 
 impl_integer_vector! { SimdI32, i32 }
 
diff --git a/crates/core_simd/src/vectors_i64.rs b/crates/core_simd/src/vectors_i64.rs
index 6a1e2094179b..017140654a51 100644
--- a/crates/core_simd/src/vectors_i64.rs
+++ b/crates/core_simd/src/vectors_i64.rs
@@ -2,7 +2,9 @@
 
 /// A SIMD vector of containing `LANES` `i64` values.
 #[repr(simd)]
-pub struct SimdI64<const LANES: usize>([i64; LANES]);
+pub struct SimdI64<const LANES: usize>([i64; LANES])
+where
+    Self: crate::LanesAtMost64;
 
 impl_integer_vector! { SimdI64, i64 }
 
diff --git a/crates/core_simd/src/vectors_i8.rs b/crates/core_simd/src/vectors_i8.rs
index 0ac5ba9efee5..e21126533b88 100644
--- a/crates/core_simd/src/vectors_i8.rs
+++ b/crates/core_simd/src/vectors_i8.rs
@@ -2,7 +2,9 @@
 
 /// A SIMD vector of containing `LANES` `i8` values.
 #[repr(simd)]
-pub struct SimdI8<const LANES: usize>([i8; LANES]);
+pub struct SimdI8<const LANES: usize>([i8; LANES])
+where
+    Self: crate::LanesAtMost64;
 
 impl_integer_vector! { SimdI8, i8 }
 
diff --git a/crates/core_simd/src/vectors_isize.rs b/crates/core_simd/src/vectors_isize.rs
index 6856f305092d..ee23dfe7d865 100644
--- a/crates/core_simd/src/vectors_isize.rs
+++ b/crates/core_simd/src/vectors_isize.rs
@@ -2,7 +2,9 @@
 
 /// A SIMD vector of containing `LANES` `isize` values.
 #[repr(simd)]
-pub struct SimdIsize<const LANES: usize>([isize; LANES]);
+pub struct SimdIsize<const LANES: usize>([isize; LANES])
+where
+    Self: crate::LanesAtMost64;
 
 impl_integer_vector! { SimdIsize, isize }
 
diff --git a/crates/core_simd/src/vectors_u128.rs b/crates/core_simd/src/vectors_u128.rs
index 522404f133e5..7931b9e088f6 100644
--- a/crates/core_simd/src/vectors_u128.rs
+++ b/crates/core_simd/src/vectors_u128.rs
@@ -2,7 +2,9 @@
 
 /// A SIMD vector of containing `LANES` `u128` values.
 #[repr(simd)]
-pub struct SimdU128<const LANES: usize>([u128; LANES]);
+pub struct SimdU128<const LANES: usize>([u128; LANES])
+where
+    Self: crate::LanesAtMost64;
 
 impl_integer_vector! { SimdU128, u128 }
 
diff --git a/crates/core_simd/src/vectors_u16.rs b/crates/core_simd/src/vectors_u16.rs
index efe7dea58dc4..91c0e6168089 100644
--- a/crates/core_simd/src/vectors_u16.rs
+++ b/crates/core_simd/src/vectors_u16.rs
@@ -2,7 +2,9 @@
 
 /// A SIMD vector of containing `LANES` `u16` values.
 #[repr(simd)]
-pub struct SimdU16<const LANES: usize>([u16; LANES]);
+pub struct SimdU16<const LANES: usize>([u16; LANES])
+where
+    Self: crate::LanesAtMost64;
 
 impl_integer_vector! { SimdU16, u16 }
 
diff --git a/crates/core_simd/src/vectors_u32.rs b/crates/core_simd/src/vectors_u32.rs
index a6cef5baeb73..b0400b5ba3a9 100644
--- a/crates/core_simd/src/vectors_u32.rs
+++ b/crates/core_simd/src/vectors_u32.rs
@@ -2,7 +2,9 @@
 
 /// A SIMD vector of containing `LANES` `u32` values.
 #[repr(simd)]
-pub struct SimdU32<const LANES: usize>([u32; LANES]);
+pub struct SimdU32<const LANES: usize>([u32; LANES])
+where
+    Self: crate::LanesAtMost64;
 
 impl_integer_vector! { SimdU32, u32 }
 
diff --git a/crates/core_simd/src/vectors_u64.rs b/crates/core_simd/src/vectors_u64.rs
index 3982e30f5705..0f3712241fe7 100644
--- a/crates/core_simd/src/vectors_u64.rs
+++ b/crates/core_simd/src/vectors_u64.rs
@@ -2,7 +2,9 @@
 
 /// A SIMD vector of containing `LANES` `u64` values.
 #[repr(simd)]
-pub struct SimdU64<const LANES: usize>([u64; LANES]);
+pub struct SimdU64<const LANES: usize>([u64; LANES])
+where
+    Self: crate::LanesAtMost64;
 
 impl_integer_vector! { SimdU64, u64 }
 
diff --git a/crates/core_simd/src/vectors_usize.rs b/crates/core_simd/src/vectors_usize.rs
index c882898f9fbe..ea089aeb9d3c 100644
--- a/crates/core_simd/src/vectors_usize.rs
+++ b/crates/core_simd/src/vectors_usize.rs
@@ -2,7 +2,9 @@
 
 /// A SIMD vector of containing `LANES` `usize` values.
 #[repr(simd)]
-pub struct SimdUsize<const LANES: usize>([usize; LANES]);
+pub struct SimdUsize<const LANES: usize>([usize; LANES])
+where
+    Self: crate::LanesAtMost64;
 
 impl_integer_vector! { SimdUsize, usize }
 

From faae170f5ff8de1c0895483c15a5f783dec21320 Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Sat, 13 Feb 2021 00:52:42 -0500
Subject: [PATCH 082/249] Remove glob import

---
 crates/core_simd/src/lib.rs | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/crates/core_simd/src/lib.rs b/crates/core_simd/src/lib.rs
index e2e4864cc758..1f216bacd9af 100644
--- a/crates/core_simd/src/lib.rs
+++ b/crates/core_simd/src/lib.rs
@@ -15,7 +15,7 @@ mod ops;
 mod round;
 
 mod lanes_at_most_64;
-pub use lanes_at_most_64::*;
+pub use lanes_at_most_64::LanesAtMost64;
 
 mod masks;
 pub use masks::*;

From 08ee3385c0716e0738730979a7fe2277b13ede1c Mon Sep 17 00:00:00 2001
From: Jubilee Young <workingjubilee@gmail.com>
Date: Wed, 27 Jan 2021 13:20:00 -0800
Subject: [PATCH 083/249] Add to glossary: vectorize, scalar, vector register

Also document the lanewise scalar helpers.
---
 beginners-guide.md                         | 8 ++++++++
 crates/core_simd/tests/helpers/lanewise.rs | 7 +++++++
 2 files changed, 15 insertions(+)

diff --git a/beginners-guide.md b/beginners-guide.md
index 998dabfed1fb..dfd357c45920 100644
--- a/beginners-guide.md
+++ b/beginners-guide.md
@@ -19,10 +19,18 @@ SIMD has a few special vocabulary terms you should know:
 
 * **Vector:** A SIMD value is called a vector. This shouldn't be confused with the `Vec<T>` type. A SIMD vector has a fixed size, known at compile time. All of the elements within the vector are of the same type. This makes vectors *similar to* arrays. One difference is that a vector is generally aligned to its *entire* size (eg: 16 bytes, 32 bytes, etc), not just the size of an individual element. Sometimes vector data is called "packed" data.
 
+* **Vectorize**: An operation that uses SIMD instructions to operate over a vector is often referred to as "vectorized".
+
+* **Autovectorization**: Also known as _implicit vectorization_. This is when a compiler can automatically recognize a situation where scalar instructions may be replaced with SIMD instructions, and use those instead.
+
+* **Scalar:** "Scalar" in mathematical contexts refers to values that can be represented as a single element, mostly numbers like 6, 3.14, or -2. It can also be used to describe "scalar operations" that use strictly scalar values, like addition. This term is mostly used to differentiate between vectorized operations that use SIMD instructions and scalar operations that don't.
+
 * **Lane:** A single element position within a vector is called a lane. If you have `N` lanes available then they're numbered from `0` to `N-1` when referring to them, again like an array. The biggest difference between an array element and a vector lane is that in general is *relatively costly* to access an individual lane value. On most architectures, the vector has to be pushed out of the SIMD register onto the stack, then an individual lane is accessed while it's on the stack (and possibly the stack value is read back into a register). For this reason, when working with SIMD you should avoid reading or writing the value of an individual lane during hot loops.
 
 * **Bit Widths:** When talking about SIMD, the bit widths used are the bit size of the vectors involved, *not* the individual elements. So "128-bit SIMD" has 128-bit vectors, and that might be `f32x4`, `i32x4`, `i16x8`, or other variations. While 128-bit SIMD is the most common, there's also 64-bit, 256-bit, and even 512-bit on the newest CPUs.
 
+* **Vector Register:** The extra-wide registers that are used for SIMD operations are commonly called vector registers, though you may also see "SIMD registers", vendor names for specific features, or even "floating-point register" as it is common for the same registers to be used with both scalar and vectorized floating-point operations.
+
 * **Vertical:** When an operation is "vertical", each lane processes individually without regard to the other lanes in the same vector. For example, a "vertical add" between two vectors would add lane 0 in `a` with lane 0 in `b`, with the total in lane 0 of `out`, and then the same thing for lanes 1, 2, etc. Most SIMD operations are vertical operations, so if your problem is a vertical problem then you can probably solve it with SIMD.
 
 * **Horizontal:** When an operation is "horizontal", the lanes within a single vector interact in some way. A "horizontal add" might add up lane 0 of `a` with lane 1 of `a`, with the total in lane 0 of `out`.
diff --git a/crates/core_simd/tests/helpers/lanewise.rs b/crates/core_simd/tests/helpers/lanewise.rs
index 15f1a88ffd5c..3a9f47968080 100644
--- a/crates/core_simd/tests/helpers/lanewise.rs
+++ b/crates/core_simd/tests/helpers/lanewise.rs
@@ -1,3 +1,10 @@
+//! These helpers provide a way to easily emulate a vectorized SIMD op on two SIMD vectors,
+//! except using scalar ops that iterate through each lane, one at a time, so as to remove
+//! the vagaries of compilation.
+//!
+//! Do note, however, that when testing that vectorized operations #[should_panic], these
+//! "scalarized SIMD ops" will trigger scalar code paths that may also normally panic.
+
 pub fn apply_unary_lanewise<T1: Copy, T2: Copy, V1: AsRef<[T1]>, V2: AsMut<[T2]> + Default>(
     x: V1,
     f: impl Fn(T1) -> T2,

From d5c227998bdc854938bdbf7dea96a58d2f7395a2 Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Sun, 3 Jan 2021 16:09:26 -0500
Subject: [PATCH 084/249] Add proptest float tests

---
 Cargo.toml                                    |   1 +
 crates/core_simd/Cargo.toml                   |   8 +
 crates/core_simd/src/macros.rs                |   6 +
 crates/core_simd/tests/float.rs               | 132 ++++++
 crates/core_simd/tests/ops_impl/f32.rs        |   6 -
 crates/core_simd/tests/ops_impl/f64.rs        |   5 -
 .../core_simd/tests/ops_impl/float_macros.rs  | 418 ------------------
 crates/core_simd/tests/ops_impl/mod.rs        |   6 -
 crates/test_helpers/Cargo.toml                |   9 +
 crates/test_helpers/src/array.rs              |  98 ++++
 crates/test_helpers/src/biteq.rs              |  94 ++++
 crates/test_helpers/src/lib.rs                | 224 ++++++++++
 12 files changed, 572 insertions(+), 435 deletions(-)
 create mode 100644 crates/core_simd/tests/float.rs
 delete mode 100644 crates/core_simd/tests/ops_impl/f32.rs
 delete mode 100644 crates/core_simd/tests/ops_impl/f64.rs
 delete mode 100644 crates/core_simd/tests/ops_impl/float_macros.rs
 create mode 100644 crates/test_helpers/Cargo.toml
 create mode 100644 crates/test_helpers/src/array.rs
 create mode 100644 crates/test_helpers/src/biteq.rs
 create mode 100644 crates/test_helpers/src/lib.rs

diff --git a/Cargo.toml b/Cargo.toml
index f3538db75590..3f1abd73519b 100644
--- a/Cargo.toml
+++ b/Cargo.toml
@@ -2,4 +2,5 @@
 
 members = [
     "crates/core_simd",
+    "crates/test_helpers",
 ]
diff --git a/crates/core_simd/Cargo.toml b/crates/core_simd/Cargo.toml
index f9e8a62e4857..d76bd547cde6 100644
--- a/crates/core_simd/Cargo.toml
+++ b/crates/core_simd/Cargo.toml
@@ -14,3 +14,11 @@ version = "0.2"
 
 [dev-dependencies.wasm-bindgen-test]
 version = "0.3"
+
+[dev-dependencies.proptest]
+version = "0.10"
+default-features = false
+features = ["alloc"]
+
+[dev-dependencies.test_helpers]
+path = "../test_helpers"
diff --git a/crates/core_simd/src/macros.rs b/crates/core_simd/src/macros.rs
index 5328f22b42ab..3e428379b74d 100644
--- a/crates/core_simd/src/macros.rs
+++ b/crates/core_simd/src/macros.rs
@@ -141,6 +141,12 @@ macro_rules! impl_vector {
             }
         }
 
+        impl <const LANES: usize> From<$name<LANES>> for [$type; LANES] {
+            fn from(vector: $name<LANES>) -> Self {
+                vector.0
+            }
+        }
+
         // splat
         impl<const LANES: usize> From<$type> for $name<LANES> where Self: crate::LanesAtMost64 {
             #[inline]
diff --git a/crates/core_simd/tests/float.rs b/crates/core_simd/tests/float.rs
new file mode 100644
index 000000000000..939c18559d2a
--- /dev/null
+++ b/crates/core_simd/tests/float.rs
@@ -0,0 +1,132 @@
+#[cfg(target_arch = "wasm32")]
+wasm_bindgen_test_configure!(run_in_browser);
+
+macro_rules! impl_op_test {
+    { unary, $vector:ty, $scalar:ty, $trait:ident :: $fn:ident } => {
+        test_helpers::test_lanes! {
+            fn $fn<const LANES: usize>() {
+                test_helpers::test_unary_elementwise(
+                    <$vector as core::ops::$trait>::$fn,
+                    <$scalar as core::ops::$trait>::$fn,
+                );
+            }
+        }
+    };
+    { binary, $vector:ty, $scalar:ty, $trait:ident :: $fn:ident, $trait_assign:ident :: $fn_assign:ident } => {
+        mod $fn {
+            use super::*;
+
+            test_helpers::test_lanes! {
+                fn normal<const LANES: usize>() {
+                    test_helpers::test_binary_elementwise(
+                        <$vector as core::ops::$trait>::$fn,
+                        <$scalar as core::ops::$trait>::$fn,
+                    );
+                }
+
+                fn scalar_rhs<const LANES: usize>() {
+                    test_helpers::test_binary_scalar_rhs_elementwise(
+                        <$vector as core::ops::$trait<$scalar>>::$fn,
+                        <$scalar as core::ops::$trait>::$fn,
+                    );
+                }
+
+                fn scalar_lhs<const LANES: usize>() {
+                    test_helpers::test_binary_scalar_lhs_elementwise(
+                        <$scalar as core::ops::$trait<$vector>>::$fn,
+                        <$scalar as core::ops::$trait>::$fn,
+                    );
+                }
+
+                fn assign<const LANES: usize>() {
+                    test_helpers::test_binary_elementwise(
+                        |mut a, b| { <$vector as core::ops::$trait_assign>::$fn_assign(&mut a, b); a },
+                        |mut a, b| { <$scalar as core::ops::$trait_assign>::$fn_assign(&mut a, b); a },
+                    )
+                }
+
+                fn assign_scalar_rhs<const LANES: usize>() {
+                    test_helpers::test_binary_scalar_rhs_elementwise(
+                        |mut a, b| { <$vector as core::ops::$trait_assign<$scalar>>::$fn_assign(&mut a, b); a },
+                        |mut a, b| { <$scalar as core::ops::$trait_assign>::$fn_assign(&mut a, b); a },
+                    )
+                }
+            }
+        }
+    };
+}
+
+macro_rules! impl_tests {
+    { $vector:ident, $scalar:tt, $int_scalar:tt } => {
+        mod $scalar {
+            type Vector<const LANES: usize> = core_simd::$vector<LANES>;
+            type Scalar = $scalar;
+            type IntScalar = $int_scalar;
+            
+            impl_op_test! { unary, Vector<LANES>, Scalar, Neg::neg }
+            impl_op_test! { binary, Vector<LANES>, Scalar, Add::add, AddAssign::add_assign }
+            impl_op_test! { binary, Vector<LANES>, Scalar, Sub::sub, SubAssign::sub_assign }
+            impl_op_test! { binary, Vector<LANES>, Scalar, Mul::mul, SubAssign::sub_assign }
+            impl_op_test! { binary, Vector<LANES>, Scalar, Div::div, DivAssign::div_assign }
+            impl_op_test! { binary, Vector<LANES>, Scalar, Rem::rem, RemAssign::rem_assign }
+
+            test_helpers::test_lanes! {
+                fn abs<const LANES: usize>() {
+                    test_helpers::test_unary_elementwise(
+                        Vector::<LANES>::abs,
+                        Scalar::abs,
+                    )
+                }
+
+                fn ceil<const LANES: usize>() {
+                    test_helpers::test_unary_elementwise(
+                        Vector::<LANES>::ceil,
+                        Scalar::ceil,
+                    )
+                }
+
+                fn floor<const LANES: usize>() {
+                    test_helpers::test_unary_elementwise(
+                        Vector::<LANES>::floor,
+                        Scalar::floor,
+                    )
+                }
+
+                fn round_from_int<const LANES: usize>() {
+                    test_helpers::test_unary_elementwise(
+                        Vector::<LANES>::round_from_int,
+                        |x| x as Scalar,
+                    )
+                }
+
+                fn to_int_unchecked<const LANES: usize>() {
+                    // The maximum integer that can be represented by the equivalently sized float has
+                    // all of the mantissa digits set to 1, pushed up to the MSB.
+                    const ALL_MANTISSA_BITS: IntScalar = ((1 << <Scalar>::MANTISSA_DIGITS) - 1);
+                    const MAX_REPRESENTABLE_VALUE: Scalar =
+                        (ALL_MANTISSA_BITS << (core::mem::size_of::<Scalar>() * 8 - <Scalar>::MANTISSA_DIGITS as usize - 1)) as Scalar;
+
+                    let mut runner = proptest::test_runner::TestRunner::default();
+                    runner.run(
+                        &test_helpers::array::UniformArrayStrategy::new(-MAX_REPRESENTABLE_VALUE..MAX_REPRESENTABLE_VALUE),
+                        |x| {
+                            let result_1 = unsafe { Vector::from_array(x).to_int_unchecked().to_array() };
+                            let result_2 = {
+                                let mut result = [0; LANES];
+                                for (i, o) in x.iter().zip(result.iter_mut()) {
+                                    *o = unsafe { i.to_int_unchecked() };
+                                }
+                                result
+                            };
+                            test_helpers::prop_assert_biteq!(result_1, result_2);
+                            Ok(())
+                        },
+                    ).unwrap();
+                }
+            }
+        }
+    }
+}
+
+impl_tests! { SimdF32, f32, i32 }
+impl_tests! { SimdF64, f64, i64 }
diff --git a/crates/core_simd/tests/ops_impl/f32.rs b/crates/core_simd/tests/ops_impl/f32.rs
deleted file mode 100644
index 1472822fe1fe..000000000000
--- a/crates/core_simd/tests/ops_impl/f32.rs
+++ /dev/null
@@ -1,6 +0,0 @@
-use super::helpers;
-
-float_tests! { f32x2, f32, i32x2, i32 }
-float_tests! { f32x4, f32, i32x4, i32 }
-float_tests! { f32x8, f32, i32x8, i32 }
-float_tests! { f32x16, f32, i32x16, i32 }
diff --git a/crates/core_simd/tests/ops_impl/f64.rs b/crates/core_simd/tests/ops_impl/f64.rs
deleted file mode 100644
index 8f573baa1ad2..000000000000
--- a/crates/core_simd/tests/ops_impl/f64.rs
+++ /dev/null
@@ -1,5 +0,0 @@
-use super::helpers;
-
-float_tests! { f64x2, f64, i64x2, i64 }
-float_tests! { f64x4, f64, i64x4, i64 }
-float_tests! { f64x8, f64, i64x8, i64 }
diff --git a/crates/core_simd/tests/ops_impl/float_macros.rs b/crates/core_simd/tests/ops_impl/float_macros.rs
deleted file mode 100644
index fe347a5362da..000000000000
--- a/crates/core_simd/tests/ops_impl/float_macros.rs
+++ /dev/null
@@ -1,418 +0,0 @@
-macro_rules! float_tests {
-    { $vector:ident, $scalar:ident, $int_vector:ident, $int_scalar:ident } => {
-        #[cfg(test)]
-        mod $vector {
-            use super::*;
-            use helpers::lanewise::*;
-
-            #[cfg(target_arch = "wasm32")]
-            use wasm_bindgen_test::*;
-
-            #[cfg(target_arch = "wasm32")]
-            wasm_bindgen_test_configure!(run_in_browser);
-
-            // TODO impl this as an associated fn on vectors
-            fn from_slice(slice: &[$scalar]) -> core_simd::$vector {
-                let mut value = core_simd::$vector::default();
-                let value_slice: &mut [_] = value.as_mut();
-                value_slice.copy_from_slice(&slice[0..value_slice.len()]);
-                value
-            }
-
-            fn slice_chunks(slice: &[$scalar]) -> impl Iterator<Item = core_simd::$vector> + '_ {
-                let lanes = core::mem::size_of::<core_simd::$vector>() / core::mem::size_of::<$scalar>();
-                slice.chunks_exact(lanes).map(from_slice)
-            }
-
-            fn from_slice_int(slice: &[$int_scalar]) -> core_simd::$int_vector {
-                let mut value = core_simd::$int_vector::default();
-                let value_slice: &mut [_] = value.as_mut();
-                value_slice.copy_from_slice(&slice[0..value_slice.len()]);
-                value
-            }
-
-            fn slice_chunks_int(slice: &[$int_scalar]) -> impl Iterator<Item = core_simd::$int_vector> + '_ {
-                let lanes = core::mem::size_of::<core_simd::$int_vector>() / core::mem::size_of::<$int_scalar>();
-                slice.chunks_exact(lanes).map(from_slice_int)
-            }
-
-            const A: [$scalar; 16] = [0.,   1.,  2.,  3.,  4.,  5.,  6.,  7.,  8.,  9., 10., 11., 12., 13., 14., 15.];
-            const B: [$scalar; 16] = [16., 17., 18., 19., 20., 21., 22., 23., 24., 25., 26., 27., 28., 29., 30., 31.];
-            const C: [$scalar; 16] = [
-                -0.0,
-                0.0,
-                -1.0,
-                1.0,
-                <$scalar>::MIN,
-                <$scalar>::MAX,
-                <$scalar>::INFINITY,
-                <$scalar>::NEG_INFINITY,
-                <$scalar>::MIN_POSITIVE,
-                -<$scalar>::MIN_POSITIVE,
-                <$scalar>::EPSILON,
-                -<$scalar>::EPSILON,
-                <$scalar>::NAN,
-                -<$scalar>::NAN,
-                // TODO: Would be nice to check sNaN...
-                100.0 / 3.0,
-                -100.0 / 3.0,
-            ];
-
-            #[test]
-            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
-            fn add() {
-                let a = from_slice(&A);
-                let b = from_slice(&B);
-                let expected = apply_binary_lanewise(a, b, core::ops::Add::add);
-                assert_biteq!(a + b, expected);
-            }
-
-            #[test]
-            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
-            fn add_assign() {
-                let mut a = from_slice(&A);
-                let b = from_slice(&B);
-                let expected = apply_binary_lanewise(a, b, core::ops::Add::add);
-                a += b;
-                assert_biteq!(a, expected);
-            }
-
-            #[test]
-            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
-            fn add_scalar_rhs() {
-                let a = from_slice(&A);
-                let b = 5.;
-                let expected = apply_binary_scalar_rhs_lanewise(a, b, core::ops::Add::add);
-                assert_biteq!(a + b, expected);
-            }
-
-            #[test]
-            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
-            fn add_scalar_lhs() {
-                let a = 5.;
-                let b = from_slice(&B);
-                let expected = apply_binary_scalar_lhs_lanewise(a, b, core::ops::Add::add);
-                assert_biteq!(a + b, expected);
-            }
-
-            #[test]
-            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
-            fn add_assign_scalar() {
-                let mut a = from_slice(&A);
-                let b = 5.;
-                let expected = apply_binary_scalar_rhs_lanewise(a, b, core::ops::Add::add);
-                a += b;
-                assert_biteq!(a, expected);
-            }
-
-            #[test]
-            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
-            fn sub() {
-                let a = from_slice(&A);
-                let b = from_slice(&B);
-                let expected = apply_binary_lanewise(a, b, core::ops::Sub::sub);
-                assert_biteq!(a - b, expected);
-            }
-
-            #[test]
-            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
-            fn sub_assign() {
-                let mut a = from_slice(&A);
-                let b = from_slice(&B);
-                let expected = apply_binary_lanewise(a, b, core::ops::Sub::sub);
-                a -= b;
-                assert_biteq!(a, expected);
-            }
-
-            #[test]
-            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
-            fn sub_scalar_rhs() {
-                let a = from_slice(&A);
-                let b = 5.;
-                let expected = apply_binary_scalar_rhs_lanewise(a, b, core::ops::Sub::sub);
-                assert_biteq!(a - b, expected);
-            }
-
-            #[test]
-            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
-            fn sub_scalar_lhs() {
-                let a = 5.;
-                let b = from_slice(&B);
-                let expected = apply_binary_scalar_lhs_lanewise(a, b, core::ops::Sub::sub);
-                assert_biteq!(a - b, expected);
-            }
-
-            #[test]
-            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
-            fn sub_assign_scalar() {
-                let mut a = from_slice(&A);
-                let b = 5.;
-                let expected = apply_binary_scalar_rhs_lanewise(a, b, core::ops::Sub::sub);
-                a -= b;
-                assert_biteq!(a, expected);
-            }
-
-            #[test]
-            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
-            fn mul() {
-                let a = from_slice(&A);
-                let b = from_slice(&B);
-                let expected = apply_binary_lanewise(a, b, core::ops::Mul::mul);
-                assert_biteq!(a * b, expected);
-            }
-
-            #[test]
-            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
-            fn mul_assign() {
-                let mut a = from_slice(&A);
-                let b = from_slice(&B);
-                let expected = apply_binary_lanewise(a, b, core::ops::Mul::mul);
-                a *= b;
-                assert_biteq!(a, expected);
-            }
-
-            #[test]
-            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
-            fn mul_scalar_rhs() {
-                let a = from_slice(&A);
-                let b = 5.;
-                let expected = apply_binary_scalar_rhs_lanewise(a, b, core::ops::Mul::mul);
-                assert_biteq!(a * b, expected);
-            }
-
-            #[test]
-            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
-            fn mul_scalar_lhs() {
-                let a = 5.;
-                let b = from_slice(&B);
-                let expected = apply_binary_scalar_lhs_lanewise(a, b, core::ops::Mul::mul);
-                assert_biteq!(a * b, expected);
-            }
-
-            #[test]
-            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
-            fn mul_assign_scalar() {
-                let mut a = from_slice(&A);
-                let b = 5.;
-                let expected = apply_binary_scalar_rhs_lanewise(a, b, core::ops::Mul::mul);
-                a *= b;
-                assert_biteq!(a, expected);
-            }
-
-            #[test]
-            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
-            fn div() {
-                let a = from_slice(&A);
-                let b = from_slice(&B);
-                let expected = apply_binary_lanewise(a, b, core::ops::Div::div);
-                assert_biteq!(a / b, expected);
-            }
-
-            #[test]
-            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
-            fn div_assign() {
-                let mut a = from_slice(&A);
-                let b = from_slice(&B);
-                let expected = apply_binary_lanewise(a, b, core::ops::Div::div);
-                a /= b;
-                assert_biteq!(a, expected);
-            }
-
-            #[test]
-            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
-            fn div_scalar_rhs() {
-                let a = from_slice(&A);
-                let b = 5.;
-                let expected = apply_binary_scalar_rhs_lanewise(a, b, core::ops::Div::div);
-                assert_biteq!(a / b, expected);
-            }
-
-            #[test]
-            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
-            fn div_scalar_lhs() {
-                let a = 5.;
-                let b = from_slice(&B);
-                let expected = apply_binary_scalar_lhs_lanewise(a, b, core::ops::Div::div);
-                assert_biteq!(a / b, expected);
-            }
-
-            #[test]
-            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
-            fn div_assign_scalar() {
-                let mut a = from_slice(&A);
-                let b = 5.;
-                let expected = apply_binary_scalar_rhs_lanewise(a, b, core::ops::Div::div);
-                a /= b;
-                assert_biteq!(a, expected);
-            }
-
-            #[test]
-            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
-            fn rem() {
-                let a = from_slice(&A);
-                let b = from_slice(&B);
-                let expected = apply_binary_lanewise(a, b, core::ops::Rem::rem);
-                assert_biteq!(a % b, expected);
-            }
-
-            #[test]
-            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
-            fn rem_assign() {
-                let mut a = from_slice(&A);
-                let b = from_slice(&B);
-                let expected = apply_binary_lanewise(a, b, core::ops::Rem::rem);
-                a %= b;
-                assert_biteq!(a, expected);
-            }
-
-            #[test]
-            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
-            fn rem_scalar_rhs() {
-                let a = from_slice(&A);
-                let b = 5.;
-                let expected = apply_binary_scalar_rhs_lanewise(a, b, core::ops::Rem::rem);
-                assert_biteq!(a % b, expected);
-            }
-
-            #[test]
-            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
-            fn rem_scalar_lhs() {
-                let a = 5.;
-                let b = from_slice(&B);
-                let expected = apply_binary_scalar_lhs_lanewise(a, b, core::ops::Rem::rem);
-                assert_biteq!(a % b, expected);
-            }
-
-            #[test]
-            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
-            fn rem_assign_scalar() {
-                let mut a = from_slice(&A);
-                let b = 5.;
-                let expected = apply_binary_scalar_rhs_lanewise(a, b, core::ops::Rem::rem);
-                a %= b;
-                assert_biteq!(a, expected);
-            }
-
-            #[test]
-            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
-            fn neg() {
-                let v = from_slice(&A);
-                let expected = apply_unary_lanewise(v, core::ops::Neg::neg);
-                assert_biteq!(-v, expected);
-            }
-
-            #[test]
-            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
-            fn neg_odd_floats() {
-                for v in slice_chunks(&C) {
-                    let expected = apply_unary_lanewise(v, core::ops::Neg::neg);
-                    assert_biteq!(-v, expected);
-                }
-            }
-
-            #[test]
-            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
-            fn abs_negative() {
-                let v = -from_slice(&A);
-                let expected = apply_unary_lanewise(v, <$scalar>::abs);
-                assert_biteq!(v.abs(), expected);
-            }
-
-            #[test]
-            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
-            fn abs_positive() {
-                let v = from_slice(&B);
-                let expected = apply_unary_lanewise(v, <$scalar>::abs);
-                assert_biteq!(v.abs(), expected);
-            }
-
-            #[test]
-            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
-            fn abs_odd_floats() {
-                for v in slice_chunks(&C) {
-                    let expected = apply_unary_lanewise(v, <$scalar>::abs);
-                    assert_biteq!(v.abs(), expected);
-                }
-            }
-
-            #[test]
-            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
-            fn ceil_odd_floats() {
-                for v in slice_chunks(&C) {
-                    let expected = apply_unary_lanewise(v, <$scalar>::ceil);
-                    assert_biteq!(v.ceil(), expected);
-                }
-            }
-
-            #[test]
-            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
-            fn floor_odd_floats() {
-                for v in slice_chunks(&C) {
-                    let expected = apply_unary_lanewise(v, <$scalar>::floor);
-                    assert_biteq!(v.floor(), expected);
-                }
-            }
-
-            #[test]
-            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
-            fn to_int_unchecked() {
-                // The maximum integer that can be represented by the equivalently sized float has
-                // all of the mantissa digits set to 1, pushed up to the MSB.
-                const ALL_MANTISSA_BITS: $int_scalar = ((1 << <$scalar>::MANTISSA_DIGITS) - 1);
-                const MAX_REPRESENTABLE_VALUE: $int_scalar =
-                    ALL_MANTISSA_BITS << (core::mem::size_of::<$scalar>() * 8 - <$scalar>::MANTISSA_DIGITS as usize - 1);
-                const VALUES: [$scalar; 16] = [
-                    -0.0,
-                    0.0,
-                    -1.0,
-                    1.0,
-                    ALL_MANTISSA_BITS as $scalar,
-                    -ALL_MANTISSA_BITS as $scalar,
-                    MAX_REPRESENTABLE_VALUE as $scalar,
-                    -MAX_REPRESENTABLE_VALUE as $scalar,
-                    (MAX_REPRESENTABLE_VALUE / 2) as $scalar,
-                    (-MAX_REPRESENTABLE_VALUE / 2) as $scalar,
-                    <$scalar>::MIN_POSITIVE,
-                    -<$scalar>::MIN_POSITIVE,
-                    <$scalar>::EPSILON,
-                    -<$scalar>::EPSILON,
-                    100.0 / 3.0,
-                    -100.0 / 3.0,
-                ];
-
-                for v in slice_chunks(&VALUES) {
-                    let expected = apply_unary_lanewise(v, |x| unsafe { x.to_int_unchecked() });
-                    assert_biteq!(unsafe { v.to_int_unchecked() }, expected);
-                }
-            }
-
-            #[test]
-            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
-            fn round_from_int() {
-                const VALUES: [$int_scalar; 16] = [
-                    0,
-                    0,
-                    1,
-                    -1,
-                    100,
-                    -100,
-                    200,
-                    -200,
-                    413,
-                    -413,
-                    1017,
-                    -1017,
-                    1234567,
-                    -1234567,
-                    <$int_scalar>::MAX,
-                    <$int_scalar>::MIN,
-                ];
-
-                for v in slice_chunks_int(&VALUES) {
-                    let expected = apply_unary_lanewise(v, |x| x as $scalar);
-                    assert_biteq!(core_simd::$vector::round_from_int(v), expected);
-                }
-            }
-        }
-    }
-}
diff --git a/crates/core_simd/tests/ops_impl/mod.rs b/crates/core_simd/tests/ops_impl/mod.rs
index 814f2d04b59c..5819eb6beafa 100644
--- a/crates/core_simd/tests/ops_impl/mod.rs
+++ b/crates/core_simd/tests/ops_impl/mod.rs
@@ -2,12 +2,6 @@
 #[path = "../helpers/mod.rs"]
 mod helpers;
 
-#[macro_use]
-mod float_macros;
-
-mod r#f32;
-mod r#f64;
-
 #[macro_use]
 mod int_macros;
 
diff --git a/crates/test_helpers/Cargo.toml b/crates/test_helpers/Cargo.toml
new file mode 100644
index 000000000000..0a8c3344334c
--- /dev/null
+++ b/crates/test_helpers/Cargo.toml
@@ -0,0 +1,9 @@
+[package]
+name = "test_helpers"
+version = "0.1.0"
+authors = ["Caleb Zulawski <caleb.zulawski@gmail.com>"]
+edition = "2018"
+publish = false
+
+[dependencies]
+proptest = "0.10"
diff --git a/crates/test_helpers/src/array.rs b/crates/test_helpers/src/array.rs
new file mode 100644
index 000000000000..d9cae96ca2ff
--- /dev/null
+++ b/crates/test_helpers/src/array.rs
@@ -0,0 +1,98 @@
+// Adapted from proptest's array code
+// Copyright 2017 Jason Lingle
+
+use proptest::{
+    strategy::{NewTree, Strategy, ValueTree},
+    test_runner::TestRunner,
+};
+use core::{
+    marker::PhantomData,
+    mem::MaybeUninit,
+};
+
+#[must_use = "strategies do nothing unless used"]
+#[derive(Clone, Copy, Debug)]
+pub struct UniformArrayStrategy<S, T> {
+    strategy: S,
+    _marker: PhantomData<T>,
+}
+
+impl<S, T> UniformArrayStrategy<S, T> {
+    pub fn new(strategy: S) -> Self {
+        Self {
+            strategy,
+            _marker: PhantomData,
+        }
+    }
+}
+
+pub struct ArrayValueTree<T> {
+    tree: T,
+    shrinker: usize,
+    last_shrinker: Option<usize>,
+}
+
+impl<T, S, const LANES: usize> Strategy for UniformArrayStrategy<S, [T; LANES]>
+where
+    T: core::fmt::Debug,
+    S: Strategy<Value = T>,
+{
+    type Tree = ArrayValueTree<[S::Tree; LANES]>;
+    type Value = [T; LANES];
+
+    fn new_tree(&self, runner: &mut TestRunner) -> NewTree<Self> {
+        let tree: [S::Tree; LANES] = unsafe {
+            let mut tree: [MaybeUninit<S::Tree>; LANES] = MaybeUninit::uninit().assume_init();
+            for t in tree.iter_mut() {
+                *t = MaybeUninit::new(self.strategy.new_tree(runner)?)
+            }
+            core::mem::transmute_copy(&tree)
+        };
+        Ok(ArrayValueTree {
+            tree,
+            shrinker: 0,
+            last_shrinker: None,
+        })
+    }
+}
+
+impl<T: ValueTree, const LANES: usize> ValueTree for ArrayValueTree<[T; LANES]> {
+    type Value = [T::Value; LANES];
+
+    fn current(&self) -> Self::Value {
+        unsafe {
+            let mut value: [MaybeUninit<T::Value>; LANES] = MaybeUninit::uninit().assume_init();
+            for (tree_elem, value_elem) in self.tree.iter().zip(value.iter_mut()) {
+                *value_elem = MaybeUninit::new(tree_elem.current());
+            }
+            core::mem::transmute_copy(&value)
+        }
+    }
+
+    fn simplify(&mut self) -> bool {
+        while self.shrinker < LANES {
+            if self.tree[self.shrinker].simplify() {
+                self.last_shrinker = Some(self.shrinker);
+                return true;
+            } else {
+                self.shrinker += 1;
+            }
+        }
+
+        false
+    }
+
+    fn complicate(&mut self) -> bool {
+        if let Some(shrinker) = self.last_shrinker {
+            self.shrinker = shrinker;
+            if self.tree[shrinker].complicate() {
+                true
+            } else {
+                self.last_shrinker = None;
+                false
+            }
+        } else {
+            false
+        }
+    }
+}
diff --git a/crates/test_helpers/src/biteq.rs b/crates/test_helpers/src/biteq.rs
new file mode 100644
index 000000000000..23aa7d4d908e
--- /dev/null
+++ b/crates/test_helpers/src/biteq.rs
@@ -0,0 +1,94 @@
+pub trait BitEq {
+    fn biteq(&self, other: &Self) -> bool;
+    fn fmt(&self, f: &mut core::fmt::Formatter) -> core::fmt::Result;
+}
+
+macro_rules! impl_integer_biteq {
+    { $($type:ty),* } => {
+        $(
+        impl BitEq for $type {
+            fn biteq(&self, other: &Self) -> bool {
+                self == other
+            }
+
+            fn fmt(&self, f: &mut core::fmt::Formatter) -> core::fmt::Result {
+                write!(f, "{:?} ({:x})", self, self)
+            }
+        }
+        )*
+    };
+}
+
+impl_integer_biteq! { u8, u16, u32, u64, u128, usize, i8, i16, i32, i64, i128, isize }
+
+macro_rules! impl_float_biteq {
+    { $($type:ty),* } => {
+        $(
+        impl BitEq for $type {
+            fn biteq(&self, other: &Self) -> bool {
+                if self.is_nan() && other.is_nan() {
+                    true // exact nan bits don't matter
+                } else {
+                    self.to_bits() == other.to_bits()
+                }
+            }
+
+            fn fmt(&self, f: &mut core::fmt::Formatter) -> core::fmt::Result {
+                write!(f, "{:?} ({:x})", self, self.to_bits())
+            }
+        }
+        )*
+    };
+}
+
+impl_float_biteq! { f32, f64 }
+
+impl<T: BitEq, const N: usize> BitEq for [T; N] {
+    fn biteq(&self, other: &Self) -> bool {
+        self.iter()
+            .zip(other.iter())
+            .fold(true, |value, (left, right)| value && left.biteq(right))
+    }
+
+    fn fmt(&self, f: &mut core::fmt::Formatter) -> core::fmt::Result {
+        #[repr(transparent)]
+        struct Wrapper<'a, T: BitEq>(&'a T);
+
+        impl<T: BitEq> core::fmt::Debug for Wrapper<'_, T> {
+            fn fmt(&self, f: &mut core::fmt::Formatter) -> core::fmt::Result {
+                self.0.fmt(f)
+            }
+        }
+
+        f.debug_list()
+            .entries(self.iter().map(|x| Wrapper(x)))
+            .finish()
+    }
+}
+
+#[doc(hidden)]
+pub struct BitEqWrapper<'a, T>(pub &'a T);
+
+impl<T: BitEq> PartialEq for BitEqWrapper<'_, T> {
+    fn eq(&self, other: &Self) -> bool {
+        self.0.biteq(other.0)
+    }
+}
+
+impl<T: BitEq> core::fmt::Debug for BitEqWrapper<'_, T> {
+    fn fmt(&self, f: &mut core::fmt::Formatter) -> core::fmt::Result {
+        self.0.fmt(f)
+    }
+}
+
+#[macro_export]
+macro_rules! prop_assert_biteq {
+    { $a:expr, $b:expr } => {
+        {
+            use $crate::biteq::BitEqWrapper;
+            let a = $a;
+            let b = $b;
+            proptest::prop_assert_eq!(BitEqWrapper(&a), BitEqWrapper(&b));
+        }
+    }
+}
diff --git a/crates/test_helpers/src/lib.rs b/crates/test_helpers/src/lib.rs
new file mode 100644
index 000000000000..2aaa4641fdfd
--- /dev/null
+++ b/crates/test_helpers/src/lib.rs
@@ -0,0 +1,224 @@
+pub mod array;
+
+#[macro_use]
+pub mod biteq;
+
+pub trait DefaultStrategy {
+    type Strategy: proptest::strategy::Strategy<Value = Self>;
+    fn default_strategy() -> Self::Strategy;
+}
+
+macro_rules! impl_num {
+    { $type:tt } => {
+        impl DefaultStrategy for $type {
+            type Strategy = proptest::num::$type::Any;
+            fn default_strategy() -> Self::Strategy {
+                proptest::num::$type::ANY
+            }
+        }
+    }
+}
+
+impl_num! { i8 }
+impl_num! { i16 }
+impl_num! { i32 }
+impl_num! { i64 }
+impl_num! { i128 }
+impl_num! { isize }
+impl_num! { u8 }
+impl_num! { u16 }
+impl_num! { u32 }
+impl_num! { u64 }
+impl_num! { u128 }
+impl_num! { usize }
+impl_num! { f32 }
+impl_num! { f64 }
+
+impl<T: core::fmt::Debug + DefaultStrategy, const LANES: usize> DefaultStrategy for [T; LANES] {
+    type Strategy = crate::array::UniformArrayStrategy<T::Strategy, Self>;
+    fn default_strategy() -> Self::Strategy {
+        Self::Strategy::new(T::default_strategy())
+    }
+}
+
+pub fn test_1<A: core::fmt::Debug + DefaultStrategy>(
+    f: impl Fn(A) -> proptest::test_runner::TestCaseResult,
+) {
+    let mut runner = proptest::test_runner::TestRunner::default();
+    runner.run(&A::default_strategy(), f).unwrap();
+}
+
+pub fn test_2<A: core::fmt::Debug + DefaultStrategy, B: core::fmt::Debug + DefaultStrategy>(
+    f: impl Fn(A, B) -> proptest::test_runner::TestCaseResult,
+) {
+    let mut runner = proptest::test_runner::TestRunner::default();
+    runner
+        .run(&(A::default_strategy(), B::default_strategy()), |(a, b)| {
+            f(a, b)
+        })
+        .unwrap();
+}
+
+pub fn test_unary_elementwise<Scalar, ScalarResult, Vector, VectorResult, const LANES: usize>(
+    fv: impl Fn(Vector) -> VectorResult,
+    fs: impl Fn(Scalar) -> ScalarResult,
+) where
+    Scalar: Copy + Default + core::fmt::Debug + DefaultStrategy,
+    ScalarResult: Copy + Default + biteq::BitEq + core::fmt::Debug + DefaultStrategy,
+    Vector: Into<[Scalar; LANES]> + From<[Scalar; LANES]> + Copy,
+    VectorResult: Into<[ScalarResult; LANES]> + From<[ScalarResult; LANES]> + Copy,
+{
+    test_1(|x: [Scalar; LANES]| {
+        let result_1: [ScalarResult; LANES] = fv(x.into()).into();
+        let result_2: [ScalarResult; LANES] = {
+            let mut result = [ScalarResult::default(); LANES];
+            for (i, o) in x.iter().zip(result.iter_mut()) {
+                *o = fs(*i);
+            }
+            result
+        };
+        crate::prop_assert_biteq!(result_1, result_2);
+        Ok(())
+    });
+}
+
+pub fn test_binary_elementwise<
+    Scalar1,
+    Scalar2,
+    ScalarResult,
+    Vector1,
+    Vector2,
+    VectorResult,
+    const LANES: usize,
+>(
+    fv: impl Fn(Vector1, Vector2) -> VectorResult,
+    fs: impl Fn(Scalar1, Scalar2) -> ScalarResult,
+) where
+    Scalar1: Copy + Default + core::fmt::Debug + DefaultStrategy,
+    Scalar2: Copy + Default + core::fmt::Debug + DefaultStrategy,
+    ScalarResult: Copy + Default + biteq::BitEq + core::fmt::Debug + DefaultStrategy,
+    Vector1: Into<[Scalar1; LANES]> + From<[Scalar1; LANES]> + Copy,
+    Vector2: Into<[Scalar2; LANES]> + From<[Scalar2; LANES]> + Copy,
+    VectorResult: Into<[ScalarResult; LANES]> + From<[ScalarResult; LANES]> + Copy,
+{
+    test_2(|x: [Scalar1; LANES], y: [Scalar2; LANES]| {
+        let result_1: [ScalarResult; LANES] = fv(x.into(), y.into()).into();
+        let result_2: [ScalarResult; LANES] = {
+            let mut result = [ScalarResult::default(); LANES];
+            for ((i1, i2), o) in x.iter().zip(y.iter()).zip(result.iter_mut()) {
+                *o = fs(*i1, *i2);
+            }
+            result
+        };
+        crate::prop_assert_biteq!(result_1, result_2);
+        Ok(())
+    });
+}
+
+pub fn test_binary_scalar_rhs_elementwise<
+    Scalar1,
+    Scalar2,
+    ScalarResult,
+    Vector,
+    VectorResult,
+    const LANES: usize,
+>(
+    fv: impl Fn(Vector, Scalar2) -> VectorResult,
+    fs: impl Fn(Scalar1, Scalar2) -> ScalarResult,
+) where
+    Scalar1: Copy + Default + core::fmt::Debug + DefaultStrategy,
+    Scalar2: Copy + Default + core::fmt::Debug + DefaultStrategy,
+    ScalarResult: Copy + Default + biteq::BitEq + core::fmt::Debug + DefaultStrategy,
+    Vector: Into<[Scalar1; LANES]> + From<[Scalar1; LANES]> + Copy,
+    VectorResult: Into<[ScalarResult; LANES]> + From<[ScalarResult; LANES]> + Copy,
+{
+    test_2(|x: [Scalar1; LANES], y: Scalar2| {
+        let result_1: [ScalarResult; LANES] = fv(x.into(), y).into();
+        let result_2: [ScalarResult; LANES] = {
+            let mut result = [ScalarResult::default(); LANES];
+            for (i, o) in x.iter().zip(result.iter_mut()) {
+                *o = fs(*i, y);
+            }
+            result
+        };
+        crate::prop_assert_biteq!(result_1, result_2);
+        Ok(())
+    });
+}
+
+pub fn test_binary_scalar_lhs_elementwise<
+    Scalar1,
+    Scalar2,
+    ScalarResult,
+    Vector,
+    VectorResult,
+    const LANES: usize,
+>(
+    fv: impl Fn(Scalar1, Vector) -> VectorResult,
+    fs: impl Fn(Scalar1, Scalar2) -> ScalarResult,
+) where
+    Scalar1: Copy + Default + core::fmt::Debug + DefaultStrategy,
+    Scalar2: Copy + Default + core::fmt::Debug + DefaultStrategy,
+    ScalarResult: Copy + Default + biteq::BitEq + core::fmt::Debug + DefaultStrategy,
+    Vector: Into<[Scalar2; LANES]> + From<[Scalar2; LANES]> + Copy,
+    VectorResult: Into<[ScalarResult; LANES]> + From<[ScalarResult; LANES]> + Copy,
+{
+    test_2(|x: Scalar1, y: [Scalar2; LANES]| {
+        let result_1: [ScalarResult; LANES] = fv(x, y.into()).into();
+        let result_2: [ScalarResult; LANES] = {
+            let mut result = [ScalarResult::default(); LANES];
+            for (i, o) in y.iter().zip(result.iter_mut()) {
+                *o = fs(x, *i);
+            }
+            result
+        };
+        crate::prop_assert_biteq!(result_1, result_2);
+        Ok(())
+    });
+}
+
+#[macro_export]
+#[doc(hidden)]
+macro_rules! test_lanes_impl {
+    {
+        fn $test:ident<const $lanes:ident: usize>() $body:tt
+
+        $($name:ident => $lanes_lit:literal,)*
+    } => {
+        mod $test {
+            use super::*;
+            $(
+                #[test]
+                #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
+                fn $name() {
+                    const $lanes: usize = $lanes_lit;
+                    $body
+                }
+            )*
+        }
+    }
+}
+
+#[macro_export]
+macro_rules! test_lanes {
+    {
+        $(fn $test:ident<const $lanes:ident: usize>() $body:tt)*
+    } => {
+        $(
+        $crate::test_lanes_impl! {
+            fn $test<const $lanes: usize>() $body
+
+            lanes_2 => 2,
+            lanes_3 => 3,
+            lanes_4 => 4,
+            lanes_7 => 7,
+            lanes_8 => 8,
+            lanes_16 => 16,
+            lanes_32 => 32,
+            lanes_64 => 64,
+            lanes_128 => 128,
+            lanes_256 => 256,
+        }
+        )*
+    }
+}

From 0ac057a354a00b4329a47266ea184e5177c7b584 Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Thu, 7 Jan 2021 01:26:29 -0500
Subject: [PATCH 085/249] Add integer tests

---
 crates/core_simd/tests/float.rs   |  12 +-
 crates/core_simd/tests/integer.rs | 189 ++++++++++++++++++++++++++++++
 crates/test_helpers/src/lib.rs    |  11 +-
 3 files changed, 208 insertions(+), 4 deletions(-)
 create mode 100644 crates/core_simd/tests/integer.rs

diff --git a/crates/core_simd/tests/float.rs b/crates/core_simd/tests/float.rs
index 939c18559d2a..56d66239e806 100644
--- a/crates/core_simd/tests/float.rs
+++ b/crates/core_simd/tests/float.rs
@@ -8,6 +8,7 @@ macro_rules! impl_op_test {
                 test_helpers::test_unary_elementwise(
                     <$vector as core::ops::$trait>::$fn,
                     <$scalar as core::ops::$trait>::$fn,
+                    |_| true,
                 );
             }
         }
@@ -21,6 +22,7 @@ macro_rules! impl_op_test {
                     test_helpers::test_binary_elementwise(
                         <$vector as core::ops::$trait>::$fn,
                         <$scalar as core::ops::$trait>::$fn,
+                        |_, _| true,
                     );
                 }
 
@@ -28,6 +30,7 @@ macro_rules! impl_op_test {
                     test_helpers::test_binary_scalar_rhs_elementwise(
                         <$vector as core::ops::$trait<$scalar>>::$fn,
                         <$scalar as core::ops::$trait>::$fn,
+                        |_, _| true,
                     );
                 }
 
@@ -35,6 +38,7 @@ macro_rules! impl_op_test {
                     test_helpers::test_binary_scalar_lhs_elementwise(
                         <$scalar as core::ops::$trait<$vector>>::$fn,
                         <$scalar as core::ops::$trait>::$fn,
+                        |_, _| true,
                     );
                 }
 
@@ -42,6 +46,7 @@ macro_rules! impl_op_test {
                     test_helpers::test_binary_elementwise(
                         |mut a, b| { <$vector as core::ops::$trait_assign>::$fn_assign(&mut a, b); a },
                         |mut a, b| { <$scalar as core::ops::$trait_assign>::$fn_assign(&mut a, b); a },
+                        |_, _| true,
                     )
                 }
 
@@ -49,6 +54,7 @@ macro_rules! impl_op_test {
                     test_helpers::test_binary_scalar_rhs_elementwise(
                         |mut a, b| { <$vector as core::ops::$trait_assign<$scalar>>::$fn_assign(&mut a, b); a },
                         |mut a, b| { <$scalar as core::ops::$trait_assign>::$fn_assign(&mut a, b); a },
+                        |_, _| true,
                     )
                 }
             }
@@ -62,7 +68,7 @@ macro_rules! impl_tests {
             type Vector<const LANES: usize> = core_simd::$vector<LANES>;
             type Scalar = $scalar;
             type IntScalar = $int_scalar;
-            
+
             impl_op_test! { unary, Vector<LANES>, Scalar, Neg::neg }
             impl_op_test! { binary, Vector<LANES>, Scalar, Add::add, AddAssign::add_assign }
             impl_op_test! { binary, Vector<LANES>, Scalar, Sub::sub, SubAssign::sub_assign }
@@ -75,6 +81,7 @@ macro_rules! impl_tests {
                     test_helpers::test_unary_elementwise(
                         Vector::<LANES>::abs,
                         Scalar::abs,
+                        |_| true,
                     )
                 }
 
@@ -82,6 +89,7 @@ macro_rules! impl_tests {
                     test_helpers::test_unary_elementwise(
                         Vector::<LANES>::ceil,
                         Scalar::ceil,
+                        |_| true,
                     )
                 }
 
@@ -89,6 +97,7 @@ macro_rules! impl_tests {
                     test_helpers::test_unary_elementwise(
                         Vector::<LANES>::floor,
                         Scalar::floor,
+                        |_| true,
                     )
                 }
 
@@ -96,6 +105,7 @@ macro_rules! impl_tests {
                     test_helpers::test_unary_elementwise(
                         Vector::<LANES>::round_from_int,
                         |x| x as Scalar,
+                        |_| true,
                     )
                 }
 
diff --git a/crates/core_simd/tests/integer.rs b/crates/core_simd/tests/integer.rs
new file mode 100644
index 000000000000..4f38cdb1ed63
--- /dev/null
+++ b/crates/core_simd/tests/integer.rs
@@ -0,0 +1,189 @@
+#[cfg(target_arch = "wasm32")]
+wasm_bindgen_test_configure!(run_in_browser);
+
+macro_rules! impl_unary_op_test {
+    { $vector:ty, $scalar:ty, $trait:ident :: $fn:ident, $scalar_fn:expr } => {
+        test_helpers::test_lanes! {
+            fn $fn<const LANES: usize>() {
+                test_helpers::test_unary_elementwise(
+                    <$vector as core::ops::$trait>::$fn,
+                    $scalar_fn,
+                    |_| true,
+                );
+            }
+        }
+    };
+    { $vector:ty, $scalar:ty, $trait:ident :: $fn:ident } => {
+        impl_unary_op_test! { $vector, $scalar, $trait::$fn, <$scalar as core::ops::$trait>::$fn }
+    };
+}
+
+macro_rules! impl_binary_op_test {
+    { $vector:ty, $scalar:ty, $trait:ident :: $fn:ident, $trait_assign:ident :: $fn_assign:ident, $scalar_fn:expr } => {
+        mod $fn {
+            use super::*;
+
+            test_helpers::test_lanes! {
+                fn normal<const LANES: usize>() {
+                    test_helpers::test_binary_elementwise(
+                        <$vector as core::ops::$trait>::$fn,
+                        $scalar_fn,
+                        |_, _| true,
+                    );
+                }
+
+                fn scalar_rhs<const LANES: usize>() {
+                    test_helpers::test_binary_scalar_rhs_elementwise(
+                        <$vector as core::ops::$trait<$scalar>>::$fn,
+                        $scalar_fn,
+                        |_, _| true,
+                    );
+                }
+
+                fn scalar_lhs<const LANES: usize>() {
+                    test_helpers::test_binary_scalar_lhs_elementwise(
+                        <$scalar as core::ops::$trait<$vector>>::$fn,
+                        $scalar_fn,
+                        |_, _| true,
+                    );
+                }
+
+                fn assign<const LANES: usize>() {
+                    test_helpers::test_binary_elementwise(
+                        |mut a, b| { <$vector as core::ops::$trait_assign>::$fn_assign(&mut a, b); a },
+                        $scalar_fn,
+                        |_, _| true,
+                    )
+                }
+
+                fn assign_scalar_rhs<const LANES: usize>() {
+                    test_helpers::test_binary_scalar_rhs_elementwise(
+                        |mut a, b| { <$vector as core::ops::$trait_assign<$scalar>>::$fn_assign(&mut a, b); a },
+                        $scalar_fn,
+                        |_, _| true,
+                    )
+                }
+            }
+        }
+    };
+    { $vector:ty, $scalar:ty, $trait:ident :: $fn:ident, $trait_assign:ident :: $fn_assign:ident } => {
+        impl_binary_op_test! { $vector, $scalar, $trait::$fn, $trait_assign::$fn_assign, <$scalar as core::ops::$trait>::$fn }
+    };
+}
+
+macro_rules! impl_binary_checked_op_test {
+    { $vector:ty, $scalar:ty, $trait:ident :: $fn:ident, $trait_assign:ident :: $fn_assign:ident, $scalar_fn:expr, $check_fn:expr } => {
+        mod $fn {
+            use super::*;
+
+            test_helpers::test_lanes! {
+                fn normal<const LANES: usize>() {
+                    test_helpers::test_binary_elementwise(
+                        <$vector as core::ops::$trait>::$fn,
+                        $scalar_fn,
+                        |x, y| x.iter().zip(y.iter()).all(|(x, y)| $check_fn(*x, *y)),
+                    );
+                }
+
+                fn scalar_rhs<const LANES: usize>() {
+                    test_helpers::test_binary_scalar_rhs_elementwise(
+                        <$vector as core::ops::$trait<$scalar>>::$fn,
+                        $scalar_fn,
+                        |x, y| x.iter().all(|x| $check_fn(*x, y)),
+                    );
+                }
+
+                fn scalar_lhs<const LANES: usize>() {
+                    test_helpers::test_binary_scalar_lhs_elementwise(
+                        <$scalar as core::ops::$trait<$vector>>::$fn,
+                        $scalar_fn,
+                        |x, y| y.iter().all(|y| $check_fn(x, *y)),
+                    );
+                }
+
+                fn assign<const LANES: usize>() {
+                    test_helpers::test_binary_elementwise(
+                        |mut a, b| { <$vector as core::ops::$trait_assign>::$fn_assign(&mut a, b); a },
+                        $scalar_fn,
+                        |x, y| x.iter().zip(y.iter()).all(|(x, y)| $check_fn(*x, *y)),
+                    )
+                }
+
+                fn assign_scalar_rhs<const LANES: usize>() {
+                    test_helpers::test_binary_scalar_rhs_elementwise(
+                        |mut a, b| { <$vector as core::ops::$trait_assign<$scalar>>::$fn_assign(&mut a, b); a },
+                        $scalar_fn,
+                        |x, y| x.iter().all(|x| $check_fn(*x, y)),
+                    )
+                }
+            }
+        }
+    };
+    { $vector:ty, $scalar:ty, $trait:ident :: $fn:ident, $trait_assign:ident :: $fn_assign:ident, $check_fn:expr } => {
+        impl_binary_nonzero_rhs_op_test! { $vector, $scalar, $trait::$fn, $trait_assign::$fn_assign, <$scalar as core::ops::$trait>::$fn, $check_fn }
+    };
+}
+
+macro_rules! impl_signed_tests {
+    { $vector:ident, $scalar:tt } => {
+        mod $scalar {
+            type Vector<const LANES: usize> = core_simd::$vector<LANES>;
+            type Scalar = $scalar;
+
+            test_helpers::test_lanes! {
+                fn neg<const LANES: usize>() {
+                    test_helpers::test_unary_elementwise(
+                        <Vector<LANES> as core::ops::Neg>::neg,
+                        <Scalar as core::ops::Neg>::neg,
+                        |x| !x.contains(&Scalar::MIN),
+                    );
+                }
+            }
+
+            impl_binary_op_test!(Vector<LANES>, Scalar, Add::add, AddAssign::add_assign, Scalar::wrapping_add);
+            impl_binary_op_test!(Vector<LANES>, Scalar, Sub::sub, SubAssign::sub_assign, Scalar::wrapping_sub);
+            impl_binary_op_test!(Vector<LANES>, Scalar, Mul::mul, MulAssign::mul_assign, Scalar::wrapping_mul);
+            impl_binary_checked_op_test!(Vector<LANES>, Scalar, Div::div, DivAssign::div_assign, Scalar::wrapping_div, |x, y| y != 0 && !(x == Scalar::MIN && y == -1));
+            impl_binary_checked_op_test!(Vector<LANES>, Scalar, Rem::rem, RemAssign::rem_assign, Scalar::wrapping_rem, |x, y| y != 0 && !(x == Scalar::MIN && y == -1));
+
+            impl_unary_op_test!(Vector<LANES>, Scalar, Not::not);
+            impl_binary_op_test!(Vector<LANES>, Scalar, BitAnd::bitand, BitAndAssign::bitand_assign);
+            impl_binary_op_test!(Vector<LANES>, Scalar, BitOr::bitor, BitOrAssign::bitor_assign);
+            impl_binary_op_test!(Vector<LANES>, Scalar, BitXor::bitxor, BitXorAssign::bitxor_assign);
+        }
+    }
+}
+
+macro_rules! impl_unsigned_tests {
+    { $vector:ident, $scalar:tt } => {
+        mod $scalar {
+            type Vector<const LANES: usize> = core_simd::$vector<LANES>;
+            type Scalar = $scalar;
+
+            impl_binary_op_test!(Vector<LANES>, Scalar, Add::add, AddAssign::add_assign, Scalar::wrapping_add);
+            impl_binary_op_test!(Vector<LANES>, Scalar, Sub::sub, SubAssign::sub_assign, Scalar::wrapping_sub);
+            impl_binary_op_test!(Vector<LANES>, Scalar, Mul::mul, MulAssign::mul_assign, Scalar::wrapping_mul);
+            impl_binary_checked_op_test!(Vector<LANES>, Scalar, Div::div, DivAssign::div_assign, Scalar::wrapping_div, |_, y| y != 0);
+            impl_binary_checked_op_test!(Vector<LANES>, Scalar, Rem::rem, RemAssign::rem_assign, Scalar::wrapping_rem, |_, y| y != 0);
+
+            impl_unary_op_test!(Vector<LANES>, Scalar, Not::not);
+            impl_binary_op_test!(Vector<LANES>, Scalar, BitAnd::bitand, BitAndAssign::bitand_assign);
+            impl_binary_op_test!(Vector<LANES>, Scalar, BitOr::bitor, BitOrAssign::bitor_assign);
+            impl_binary_op_test!(Vector<LANES>, Scalar, BitXor::bitxor, BitXorAssign::bitxor_assign);
+        }
+    }
+}
+
+impl_signed_tests! { SimdI8, i8 }
+impl_signed_tests! { SimdI16, i16 }
+impl_signed_tests! { SimdI32, i32 }
+impl_signed_tests! { SimdI64, i64 }
+impl_signed_tests! { SimdI128, i128 }
+impl_signed_tests! { SimdIsize, isize }
+
+impl_unsigned_tests! { SimdU8, u8 }
+impl_unsigned_tests! { SimdU16, u16 }
+impl_unsigned_tests! { SimdU32, u32 }
+impl_unsigned_tests! { SimdU64, u64 }
+impl_unsigned_tests! { SimdU128, u128 }
+impl_unsigned_tests! { SimdUsize, usize }
diff --git a/crates/test_helpers/src/lib.rs b/crates/test_helpers/src/lib.rs
index 2aaa4641fdfd..c9b2858ccc24 100644
--- a/crates/test_helpers/src/lib.rs
+++ b/crates/test_helpers/src/lib.rs
@@ -62,6 +62,7 @@ pub fn test_2<A: core::fmt::Debug + DefaultStrategy, B: core::fmt::Debug + Defau
 pub fn test_unary_elementwise<Scalar, ScalarResult, Vector, VectorResult, const LANES: usize>(
     fv: impl Fn(Vector) -> VectorResult,
     fs: impl Fn(Scalar) -> ScalarResult,
+    check: impl Fn([Scalar; LANES]) -> bool,
 ) where
     Scalar: Copy + Default + core::fmt::Debug + DefaultStrategy,
     ScalarResult: Copy + Default + biteq::BitEq + core::fmt::Debug + DefaultStrategy,
@@ -69,6 +70,7 @@ pub fn test_unary_elementwise<Scalar, ScalarResult, Vector, VectorResult, const
     VectorResult: Into<[ScalarResult; LANES]> + From<[ScalarResult; LANES]> + Copy,
 {
     test_1(|x: [Scalar; LANES]| {
+        proptest::prop_assume!(check(x));
         let result_1: [ScalarResult; LANES] = fv(x.into()).into();
         let result_2: [ScalarResult; LANES] = {
             let mut result = [ScalarResult::default(); LANES];
@@ -93,6 +95,7 @@ pub fn test_binary_elementwise<
 >(
     fv: impl Fn(Vector1, Vector2) -> VectorResult,
     fs: impl Fn(Scalar1, Scalar2) -> ScalarResult,
+    check: impl Fn([Scalar1; LANES], [Scalar2; LANES]) -> bool,
 ) where
     Scalar1: Copy + Default + core::fmt::Debug + DefaultStrategy,
     Scalar2: Copy + Default + core::fmt::Debug + DefaultStrategy,
@@ -102,6 +105,7 @@ pub fn test_binary_elementwise<
     VectorResult: Into<[ScalarResult; LANES]> + From<[ScalarResult; LANES]> + Copy,
 {
     test_2(|x: [Scalar1; LANES], y: [Scalar2; LANES]| {
+        proptest::prop_assume!(check(x, y));
         let result_1: [ScalarResult; LANES] = fv(x.into(), y.into()).into();
         let result_2: [ScalarResult; LANES] = {
             let mut result = [ScalarResult::default(); LANES];
@@ -125,6 +129,7 @@ pub fn test_binary_scalar_rhs_elementwise<
 >(
     fv: impl Fn(Vector, Scalar2) -> VectorResult,
     fs: impl Fn(Scalar1, Scalar2) -> ScalarResult,
+    check: impl Fn([Scalar1; LANES], Scalar2) -> bool,
 ) where
     Scalar1: Copy + Default + core::fmt::Debug + DefaultStrategy,
     Scalar2: Copy + Default + core::fmt::Debug + DefaultStrategy,
@@ -133,6 +138,7 @@ pub fn test_binary_scalar_rhs_elementwise<
     VectorResult: Into<[ScalarResult; LANES]> + From<[ScalarResult; LANES]> + Copy,
 {
     test_2(|x: [Scalar1; LANES], y: Scalar2| {
+        proptest::prop_assume!(check(x, y));
         let result_1: [ScalarResult; LANES] = fv(x.into(), y).into();
         let result_2: [ScalarResult; LANES] = {
             let mut result = [ScalarResult::default(); LANES];
@@ -156,6 +162,7 @@ pub fn test_binary_scalar_lhs_elementwise<
 >(
     fv: impl Fn(Scalar1, Vector) -> VectorResult,
     fs: impl Fn(Scalar1, Scalar2) -> ScalarResult,
+    check: impl Fn(Scalar1, [Scalar2; LANES]) -> bool,
 ) where
     Scalar1: Copy + Default + core::fmt::Debug + DefaultStrategy,
     Scalar2: Copy + Default + core::fmt::Debug + DefaultStrategy,
@@ -164,6 +171,7 @@ pub fn test_binary_scalar_lhs_elementwise<
     VectorResult: Into<[ScalarResult; LANES]> + From<[ScalarResult; LANES]> + Copy,
 {
     test_2(|x: Scalar1, y: [Scalar2; LANES]| {
+        proptest::prop_assume!(check(x, y));
         let result_1: [ScalarResult; LANES] = fv(x, y.into()).into();
         let result_2: [ScalarResult; LANES] = {
             let mut result = [ScalarResult::default(); LANES];
@@ -215,9 +223,6 @@ macro_rules! test_lanes {
             lanes_8 => 8,
             lanes_16 => 16,
             lanes_32 => 32,
-            lanes_64 => 64,
-            lanes_128 => 128,
-            lanes_256 => 256,
         }
         )*
     }

From 5b0818a2217f2d8030f25f99b566ed5a35394f6c Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Thu, 7 Jan 2021 01:28:17 -0500
Subject: [PATCH 086/249] Remove old integer tests

---
 crates/core_simd/tests/ops_impl/i128.rs       |   4 -
 crates/core_simd/tests/ops_impl/i16.rs        |   6 -
 crates/core_simd/tests/ops_impl/i32.rs        |   6 -
 crates/core_simd/tests/ops_impl/i64.rs        |   5 -
 crates/core_simd/tests/ops_impl/i8.rs         |   6 -
 crates/core_simd/tests/ops_impl/int_macros.rs | 392 ------------------
 crates/core_simd/tests/ops_impl/isize.rs      |   5 -
 crates/core_simd/tests/ops_impl/mod.rs        |  20 -
 8 files changed, 444 deletions(-)
 delete mode 100644 crates/core_simd/tests/ops_impl/i128.rs
 delete mode 100644 crates/core_simd/tests/ops_impl/i16.rs
 delete mode 100644 crates/core_simd/tests/ops_impl/i32.rs
 delete mode 100644 crates/core_simd/tests/ops_impl/i64.rs
 delete mode 100644 crates/core_simd/tests/ops_impl/i8.rs
 delete mode 100644 crates/core_simd/tests/ops_impl/isize.rs

diff --git a/crates/core_simd/tests/ops_impl/i128.rs b/crates/core_simd/tests/ops_impl/i128.rs
deleted file mode 100644
index 8a0a279b8dce..000000000000
--- a/crates/core_simd/tests/ops_impl/i128.rs
+++ /dev/null
@@ -1,4 +0,0 @@
-use super::helpers;
-
-int_tests! { i128x2, i128 }
-int_tests! { i128x4, i128 }
diff --git a/crates/core_simd/tests/ops_impl/i16.rs b/crates/core_simd/tests/ops_impl/i16.rs
deleted file mode 100644
index 445436b77a89..000000000000
--- a/crates/core_simd/tests/ops_impl/i16.rs
+++ /dev/null
@@ -1,6 +0,0 @@
-use super::helpers;
-
-int_tests! { i16x4, i16 }
-int_tests! { i16x8, i16 }
-int_tests! { i16x16, i16 }
-int_tests! { i16x32, i16 }
diff --git a/crates/core_simd/tests/ops_impl/i32.rs b/crates/core_simd/tests/ops_impl/i32.rs
deleted file mode 100644
index f13ab833a335..000000000000
--- a/crates/core_simd/tests/ops_impl/i32.rs
+++ /dev/null
@@ -1,6 +0,0 @@
-use super::helpers;
-
-int_tests! { i32x2, i32 }
-int_tests! { i32x4, i32 }
-int_tests! { i32x8, i32 }
-int_tests! { i32x16, i32 }
diff --git a/crates/core_simd/tests/ops_impl/i64.rs b/crates/core_simd/tests/ops_impl/i64.rs
deleted file mode 100644
index 08479c4b994b..000000000000
--- a/crates/core_simd/tests/ops_impl/i64.rs
+++ /dev/null
@@ -1,5 +0,0 @@
-use super::helpers;
-
-int_tests! { i64x2, i64 }
-int_tests! { i64x4, i64 }
-int_tests! { i64x8, i64 }
diff --git a/crates/core_simd/tests/ops_impl/i8.rs b/crates/core_simd/tests/ops_impl/i8.rs
deleted file mode 100644
index 2a7db7906ac1..000000000000
--- a/crates/core_simd/tests/ops_impl/i8.rs
+++ /dev/null
@@ -1,6 +0,0 @@
-use super::helpers;
-
-int_tests! { i8x8, i8 }
-int_tests! { i8x16, i8 }
-int_tests! { i8x32, i8 }
-int_tests! { i8x64, i8 }
diff --git a/crates/core_simd/tests/ops_impl/int_macros.rs b/crates/core_simd/tests/ops_impl/int_macros.rs
index 43c75c6a9c49..f565ae2f04df 100644
--- a/crates/core_simd/tests/ops_impl/int_macros.rs
+++ b/crates/core_simd/tests/ops_impl/int_macros.rs
@@ -40,194 +40,6 @@ macro_rules! int_tests {
                  -1,  -2,  -3,  -4,  -5,  -6,  -7,  -8,
             ];
 
-            #[test]
-            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
-            fn add() {
-                let a = from_slice(&A);
-                let b = from_slice(&B);
-                let expected = apply_binary_lanewise(a, b, core::ops::Add::add);
-                assert_biteq!(a + b, expected);
-            }
-
-            #[test]
-            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
-            fn add_assign() {
-                let mut a = from_slice(&A);
-                let b = from_slice(&B);
-                let expected = apply_binary_lanewise(a, b, core::ops::Add::add);
-                a += b;
-                assert_biteq!(a, expected);
-            }
-
-            #[test]
-            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
-            fn add_scalar_rhs() {
-                let a = from_slice(&A);
-                let b = 5;
-                let expected = apply_binary_scalar_rhs_lanewise(a, b, core::ops::Add::add);
-                assert_biteq!(a + b, expected);
-            }
-
-            #[test]
-            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
-            fn add_scalar_lhs() {
-                let a = 5;
-                let b = from_slice(&B);
-                let expected = apply_binary_scalar_lhs_lanewise(a, b, core::ops::Add::add);
-                assert_biteq!(a + b, expected);
-            }
-
-            #[test]
-            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
-            fn add_assign_scalar() {
-                let mut a = from_slice(&A);
-                let b = 5;
-                let expected = apply_binary_scalar_rhs_lanewise(a, b, core::ops::Add::add);
-                a += b;
-                assert_biteq!(a, expected);
-            }
-
-            #[test]
-            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
-            fn sub() {
-                let a = from_slice(&A);
-                let b = from_slice(&B);
-                let expected = apply_binary_lanewise(a, b, core::ops::Sub::sub);
-                assert_biteq!(a - b, expected);
-            }
-
-            #[test]
-            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
-            fn sub_assign() {
-                let mut a = from_slice(&A);
-                let b = from_slice(&B);
-                let expected = apply_binary_lanewise(a, b, core::ops::Sub::sub);
-                a -= b;
-                assert_biteq!(a, expected);
-            }
-
-            #[test]
-            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
-            fn sub_scalar_rhs() {
-                let a = from_slice(&A);
-                let b = 5;
-                let expected = apply_binary_scalar_rhs_lanewise(a, b, core::ops::Sub::sub);
-                assert_biteq!(a - b, expected);
-            }
-
-            #[test]
-            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
-            fn sub_scalar_lhs() {
-                let a = 5;
-                let b = from_slice(&B);
-                let expected = apply_binary_scalar_lhs_lanewise(a, b, core::ops::Sub::sub);
-                assert_biteq!(a - b, expected);
-            }
-
-            #[test]
-            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
-            fn sub_assign_scalar() {
-                let mut a = from_slice(&A);
-                let b = 5;
-                let expected = apply_binary_scalar_rhs_lanewise(a, b, core::ops::Sub::sub);
-                a -= b;
-                assert_biteq!(a, expected);
-            }
-
-            #[test]
-            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
-            fn mul() {
-                let a = from_slice(&A);
-                let b = from_slice(&B);
-                let expected = apply_binary_lanewise(a, b, core::ops::Mul::mul);
-                assert_biteq!(a * b, expected);
-            }
-
-            #[test]
-            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
-            fn mul_assign() {
-                let mut a = from_slice(&A);
-                let b = from_slice(&B);
-                let expected = apply_binary_lanewise(a, b, core::ops::Mul::mul);
-                a *= b;
-                assert_biteq!(a, expected);
-            }
-
-            #[test]
-            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
-            fn mul_scalar_rhs() {
-                let a = from_slice(&A);
-                let b = 5;
-                let expected = apply_binary_scalar_rhs_lanewise(a, b, core::ops::Mul::mul);
-                assert_biteq!(a * b, expected);
-            }
-
-            #[test]
-            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
-            fn mul_scalar_lhs() {
-                let a = 5;
-                let b = from_slice(&B);
-                let expected = apply_binary_scalar_lhs_lanewise(a, b, core::ops::Mul::mul);
-                assert_biteq!(a * b, expected);
-            }
-
-            #[test]
-            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
-            fn mul_assign_scalar() {
-                let mut a = from_slice(&A);
-                let b = 5;
-                let expected = apply_binary_scalar_rhs_lanewise(a, b, core::ops::Mul::mul);
-                a *= b;
-                assert_biteq!(a, expected);
-            }
-
-            #[test]
-            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
-            fn div() {
-                let a = from_slice(&A);
-                let b = from_slice(&B);
-                let expected = apply_binary_lanewise(a, b, core::ops::Div::div);
-                assert_biteq!(a / b, expected);
-            }
-
-            #[test]
-            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
-            fn div_assign() {
-                let mut a = from_slice(&A);
-                let b = from_slice(&B);
-                let expected = apply_binary_lanewise(a, b, core::ops::Div::div);
-                a /= b;
-                assert_biteq!(a, expected);
-            }
-
-            #[test]
-            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
-            fn div_scalar_rhs() {
-                let a = from_slice(&A);
-                let b = 5;
-                let expected = apply_binary_scalar_rhs_lanewise(a, b, core::ops::Div::div);
-                assert_biteq!(a / b, expected);
-            }
-
-            #[test]
-            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
-            fn div_scalar_lhs() {
-                let a = 5;
-                let b = from_slice(&B);
-                let expected = apply_binary_scalar_lhs_lanewise(a, b, core::ops::Div::div);
-                assert_biteq!(a / b, expected);
-            }
-
-            #[test]
-            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
-            fn div_assign_scalar() {
-                let mut a = from_slice(&A);
-                let b = 5;
-                let expected = apply_binary_scalar_rhs_lanewise(a, b, core::ops::Div::div);
-                a /= b;
-                assert_biteq!(a, expected);
-            }
-
             #[test]
             #[should_panic]
             fn div_min_panics() {
@@ -261,53 +73,6 @@ macro_rules! int_tests {
                 let _ = a / b;
             }
 
-            #[test]
-            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
-            fn rem() {
-                let a = from_slice(&A);
-                let b = from_slice(&B);
-                let expected = apply_binary_lanewise(a, b, core::ops::Rem::rem);
-                assert_biteq!(a % b, expected);
-            }
-
-            #[test]
-            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
-            fn rem_assign() {
-                let mut a = from_slice(&A);
-                let b = from_slice(&B);
-                let expected = apply_binary_lanewise(a, b, core::ops::Rem::rem);
-                a %= b;
-                assert_biteq!(a, expected);
-            }
-
-            #[test]
-            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
-            fn rem_scalar_rhs() {
-                let a = from_slice(&A);
-                let b = 5;
-                let expected = apply_binary_scalar_rhs_lanewise(a, b, core::ops::Rem::rem);
-                assert_biteq!(a % b, expected);
-            }
-
-            #[test]
-            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
-            fn rem_scalar_lhs() {
-                let a = 5;
-                let b = from_slice(&B);
-                let expected = apply_binary_scalar_lhs_lanewise(a, b, core::ops::Rem::rem);
-                assert_biteq!(a % b, expected);
-            }
-
-            #[test]
-            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
-            fn rem_assign_scalar() {
-                let mut a = from_slice(&A);
-                let b = 5;
-                let expected = apply_binary_scalar_rhs_lanewise(a, b, core::ops::Rem::rem);
-                a %= b;
-                assert_biteq!(a, expected);
-            }
-
             #[test]
             #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn rem_min_neg_one_no_panic() {
@@ -331,163 +96,6 @@ macro_rules! int_tests {
                 let b = from_slice(&vec![0 ; 64]);
                 let _ = a % b;
             }
-
-            #[test]
-            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
-            fn bitand() {
-                let a = from_slice(&A);
-                let b = from_slice(&B);
-                let expected = apply_binary_lanewise(a, b, core::ops::BitAnd::bitand);
-                assert_biteq!(a & b, expected);
-            }
-
-            #[test]
-            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
-            fn bitand_assign() {
-                let mut a = from_slice(&A);
-                let b = from_slice(&B);
-                let expected = apply_binary_lanewise(a, b, core::ops::BitAnd::bitand);
-                a &= b;
-                assert_biteq!(a, expected);
-            }
-
-            #[test]
-            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
-            fn bitand_scalar_rhs() {
-                let a = from_slice(&A);
-                let b = 5;
-                let expected = apply_binary_scalar_rhs_lanewise(a, b, core::ops::BitAnd::bitand);
-                assert_biteq!(a & b, expected);
-            }
-
-            #[test]
-            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
-            fn bitand_scalar_lhs() {
-                let a = 5;
-                let b = from_slice(&B);
-                let expected = apply_binary_scalar_lhs_lanewise(a, b, core::ops::BitAnd::bitand);
-                assert_biteq!(a & b, expected);
-            }
-
-            #[test]
-            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
-            fn bitand_assign_scalar() {
-                let mut a = from_slice(&A);
-                let b = 5;
-                let expected = apply_binary_scalar_rhs_lanewise(a, b, core::ops::BitAnd::bitand);
-                a &= b;
-                assert_biteq!(a, expected);
-            }
-
-            #[test]
-            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
-            fn bitor() {
-                let a = from_slice(&A);
-                let b = from_slice(&B);
-                let expected = apply_binary_lanewise(a, b, core::ops::BitOr::bitor);
-                assert_biteq!(a | b, expected);
-            }
-
-            #[test]
-            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
-            fn bitor_assign() {
-                let mut a = from_slice(&A);
-                let b = from_slice(&B);
-                let expected = apply_binary_lanewise(a, b, core::ops::BitOr::bitor);
-                a |= b;
-                assert_biteq!(a, expected);
-            }
-
-            #[test]
-            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
-            fn bitor_scalar_rhs() {
-                let a = from_slice(&A);
-                let b = 5;
-                let expected = apply_binary_scalar_rhs_lanewise(a, b, core::ops::BitOr::bitor);
-                assert_biteq!(a | b, expected);
-            }
-
-            #[test]
-            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
-            fn bitor_scalar_lhs() {
-                let a = 5;
-                let b = from_slice(&B);
-                let expected = apply_binary_scalar_lhs_lanewise(a, b, core::ops::BitOr::bitor);
-                assert_biteq!(a | b, expected);
-            }
-
-            #[test]
-            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
-            fn bitor_assign_scalar() {
-                let mut a = from_slice(&A);
-                let b = 5;
-                let expected = apply_binary_scalar_rhs_lanewise(a, b, core::ops::BitOr::bitor);
-                a |= b;
-                assert_biteq!(a, expected);
-            }
-
-            #[test]
-            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
-            fn bitxor() {
-                let a = from_slice(&A);
-                let b = from_slice(&B);
-                let expected = apply_binary_lanewise(a, b, core::ops::BitXor::bitxor);
-                assert_biteq!(a ^ b, expected);
-            }
-
-            #[test]
-            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
-            fn bitxor_assign() {
-                let mut a = from_slice(&A);
-                let b = from_slice(&B);
-                let expected = apply_binary_lanewise(a, b, core::ops::BitXor::bitxor);
-                a ^= b;
-                assert_biteq!(a, expected);
-            }
-
-            #[test]
-            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
-            fn bitxor_scalar_rhs() {
-                let a = from_slice(&A);
-                let b = 5;
-                let expected = apply_binary_scalar_rhs_lanewise(a, b, core::ops::BitXor::bitxor);
-                assert_biteq!(a ^ b, expected);
-            }
-
-            #[test]
-            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
-            fn bitxor_scalar_lhs() {
-                let a = 5;
-                let b = from_slice(&B);
-                let expected = apply_binary_scalar_lhs_lanewise(a, b, core::ops::BitXor::bitxor);
-                assert_biteq!(a ^ b, expected);
-            }
-
-            #[test]
-            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
-            fn bitxor_assign_scalar() {
-                let mut a = from_slice(&A);
-                let b = 5;
-                let expected = apply_binary_scalar_rhs_lanewise(a, b, core::ops::BitXor::bitxor);
-                a ^= b;
-                assert_biteq!(a, expected);
-            }
-
-            #[test]
-            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
-            fn neg() {
-                let v = from_slice(&A);
-                let expected = apply_unary_lanewise(v, core::ops::Neg::neg);
-                assert_biteq!(-v, expected);
-            }
-
-            #[test]
-            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
-            fn not() {
-                let v = from_slice(&A);
-                let expected = apply_unary_lanewise(v, core::ops::Not::not);
-                assert_biteq!(!v, expected);
-            }
         }
     }
 }
diff --git a/crates/core_simd/tests/ops_impl/isize.rs b/crates/core_simd/tests/ops_impl/isize.rs
deleted file mode 100644
index 9943e9c357a4..000000000000
--- a/crates/core_simd/tests/ops_impl/isize.rs
+++ /dev/null
@@ -1,5 +0,0 @@
-use super::helpers;
-
-int_tests! { isizex2, isize }
-int_tests! { isizex4, isize }
-int_tests! { isizex8, isize }
diff --git a/crates/core_simd/tests/ops_impl/mod.rs b/crates/core_simd/tests/ops_impl/mod.rs
index 5819eb6beafa..89f5e1b0b52a 100644
--- a/crates/core_simd/tests/ops_impl/mod.rs
+++ b/crates/core_simd/tests/ops_impl/mod.rs
@@ -2,26 +2,6 @@
 #[path = "../helpers/mod.rs"]
 mod helpers;
 
-#[macro_use]
-mod int_macros;
-
-mod r#i8;
-mod r#i16;
-mod r#i32;
-mod r#i64;
-mod r#i128;
-mod r#isize;
-
-#[macro_use]
-mod uint_macros;
-
-mod r#u8;
-mod r#u16;
-mod r#u32;
-mod r#u64;
-mod r#u128;
-mod r#usize;
-
 #[macro_use]
 mod mask_macros;
 

From 223daea83ea60c725944e62815ffae2639a8d652 Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Wed, 10 Feb 2021 23:22:32 -0500
Subject: [PATCH 087/249] Update supported lane counts

---
 crates/test_helpers/src/lib.rs | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/crates/test_helpers/src/lib.rs b/crates/test_helpers/src/lib.rs
index c9b2858ccc24..845dc1fcc44d 100644
--- a/crates/test_helpers/src/lib.rs
+++ b/crates/test_helpers/src/lib.rs
@@ -216,13 +216,13 @@ macro_rules! test_lanes {
         $crate::test_lanes_impl! {
             fn $test<const $lanes: usize>() $body
 
+            lanes_1 => 1,
             lanes_2 => 2,
-            lanes_3 => 3,
             lanes_4 => 4,
-            lanes_7 => 7,
             lanes_8 => 8,
             lanes_16 => 16,
             lanes_32 => 32,
+            lanes_64 => 64,
         }
         )*
     }

From b38d342d7727287c994aefcaa9b74c87e22e2dab Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Sat, 13 Feb 2021 00:41:58 -0500
Subject: [PATCH 088/249] Simplify test creation

---
 crates/test_helpers/src/lib.rs | 78 ++++++++++++++++++++--------------
 1 file changed, 46 insertions(+), 32 deletions(-)

diff --git a/crates/test_helpers/src/lib.rs b/crates/test_helpers/src/lib.rs
index 845dc1fcc44d..77dafe38a10a 100644
--- a/crates/test_helpers/src/lib.rs
+++ b/crates/test_helpers/src/lib.rs
@@ -185,45 +185,59 @@ pub fn test_binary_scalar_lhs_elementwise<
     });
 }
 
-#[macro_export]
-#[doc(hidden)]
-macro_rules! test_lanes_impl {
-    {
-        fn $test:ident<const $lanes:ident: usize>() $body:tt
-
-        $($name:ident => $lanes_lit:literal,)*
-    } => {
-        mod $test {
-            use super::*;
-            $(
-                #[test]
-                #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
-                fn $name() {
-                    const $lanes: usize = $lanes_lit;
-                    $body
-                }
-            )*
-        }
-    }
-}
-
 #[macro_export]
 macro_rules! test_lanes {
     {
         $(fn $test:ident<const $lanes:ident: usize>() $body:tt)*
     } => {
         $(
-        $crate::test_lanes_impl! {
-            fn $test<const $lanes: usize>() $body
+            mod $test {
+                use super::*;
 
-            lanes_1 => 1,
-            lanes_2 => 2,
-            lanes_4 => 4,
-            lanes_8 => 8,
-            lanes_16 => 16,
-            lanes_32 => 32,
-            lanes_64 => 64,
-        }
+                fn implementation<const $lanes: usize>() $body
+
+                #[test]
+                #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
+                fn lanes_1() {
+                    implementation::<1>();
+                }
+
+                #[test]
+                #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
+                fn lanes_2() {
+                    implementation::<2>();
+                }
+
+                #[test]
+                #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
+                fn lanes_4() {
+                    implementation::<4>();
+                }
+
+                #[test]
+                #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
+                fn lanes_8() {
+                    implementation::<8>();
+                }
+
+                #[test]
+                #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
+                fn lanes_16() {
+                    implementation::<16>();
+                }
+
+                #[test]
+                #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
+                fn lanes_32() {
+                    implementation::<32>();
+                }
+
+                #[test]
+                #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
+                fn lanes_64() {
+                    implementation::<64>();
+                }
+            }
         )*
     }
 }

From 38b18904d0399d8dcfee5dfc94e38d78a8fbba66 Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Sat, 13 Feb 2021 01:11:01 -0500
Subject: [PATCH 089/249] Remove obsolete helpers

---
 crates/core_simd/tests/helpers/lanewise.rs | 61 ----------------------
 crates/core_simd/tests/helpers/mod.rs      |  2 -
 2 files changed, 63 deletions(-)
 delete mode 100644 crates/core_simd/tests/helpers/lanewise.rs

diff --git a/crates/core_simd/tests/helpers/lanewise.rs b/crates/core_simd/tests/helpers/lanewise.rs
deleted file mode 100644
index 3a9f47968080..000000000000
--- a/crates/core_simd/tests/helpers/lanewise.rs
+++ /dev/null
@@ -1,61 +0,0 @@
-//! These helpers provide a way to easily emulate a vectorized SIMD op on two SIMD vectors,
-//! except using scalar ops that iterate through each lane, one at a time, so as to remove
-//! the vagaries of compilation.
-//!
-//! Do note, however, that when testing that vectorized operations #[should_panic], these
-//! "scalarized SIMD ops" will trigger scalar code paths that may also normally panic.
-
-pub fn apply_unary_lanewise<T1: Copy, T2: Copy, V1: AsRef<[T1]>, V2: AsMut<[T2]> + Default>(
-    x: V1,
-    f: impl Fn(T1) -> T2,
-) -> V2 {
-    let mut y = V2::default();
-    assert_eq!(x.as_ref().len(), y.as_mut().len());
-    for (x, y) in x.as_ref().iter().zip(y.as_mut().iter_mut()) {
-        *y = f(*x);
-    }
-    y
-}
-
-pub fn apply_binary_lanewise<T: Copy, V: AsRef<[T]> + AsMut<[T]> + Default>(
-    a: V,
-    b: V,
-    f: impl Fn(T, T) -> T,
-) -> V {
-    let mut out = V::default();
-    let out_slice = out.as_mut();
-    let a_slice = a.as_ref();
-    let b_slice = b.as_ref();
-    for (o, (a, b)) in out_slice.iter_mut().zip(a_slice.iter().zip(b_slice.iter())) {
-        *o = f(*a, *b);
-    }
-    out
-}
-
-pub fn apply_binary_scalar_rhs_lanewise<T: Copy, V: AsRef<[T]> + AsMut<[T]> + Default>(
-    a: V,
-    b: T,
-    f: impl Fn(T, T) -> T,
-) -> V {
-    let mut out = V::default();
-    let out_slice = out.as_mut();
-    let a_slice = a.as_ref();
-    for (o, a) in out_slice.iter_mut().zip(a_slice.iter()) {
-        *o = f(*a, b);
-    }
-    out
-}
-
-pub fn apply_binary_scalar_lhs_lanewise<T: Copy, V: AsRef<[T]> + AsMut<[T]> + Default>(
-    a: T,
-    b: V,
-    f: impl Fn(T, T) -> T,
-) -> V {
-    let mut out = V::default();
-    let out_slice = out.as_mut();
-    let b_slice = b.as_ref();
-    for (o, b) in out_slice.iter_mut().zip(b_slice.iter()) {
-        *o = f(a, *b);
-    }
-    out
-}
diff --git a/crates/core_simd/tests/helpers/mod.rs b/crates/core_simd/tests/helpers/mod.rs
index b128f8251ca3..41b4fea9d95d 100644
--- a/crates/core_simd/tests/helpers/mod.rs
+++ b/crates/core_simd/tests/helpers/mod.rs
@@ -1,4 +1,2 @@
 #[macro_use]
 pub mod biteq;
-
-pub mod lanewise;

From 8d5702e437e051333dcf1005fcce140c5c96f759 Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Sat, 13 Feb 2021 02:14:47 -0500
Subject: [PATCH 090/249] Fix performance issues

---
 crates/core_simd/tests/float.rs   | 60 ++++++++++++------------
 crates/core_simd/tests/integer.rs | 76 +++++++++++++++----------------
 crates/test_helpers/src/lib.rs    | 40 ++++++++--------
 3 files changed, 90 insertions(+), 86 deletions(-)

diff --git a/crates/core_simd/tests/float.rs b/crates/core_simd/tests/float.rs
index 56d66239e806..03d132ae0468 100644
--- a/crates/core_simd/tests/float.rs
+++ b/crates/core_simd/tests/float.rs
@@ -6,9 +6,9 @@ macro_rules! impl_op_test {
         test_helpers::test_lanes! {
             fn $fn<const LANES: usize>() {
                 test_helpers::test_unary_elementwise(
-                    <$vector as core::ops::$trait>::$fn,
-                    <$scalar as core::ops::$trait>::$fn,
-                    |_| true,
+                    &<$vector as core::ops::$trait>::$fn,
+                    &<$scalar as core::ops::$trait>::$fn,
+                    &|_| true,
                 );
             }
         }
@@ -20,41 +20,41 @@ macro_rules! impl_op_test {
             test_helpers::test_lanes! {
                 fn normal<const LANES: usize>() {
                     test_helpers::test_binary_elementwise(
-                        <$vector as core::ops::$trait>::$fn,
-                        <$scalar as core::ops::$trait>::$fn,
-                        |_, _| true,
+                        &<$vector as core::ops::$trait>::$fn,
+                        &<$scalar as core::ops::$trait>::$fn,
+                        &|_, _| true,
                     );
                 }
 
                 fn scalar_rhs<const LANES: usize>() {
                     test_helpers::test_binary_scalar_rhs_elementwise(
-                        <$vector as core::ops::$trait<$scalar>>::$fn,
-                        <$scalar as core::ops::$trait>::$fn,
-                        |_, _| true,
+                        &<$vector as core::ops::$trait<$scalar>>::$fn,
+                        &<$scalar as core::ops::$trait>::$fn,
+                        &|_, _| true,
                     );
                 }
 
                 fn scalar_lhs<const LANES: usize>() {
                     test_helpers::test_binary_scalar_lhs_elementwise(
-                        <$scalar as core::ops::$trait<$vector>>::$fn,
-                        <$scalar as core::ops::$trait>::$fn,
-                        |_, _| true,
+                        &<$scalar as core::ops::$trait<$vector>>::$fn,
+                        &<$scalar as core::ops::$trait>::$fn,
+                        &|_, _| true,
                     );
                 }
 
                 fn assign<const LANES: usize>() {
                     test_helpers::test_binary_elementwise(
-                        |mut a, b| { <$vector as core::ops::$trait_assign>::$fn_assign(&mut a, b); a },
-                        |mut a, b| { <$scalar as core::ops::$trait_assign>::$fn_assign(&mut a, b); a },
-                        |_, _| true,
+                        &|mut a, b| { <$vector as core::ops::$trait_assign>::$fn_assign(&mut a, b); a },
+                        &|mut a, b| { <$scalar as core::ops::$trait_assign>::$fn_assign(&mut a, b); a },
+                        &|_, _| true,
                     )
                 }
 
                 fn assign_scalar_rhs<const LANES: usize>() {
                     test_helpers::test_binary_scalar_rhs_elementwise(
-                        |mut a, b| { <$vector as core::ops::$trait_assign<$scalar>>::$fn_assign(&mut a, b); a },
-                        |mut a, b| { <$scalar as core::ops::$trait_assign>::$fn_assign(&mut a, b); a },
-                        |_, _| true,
+                        &|mut a, b| { <$vector as core::ops::$trait_assign<$scalar>>::$fn_assign(&mut a, b); a },
+                        &|mut a, b| { <$scalar as core::ops::$trait_assign>::$fn_assign(&mut a, b); a },
+                        &|_, _| true,
                     )
                 }
             }
@@ -79,33 +79,33 @@ macro_rules! impl_tests {
             test_helpers::test_lanes! {
                 fn abs<const LANES: usize>() {
                     test_helpers::test_unary_elementwise(
-                        Vector::<LANES>::abs,
-                        Scalar::abs,
-                        |_| true,
+                        &Vector::<LANES>::abs,
+                        &Scalar::abs,
+                        &|_| true,
                     )
                 }
 
                 fn ceil<const LANES: usize>() {
                     test_helpers::test_unary_elementwise(
-                        Vector::<LANES>::ceil,
-                        Scalar::ceil,
-                        |_| true,
+                        &Vector::<LANES>::ceil,
+                        &Scalar::ceil,
+                        &|_| true,
                     )
                 }
 
                 fn floor<const LANES: usize>() {
                     test_helpers::test_unary_elementwise(
-                        Vector::<LANES>::floor,
-                        Scalar::floor,
-                        |_| true,
+                        &Vector::<LANES>::floor,
+                        &Scalar::floor,
+                        &|_| true,
                     )
                 }
 
                 fn round_from_int<const LANES: usize>() {
                     test_helpers::test_unary_elementwise(
-                        Vector::<LANES>::round_from_int,
-                        |x| x as Scalar,
-                        |_| true,
+                        &Vector::<LANES>::round_from_int,
+                        &|x| x as Scalar,
+                        &|_| true,
                     )
                 }
 
diff --git a/crates/core_simd/tests/integer.rs b/crates/core_simd/tests/integer.rs
index 4f38cdb1ed63..878b3f0329a4 100644
--- a/crates/core_simd/tests/integer.rs
+++ b/crates/core_simd/tests/integer.rs
@@ -6,9 +6,9 @@ macro_rules! impl_unary_op_test {
         test_helpers::test_lanes! {
             fn $fn<const LANES: usize>() {
                 test_helpers::test_unary_elementwise(
-                    <$vector as core::ops::$trait>::$fn,
-                    $scalar_fn,
-                    |_| true,
+                    &<$vector as core::ops::$trait>::$fn,
+                    &$scalar_fn,
+                    &|_| true,
                 );
             }
         }
@@ -26,42 +26,42 @@ macro_rules! impl_binary_op_test {
             test_helpers::test_lanes! {
                 fn normal<const LANES: usize>() {
                     test_helpers::test_binary_elementwise(
-                        <$vector as core::ops::$trait>::$fn,
-                        $scalar_fn,
-                        |_, _| true,
+                        &<$vector as core::ops::$trait>::$fn,
+                        &$scalar_fn,
+                        &|_, _| true,
                     );
                 }
 
                 fn scalar_rhs<const LANES: usize>() {
                     test_helpers::test_binary_scalar_rhs_elementwise(
-                        <$vector as core::ops::$trait<$scalar>>::$fn,
-                        $scalar_fn,
-                        |_, _| true,
+                        &<$vector as core::ops::$trait<$scalar>>::$fn,
+                        &$scalar_fn,
+                        &|_, _| true,
                     );
                 }
 
                 fn scalar_lhs<const LANES: usize>() {
                     test_helpers::test_binary_scalar_lhs_elementwise(
-                        <$scalar as core::ops::$trait<$vector>>::$fn,
-                        $scalar_fn,
-                        |_, _| true,
+                        &<$scalar as core::ops::$trait<$vector>>::$fn,
+                        &$scalar_fn,
+                        &|_, _| true,
                     );
                 }
 
                 fn assign<const LANES: usize>() {
                     test_helpers::test_binary_elementwise(
-                        |mut a, b| { <$vector as core::ops::$trait_assign>::$fn_assign(&mut a, b); a },
-                        $scalar_fn,
-                        |_, _| true,
-                    )
+                        &|mut a, b| { <$vector as core::ops::$trait_assign>::$fn_assign(&mut a, b); a },
+                        &$scalar_fn,
+                        &|_, _| true,
+                    );
                 }
 
                 fn assign_scalar_rhs<const LANES: usize>() {
                     test_helpers::test_binary_scalar_rhs_elementwise(
-                        |mut a, b| { <$vector as core::ops::$trait_assign<$scalar>>::$fn_assign(&mut a, b); a },
-                        $scalar_fn,
-                        |_, _| true,
-                    )
+                        &|mut a, b| { <$vector as core::ops::$trait_assign<$scalar>>::$fn_assign(&mut a, b); a },
+                        &$scalar_fn,
+                        &|_, _| true,
+                    );
                 }
             }
         }
@@ -79,41 +79,41 @@ macro_rules! impl_binary_checked_op_test {
             test_helpers::test_lanes! {
                 fn normal<const LANES: usize>() {
                     test_helpers::test_binary_elementwise(
-                        <$vector as core::ops::$trait>::$fn,
-                        $scalar_fn,
-                        |x, y| x.iter().zip(y.iter()).all(|(x, y)| $check_fn(*x, *y)),
+                        &<$vector as core::ops::$trait>::$fn,
+                        &$scalar_fn,
+                        &|x, y| x.iter().zip(y.iter()).all(|(x, y)| $check_fn(*x, *y)),
                     );
                 }
 
                 fn scalar_rhs<const LANES: usize>() {
                     test_helpers::test_binary_scalar_rhs_elementwise(
-                        <$vector as core::ops::$trait<$scalar>>::$fn,
-                        $scalar_fn,
-                        |x, y| x.iter().all(|x| $check_fn(*x, y)),
+                        &<$vector as core::ops::$trait<$scalar>>::$fn,
+                        &$scalar_fn,
+                        &|x, y| x.iter().all(|x| $check_fn(*x, y)),
                     );
                 }
 
                 fn scalar_lhs<const LANES: usize>() {
                     test_helpers::test_binary_scalar_lhs_elementwise(
-                        <$scalar as core::ops::$trait<$vector>>::$fn,
-                        $scalar_fn,
-                        |x, y| y.iter().all(|y| $check_fn(x, *y)),
+                        &<$scalar as core::ops::$trait<$vector>>::$fn,
+                        &$scalar_fn,
+                        &|x, y| y.iter().all(|y| $check_fn(x, *y)),
                     );
                 }
 
                 fn assign<const LANES: usize>() {
                     test_helpers::test_binary_elementwise(
-                        |mut a, b| { <$vector as core::ops::$trait_assign>::$fn_assign(&mut a, b); a },
-                        $scalar_fn,
-                        |x, y| x.iter().zip(y.iter()).all(|(x, y)| $check_fn(*x, *y)),
+                        &|mut a, b| { <$vector as core::ops::$trait_assign>::$fn_assign(&mut a, b); a },
+                        &$scalar_fn,
+                        &|x, y| x.iter().zip(y.iter()).all(|(x, y)| $check_fn(*x, *y)),
                     )
                 }
 
                 fn assign_scalar_rhs<const LANES: usize>() {
                     test_helpers::test_binary_scalar_rhs_elementwise(
-                        |mut a, b| { <$vector as core::ops::$trait_assign<$scalar>>::$fn_assign(&mut a, b); a },
-                        $scalar_fn,
-                        |x, y| x.iter().all(|x| $check_fn(*x, y)),
+                        &|mut a, b| { <$vector as core::ops::$trait_assign<$scalar>>::$fn_assign(&mut a, b); a },
+                        &$scalar_fn,
+                        &|x, y| x.iter().all(|x| $check_fn(*x, y)),
                     )
                 }
             }
@@ -133,9 +133,9 @@ macro_rules! impl_signed_tests {
             test_helpers::test_lanes! {
                 fn neg<const LANES: usize>() {
                     test_helpers::test_unary_elementwise(
-                        <Vector<LANES> as core::ops::Neg>::neg,
-                        <Scalar as core::ops::Neg>::neg,
-                        |x| !x.contains(&Scalar::MIN),
+                        &<Vector<LANES> as core::ops::Neg>::neg,
+                        &<Scalar as core::ops::Neg>::neg,
+                        &|x| !x.contains(&Scalar::MIN),
                     );
                 }
             }
diff --git a/crates/test_helpers/src/lib.rs b/crates/test_helpers/src/lib.rs
index 77dafe38a10a..134b4073a4e2 100644
--- a/crates/test_helpers/src/lib.rs
+++ b/crates/test_helpers/src/lib.rs
@@ -42,14 +42,14 @@ impl<T: core::fmt::Debug + DefaultStrategy, const LANES: usize> DefaultStrategy
 }
 
 pub fn test_1<A: core::fmt::Debug + DefaultStrategy>(
-    f: impl Fn(A) -> proptest::test_runner::TestCaseResult,
+    f: &dyn Fn(A) -> proptest::test_runner::TestCaseResult,
 ) {
     let mut runner = proptest::test_runner::TestRunner::default();
     runner.run(&A::default_strategy(), f).unwrap();
 }
 
 pub fn test_2<A: core::fmt::Debug + DefaultStrategy, B: core::fmt::Debug + DefaultStrategy>(
-    f: impl Fn(A, B) -> proptest::test_runner::TestCaseResult,
+    f: &dyn Fn(A, B) -> proptest::test_runner::TestCaseResult,
 ) {
     let mut runner = proptest::test_runner::TestRunner::default();
     runner
@@ -59,17 +59,18 @@ pub fn test_2<A: core::fmt::Debug + DefaultStrategy, B: core::fmt::Debug + Defau
         .unwrap();
 }
 
+#[inline(never)]
 pub fn test_unary_elementwise<Scalar, ScalarResult, Vector, VectorResult, const LANES: usize>(
-    fv: impl Fn(Vector) -> VectorResult,
-    fs: impl Fn(Scalar) -> ScalarResult,
-    check: impl Fn([Scalar; LANES]) -> bool,
+    fv: &dyn Fn(Vector) -> VectorResult,
+    fs: &dyn Fn(Scalar) -> ScalarResult,
+    check: &dyn Fn([Scalar; LANES]) -> bool,
 ) where
     Scalar: Copy + Default + core::fmt::Debug + DefaultStrategy,
     ScalarResult: Copy + Default + biteq::BitEq + core::fmt::Debug + DefaultStrategy,
     Vector: Into<[Scalar; LANES]> + From<[Scalar; LANES]> + Copy,
     VectorResult: Into<[ScalarResult; LANES]> + From<[ScalarResult; LANES]> + Copy,
 {
-    test_1(|x: [Scalar; LANES]| {
+    test_1(&|x: [Scalar; LANES]| {
         proptest::prop_assume!(check(x));
         let result_1: [ScalarResult; LANES] = fv(x.into()).into();
         let result_2: [ScalarResult; LANES] = {
@@ -84,6 +85,7 @@ pub fn test_unary_elementwise<Scalar, ScalarResult, Vector, VectorResult, const
     });
 }
 
+#[inline(never)]
 pub fn test_binary_elementwise<
     Scalar1,
     Scalar2,
@@ -93,9 +95,9 @@ pub fn test_binary_elementwise<
     VectorResult,
     const LANES: usize,
 >(
-    fv: impl Fn(Vector1, Vector2) -> VectorResult,
-    fs: impl Fn(Scalar1, Scalar2) -> ScalarResult,
-    check: impl Fn([Scalar1; LANES], [Scalar2; LANES]) -> bool,
+    fv: &dyn Fn(Vector1, Vector2) -> VectorResult,
+    fs: &dyn Fn(Scalar1, Scalar2) -> ScalarResult,
+    check: &dyn Fn([Scalar1; LANES], [Scalar2; LANES]) -> bool,
 ) where
     Scalar1: Copy + Default + core::fmt::Debug + DefaultStrategy,
     Scalar2: Copy + Default + core::fmt::Debug + DefaultStrategy,
@@ -104,7 +106,7 @@ pub fn test_binary_elementwise<
     Vector2: Into<[Scalar2; LANES]> + From<[Scalar2; LANES]> + Copy,
     VectorResult: Into<[ScalarResult; LANES]> + From<[ScalarResult; LANES]> + Copy,
 {
-    test_2(|x: [Scalar1; LANES], y: [Scalar2; LANES]| {
+    test_2(&|x: [Scalar1; LANES], y: [Scalar2; LANES]| {
         proptest::prop_assume!(check(x, y));
         let result_1: [ScalarResult; LANES] = fv(x.into(), y.into()).into();
         let result_2: [ScalarResult; LANES] = {
@@ -119,6 +121,7 @@ pub fn test_binary_elementwise<
     });
 }
 
+#[inline(never)]
 pub fn test_binary_scalar_rhs_elementwise<
     Scalar1,
     Scalar2,
@@ -127,9 +130,9 @@ pub fn test_binary_scalar_rhs_elementwise<
     VectorResult,
     const LANES: usize,
 >(
-    fv: impl Fn(Vector, Scalar2) -> VectorResult,
-    fs: impl Fn(Scalar1, Scalar2) -> ScalarResult,
-    check: impl Fn([Scalar1; LANES], Scalar2) -> bool,
+    fv: &dyn Fn(Vector, Scalar2) -> VectorResult,
+    fs: &dyn Fn(Scalar1, Scalar2) -> ScalarResult,
+    check: &dyn Fn([Scalar1; LANES], Scalar2) -> bool,
 ) where
     Scalar1: Copy + Default + core::fmt::Debug + DefaultStrategy,
     Scalar2: Copy + Default + core::fmt::Debug + DefaultStrategy,
@@ -137,7 +140,7 @@ pub fn test_binary_scalar_rhs_elementwise<
     Vector: Into<[Scalar1; LANES]> + From<[Scalar1; LANES]> + Copy,
     VectorResult: Into<[ScalarResult; LANES]> + From<[ScalarResult; LANES]> + Copy,
 {
-    test_2(|x: [Scalar1; LANES], y: Scalar2| {
+    test_2(&|x: [Scalar1; LANES], y: Scalar2| {
         proptest::prop_assume!(check(x, y));
         let result_1: [ScalarResult; LANES] = fv(x.into(), y).into();
         let result_2: [ScalarResult; LANES] = {
@@ -152,6 +155,7 @@ pub fn test_binary_scalar_rhs_elementwise<
     });
 }
 
+#[inline(never)]
 pub fn test_binary_scalar_lhs_elementwise<
     Scalar1,
     Scalar2,
@@ -160,9 +164,9 @@ pub fn test_binary_scalar_lhs_elementwise<
     VectorResult,
     const LANES: usize,
 >(
-    fv: impl Fn(Scalar1, Vector) -> VectorResult,
-    fs: impl Fn(Scalar1, Scalar2) -> ScalarResult,
-    check: impl Fn(Scalar1, [Scalar2; LANES]) -> bool,
+    fv: &dyn Fn(Scalar1, Vector) -> VectorResult,
+    fs: &dyn Fn(Scalar1, Scalar2) -> ScalarResult,
+    check: &dyn Fn(Scalar1, [Scalar2; LANES]) -> bool,
 ) where
     Scalar1: Copy + Default + core::fmt::Debug + DefaultStrategy,
     Scalar2: Copy + Default + core::fmt::Debug + DefaultStrategy,
@@ -170,7 +174,7 @@ pub fn test_binary_scalar_lhs_elementwise<
     Vector: Into<[Scalar2; LANES]> + From<[Scalar2; LANES]> + Copy,
     VectorResult: Into<[ScalarResult; LANES]> + From<[ScalarResult; LANES]> + Copy,
 {
-    test_2(|x: Scalar1, y: [Scalar2; LANES]| {
+    test_2(&|x: Scalar1, y: [Scalar2; LANES]| {
         proptest::prop_assume!(check(x, y));
         let result_1: [ScalarResult; LANES] = fv(x, y.into()).into();
         let result_2: [ScalarResult; LANES] = {

From 976fafcf4fb8008255ce57bc62738a46f8a9152f Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Sat, 13 Feb 2021 11:40:10 -0500
Subject: [PATCH 091/249] Fix wasm tests

---
 crates/core_simd/src/macros.rs    |  2 +-
 crates/core_simd/tests/float.rs   |  3 --
 crates/core_simd/tests/integer.rs |  3 --
 crates/test_helpers/Cargo.toml    |  6 ++--
 crates/test_helpers/src/array.rs  |  2 +-
 crates/test_helpers/src/lib.rs    | 54 +++++++++++++++++++++++++------
 crates/test_helpers/src/wasm.rs   | 49 ++++++++++++++++++++++++++++
 7 files changed, 100 insertions(+), 19 deletions(-)
 create mode 100644 crates/test_helpers/src/wasm.rs

diff --git a/crates/core_simd/src/macros.rs b/crates/core_simd/src/macros.rs
index 3e428379b74d..75104dc50e45 100644
--- a/crates/core_simd/src/macros.rs
+++ b/crates/core_simd/src/macros.rs
@@ -143,7 +143,7 @@ macro_rules! impl_vector {
 
         impl <const LANES: usize> From<$name<LANES>> for [$type; LANES] {
             fn from(vector: $name<LANES>) -> Self {
-                vector.0
+                vector.to_array()
             }
         }
 
diff --git a/crates/core_simd/tests/float.rs b/crates/core_simd/tests/float.rs
index 03d132ae0468..618a75250bda 100644
--- a/crates/core_simd/tests/float.rs
+++ b/crates/core_simd/tests/float.rs
@@ -1,6 +1,3 @@
-#[cfg(target_arch = "wasm32")]
-wasm_bindgen_test_configure!(run_in_browser);
-
 macro_rules! impl_op_test {
     { unary, $vector:ty, $scalar:ty, $trait:ident :: $fn:ident } => {
         test_helpers::test_lanes! {
diff --git a/crates/core_simd/tests/integer.rs b/crates/core_simd/tests/integer.rs
index 878b3f0329a4..336126283563 100644
--- a/crates/core_simd/tests/integer.rs
+++ b/crates/core_simd/tests/integer.rs
@@ -1,6 +1,3 @@
-#[cfg(target_arch = "wasm32")]
-wasm_bindgen_test_configure!(run_in_browser);
-
 macro_rules! impl_unary_op_test {
     { $vector:ty, $scalar:ty, $trait:ident :: $fn:ident, $scalar_fn:expr } => {
         test_helpers::test_lanes! {
diff --git a/crates/test_helpers/Cargo.toml b/crates/test_helpers/Cargo.toml
index 0a8c3344334c..c9f6397b23b9 100644
--- a/crates/test_helpers/Cargo.toml
+++ b/crates/test_helpers/Cargo.toml
@@ -5,5 +5,7 @@ authors = ["Caleb Zulawski <caleb.zulawski@gmail.com>"]
 edition = "2018"
 publish = false
 
-[dependencies]
-proptest = "0.10"
+[dependencies.proptest]
+version = "0.10"
+default-features = false
+features = ["alloc"]
diff --git a/crates/test_helpers/src/array.rs b/crates/test_helpers/src/array.rs
index d9cae96ca2ff..3953d0bbea5e 100644
--- a/crates/test_helpers/src/array.rs
+++ b/crates/test_helpers/src/array.rs
@@ -18,7 +18,7 @@ pub struct UniformArrayStrategy<S, T> {
 }
 
 impl<S, T> UniformArrayStrategy<S, T> {
-    pub fn new(strategy: S) -> Self {
+    pub const fn new(strategy: S) -> Self {
         Self {
             strategy,
             _marker: PhantomData,
diff --git a/crates/test_helpers/src/lib.rs b/crates/test_helpers/src/lib.rs
index 134b4073a4e2..e1832bf63779 100644
--- a/crates/test_helpers/src/lib.rs
+++ b/crates/test_helpers/src/lib.rs
@@ -1,5 +1,8 @@
 pub mod array;
 
+#[cfg(target_arch = "wasm32")]
+pub mod wasm;
+
 #[macro_use]
 pub mod biteq;
 
@@ -23,17 +26,47 @@ impl_num! { i8 }
 impl_num! { i16 }
 impl_num! { i32 }
 impl_num! { i64 }
-impl_num! { i128 }
 impl_num! { isize }
 impl_num! { u8 }
 impl_num! { u16 }
 impl_num! { u32 }
 impl_num! { u64 }
-impl_num! { u128 }
 impl_num! { usize }
 impl_num! { f32 }
 impl_num! { f64 }
 
+#[cfg(not(target_arch = "wasm32"))]
+impl DefaultStrategy for u128 {
+    type Strategy = proptest::num::u128::Any;
+    fn default_strategy() -> Self::Strategy {
+        proptest::num::u128::ANY
+    }
+}
+
+#[cfg(not(target_arch = "wasm32"))]
+impl DefaultStrategy for i128 {
+    type Strategy = proptest::num::i128::Any;
+    fn default_strategy() -> Self::Strategy {
+        proptest::num::i128::ANY
+    }
+}
+
+#[cfg(target_arch = "wasm32")]
+impl DefaultStrategy for u128 {
+    type Strategy = crate::wasm::u128::Any;
+    fn default_strategy() -> Self::Strategy {
+        crate::wasm::u128::ANY
+    }
+}
+
+#[cfg(target_arch = "wasm32")]
+impl DefaultStrategy for i128 {
+    type Strategy = crate::wasm::i128::Any;
+    fn default_strategy() -> Self::Strategy {
+        crate::wasm::i128::ANY
+    }
+}
+
 impl<T: core::fmt::Debug + DefaultStrategy, const LANES: usize> DefaultStrategy for [T; LANES] {
     type Strategy = crate::array::UniformArrayStrategy<T::Strategy, Self>;
     fn default_strategy() -> Self::Strategy {
@@ -200,44 +233,47 @@ macro_rules! test_lanes {
 
                 fn implementation<const $lanes: usize>() $body
 
+                #[cfg(target_arch = "wasm32")]
+                wasm_bindgen_test::wasm_bindgen_test_configure!(run_in_browser);
+
                 #[test]
-                #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
+                #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test::wasm_bindgen_test)]
                 fn lanes_1() {
                     implementation::<1>();
                 }
 
                 #[test]
-                #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
+                #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test::wasm_bindgen_test)]
                 fn lanes_2() {
                     implementation::<2>();
                 }
 
                 #[test]
-                #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
+                #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test::wasm_bindgen_test)]
                 fn lanes_4() {
                     implementation::<4>();
                 }
 
                 #[test]
-                #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
+                #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test::wasm_bindgen_test)]
                 fn lanes_8() {
                     implementation::<8>();
                 }
 
                 #[test]
-                #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
+                #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test::wasm_bindgen_test)]
                 fn lanes_16() {
                     implementation::<16>();
                 }
 
                 #[test]
-                #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
+                #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test::wasm_bindgen_test)]
                 fn lanes_32() {
                     implementation::<32>();
                 }
 
                 #[test]
-                #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
+                #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test::wasm_bindgen_test)]
                 fn lanes_64() {
                     implementation::<64>();
                 }
diff --git a/crates/test_helpers/src/wasm.rs b/crates/test_helpers/src/wasm.rs
new file mode 100644
index 000000000000..02cb9264d7c7
--- /dev/null
+++ b/crates/test_helpers/src/wasm.rs
@@ -0,0 +1,49 @@
+macro_rules! impl_num {
+    { $name:ident } => {
+        pub(crate) mod $name {
+            type InnerStrategy = crate::array::UniformArrayStrategy<proptest::num::u64::Any, [u64; 2]>;
+            use proptest::strategy::{Strategy, ValueTree, NewTree};
+
+
+            #[must_use = "strategies do nothing unless used"]
+            #[derive(Clone, Copy, Debug)]
+            pub struct Any {
+                strategy: InnerStrategy,
+            }
+
+            pub struct BinarySearch {
+                inner: <InnerStrategy as Strategy>::Tree,
+            }
+
+            impl ValueTree for BinarySearch {
+                type Value = $name;
+
+                fn current(&self) -> $name {
+                    unsafe { core::mem::transmute(self.inner.current()) }
+                }
+
+                fn simplify(&mut self) -> bool {
+                    self.inner.simplify()
+                }
+
+                fn complicate(&mut self) -> bool {
+                    self.inner.complicate()
+                }
+            }
+
+            impl Strategy for Any {
+                type Tree = BinarySearch;
+                type Value = $name;
+
+                fn new_tree(&self, runner: &mut proptest::test_runner::TestRunner) -> NewTree<Self> {
+                    Ok(BinarySearch { inner: self.strategy.new_tree(runner)? })
+                }
+            }
+
+            pub const ANY: Any = Any { strategy: InnerStrategy::new(proptest::num::u64::ANY) };
+        }
+    }
+}
+
+impl_num! { u128 }
+impl_num! { i128 }

From 0ec3ecfab15fe5baf45cfd452c2317a7f18f69bd Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Sat, 13 Feb 2021 14:19:16 -0500
Subject: [PATCH 092/249] Split ops tests

---
 crates/core_simd/tests/f32_ops.rs             |   4 +
 crates/core_simd/tests/f64_ops.rs             |   4 +
 crates/core_simd/tests/float.rs               | 139 ------------------
 crates/core_simd/tests/i128_ops.rs            |   4 +
 crates/core_simd/tests/i16_ops.rs             |   4 +
 crates/core_simd/tests/i32_ops.rs             |   4 +
 crates/core_simd/tests/i64_ops.rs             |   4 +
 crates/core_simd/tests/i8_ops.rs              |   4 +
 crates/core_simd/tests/isize_ops.rs           |   4 +
 .../tests/{integer.rs => ops_macros.rs}       |  92 ++++++++++--
 crates/core_simd/tests/u128_ops.rs            |   4 +
 crates/core_simd/tests/u16_ops.rs             |   4 +
 crates/core_simd/tests/u32_ops.rs             |   4 +
 crates/core_simd/tests/u64_ops.rs             |   4 +
 crates/core_simd/tests/u8_ops.rs              |   4 +
 crates/core_simd/tests/usize_ops.rs           |   4 +
 16 files changed, 136 insertions(+), 151 deletions(-)
 create mode 100644 crates/core_simd/tests/f32_ops.rs
 create mode 100644 crates/core_simd/tests/f64_ops.rs
 delete mode 100644 crates/core_simd/tests/float.rs
 create mode 100644 crates/core_simd/tests/i128_ops.rs
 create mode 100644 crates/core_simd/tests/i16_ops.rs
 create mode 100644 crates/core_simd/tests/i32_ops.rs
 create mode 100644 crates/core_simd/tests/i64_ops.rs
 create mode 100644 crates/core_simd/tests/i8_ops.rs
 create mode 100644 crates/core_simd/tests/isize_ops.rs
 rename crates/core_simd/tests/{integer.rs => ops_macros.rs} (69%)
 create mode 100644 crates/core_simd/tests/u128_ops.rs
 create mode 100644 crates/core_simd/tests/u16_ops.rs
 create mode 100644 crates/core_simd/tests/u32_ops.rs
 create mode 100644 crates/core_simd/tests/u64_ops.rs
 create mode 100644 crates/core_simd/tests/u8_ops.rs
 create mode 100644 crates/core_simd/tests/usize_ops.rs

diff --git a/crates/core_simd/tests/f32_ops.rs b/crates/core_simd/tests/f32_ops.rs
new file mode 100644
index 000000000000..6e3802aae6a4
--- /dev/null
+++ b/crates/core_simd/tests/f32_ops.rs
@@ -0,0 +1,4 @@
+#[macro_use]
+#[path = "ops_macros.rs"]
+mod macros;
+impl_float_tests! { SimdF32, f32, i32 }
diff --git a/crates/core_simd/tests/f64_ops.rs b/crates/core_simd/tests/f64_ops.rs
new file mode 100644
index 000000000000..da31cc3161bd
--- /dev/null
+++ b/crates/core_simd/tests/f64_ops.rs
@@ -0,0 +1,4 @@
+#[macro_use]
+#[path = "ops_macros.rs"]
+mod macros;
+impl_float_tests! { SimdF64, f64, i64 }
diff --git a/crates/core_simd/tests/float.rs b/crates/core_simd/tests/float.rs
deleted file mode 100644
index 618a75250bda..000000000000
--- a/crates/core_simd/tests/float.rs
+++ /dev/null
@@ -1,139 +0,0 @@
-macro_rules! impl_op_test {
-    { unary, $vector:ty, $scalar:ty, $trait:ident :: $fn:ident } => {
-        test_helpers::test_lanes! {
-            fn $fn<const LANES: usize>() {
-                test_helpers::test_unary_elementwise(
-                    &<$vector as core::ops::$trait>::$fn,
-                    &<$scalar as core::ops::$trait>::$fn,
-                    &|_| true,
-                );
-            }
-        }
-    };
-    { binary, $vector:ty, $scalar:ty, $trait:ident :: $fn:ident, $trait_assign:ident :: $fn_assign:ident } => {
-        mod $fn {
-            use super::*;
-
-            test_helpers::test_lanes! {
-                fn normal<const LANES: usize>() {
-                    test_helpers::test_binary_elementwise(
-                        &<$vector as core::ops::$trait>::$fn,
-                        &<$scalar as core::ops::$trait>::$fn,
-                        &|_, _| true,
-                    );
-                }
-
-                fn scalar_rhs<const LANES: usize>() {
-                    test_helpers::test_binary_scalar_rhs_elementwise(
-                        &<$vector as core::ops::$trait<$scalar>>::$fn,
-                        &<$scalar as core::ops::$trait>::$fn,
-                        &|_, _| true,
-                    );
-                }
-
-                fn scalar_lhs<const LANES: usize>() {
-                    test_helpers::test_binary_scalar_lhs_elementwise(
-                        &<$scalar as core::ops::$trait<$vector>>::$fn,
-                        &<$scalar as core::ops::$trait>::$fn,
-                        &|_, _| true,
-                    );
-                }
-
-                fn assign<const LANES: usize>() {
-                    test_helpers::test_binary_elementwise(
-                        &|mut a, b| { <$vector as core::ops::$trait_assign>::$fn_assign(&mut a, b); a },
-                        &|mut a, b| { <$scalar as core::ops::$trait_assign>::$fn_assign(&mut a, b); a },
-                        &|_, _| true,
-                    )
-                }
-
-                fn assign_scalar_rhs<const LANES: usize>() {
-                    test_helpers::test_binary_scalar_rhs_elementwise(
-                        &|mut a, b| { <$vector as core::ops::$trait_assign<$scalar>>::$fn_assign(&mut a, b); a },
-                        &|mut a, b| { <$scalar as core::ops::$trait_assign>::$fn_assign(&mut a, b); a },
-                        &|_, _| true,
-                    )
-                }
-            }
-        }
-    };
-}
-
-macro_rules! impl_tests {
-    { $vector:ident, $scalar:tt, $int_scalar:tt } => {
-        mod $scalar {
-            type Vector<const LANES: usize> = core_simd::$vector<LANES>;
-            type Scalar = $scalar;
-            type IntScalar = $int_scalar;
-
-            impl_op_test! { unary, Vector<LANES>, Scalar, Neg::neg }
-            impl_op_test! { binary, Vector<LANES>, Scalar, Add::add, AddAssign::add_assign }
-            impl_op_test! { binary, Vector<LANES>, Scalar, Sub::sub, SubAssign::sub_assign }
-            impl_op_test! { binary, Vector<LANES>, Scalar, Mul::mul, SubAssign::sub_assign }
-            impl_op_test! { binary, Vector<LANES>, Scalar, Div::div, DivAssign::div_assign }
-            impl_op_test! { binary, Vector<LANES>, Scalar, Rem::rem, RemAssign::rem_assign }
-
-            test_helpers::test_lanes! {
-                fn abs<const LANES: usize>() {
-                    test_helpers::test_unary_elementwise(
-                        &Vector::<LANES>::abs,
-                        &Scalar::abs,
-                        &|_| true,
-                    )
-                }
-
-                fn ceil<const LANES: usize>() {
-                    test_helpers::test_unary_elementwise(
-                        &Vector::<LANES>::ceil,
-                        &Scalar::ceil,
-                        &|_| true,
-                    )
-                }
-
-                fn floor<const LANES: usize>() {
-                    test_helpers::test_unary_elementwise(
-                        &Vector::<LANES>::floor,
-                        &Scalar::floor,
-                        &|_| true,
-                    )
-                }
-
-                fn round_from_int<const LANES: usize>() {
-                    test_helpers::test_unary_elementwise(
-                        &Vector::<LANES>::round_from_int,
-                        &|x| x as Scalar,
-                        &|_| true,
-                    )
-                }
-
-                fn to_int_unchecked<const LANES: usize>() {
-                    // The maximum integer that can be represented by the equivalently sized float has
-                    // all of the mantissa digits set to 1, pushed up to the MSB.
-                    const ALL_MANTISSA_BITS: IntScalar = ((1 << <Scalar>::MANTISSA_DIGITS) - 1);
-                    const MAX_REPRESENTABLE_VALUE: Scalar =
-                        (ALL_MANTISSA_BITS << (core::mem::size_of::<Scalar>() * 8 - <Scalar>::MANTISSA_DIGITS as usize - 1)) as Scalar;
-
-                    let mut runner = proptest::test_runner::TestRunner::default();
-                    runner.run(
-                        &test_helpers::array::UniformArrayStrategy::new(-MAX_REPRESENTABLE_VALUE..MAX_REPRESENTABLE_VALUE),
-                        |x| {
-                            let result_1 = unsafe { Vector::from_array(x).to_int_unchecked().to_array() };
-                            let result_2 = {
-                                let mut result = [0; LANES];
-                                for (i, o) in x.iter().zip(result.iter_mut()) {
-                                    *o = unsafe { i.to_int_unchecked() };
-                                }
-                                result
-                            };
-                            test_helpers::prop_assert_biteq!(result_1, result_2);
-                            Ok(())
-                        },
-                    ).unwrap();
-                }
-            }
-        }
-    }
-}
-
-impl_tests! { SimdF32, f32, i32 }
-impl_tests! { SimdF64, f64, i64 }
diff --git a/crates/core_simd/tests/i128_ops.rs b/crates/core_simd/tests/i128_ops.rs
new file mode 100644
index 000000000000..874324463cf1
--- /dev/null
+++ b/crates/core_simd/tests/i128_ops.rs
@@ -0,0 +1,4 @@
+#[macro_use]
+#[path = "ops_macros.rs"]
+mod macros;
+impl_signed_tests! { SimdI128, i128 }
diff --git a/crates/core_simd/tests/i16_ops.rs b/crates/core_simd/tests/i16_ops.rs
new file mode 100644
index 000000000000..ebdbf60bce49
--- /dev/null
+++ b/crates/core_simd/tests/i16_ops.rs
@@ -0,0 +1,4 @@
+#[macro_use]
+#[path = "ops_macros.rs"]
+mod macros;
+impl_signed_tests! { SimdI16, i16 }
diff --git a/crates/core_simd/tests/i32_ops.rs b/crates/core_simd/tests/i32_ops.rs
new file mode 100644
index 000000000000..5c2c41cdb18d
--- /dev/null
+++ b/crates/core_simd/tests/i32_ops.rs
@@ -0,0 +1,4 @@
+#[macro_use]
+#[path = "ops_macros.rs"]
+mod macros;
+impl_signed_tests! { SimdI32, i32 }
diff --git a/crates/core_simd/tests/i64_ops.rs b/crates/core_simd/tests/i64_ops.rs
new file mode 100644
index 000000000000..9321755d6719
--- /dev/null
+++ b/crates/core_simd/tests/i64_ops.rs
@@ -0,0 +1,4 @@
+#[macro_use]
+#[path = "ops_macros.rs"]
+mod macros;
+impl_signed_tests! { SimdI64, i64 }
diff --git a/crates/core_simd/tests/i8_ops.rs b/crates/core_simd/tests/i8_ops.rs
new file mode 100644
index 000000000000..bea49c3a6468
--- /dev/null
+++ b/crates/core_simd/tests/i8_ops.rs
@@ -0,0 +1,4 @@
+#[macro_use]
+#[path = "ops_macros.rs"]
+mod macros;
+impl_signed_tests! { SimdI8, i8 }
diff --git a/crates/core_simd/tests/isize_ops.rs b/crates/core_simd/tests/isize_ops.rs
new file mode 100644
index 000000000000..5ec29f232738
--- /dev/null
+++ b/crates/core_simd/tests/isize_ops.rs
@@ -0,0 +1,4 @@
+#[macro_use]
+#[path = "ops_macros.rs"]
+mod macros;
+impl_signed_tests! { SimdIsize, isize }
diff --git a/crates/core_simd/tests/integer.rs b/crates/core_simd/tests/ops_macros.rs
similarity index 69%
rename from crates/core_simd/tests/integer.rs
rename to crates/core_simd/tests/ops_macros.rs
index 336126283563..dc920c649d55 100644
--- a/crates/core_simd/tests/integer.rs
+++ b/crates/core_simd/tests/ops_macros.rs
@@ -1,3 +1,4 @@
+#[macro_export]
 macro_rules! impl_unary_op_test {
     { $vector:ty, $scalar:ty, $trait:ident :: $fn:ident, $scalar_fn:expr } => {
         test_helpers::test_lanes! {
@@ -15,6 +16,7 @@ macro_rules! impl_unary_op_test {
     };
 }
 
+#[macro_export]
 macro_rules! impl_binary_op_test {
     { $vector:ty, $scalar:ty, $trait:ident :: $fn:ident, $trait_assign:ident :: $fn_assign:ident, $scalar_fn:expr } => {
         mod $fn {
@@ -68,6 +70,7 @@ macro_rules! impl_binary_op_test {
     };
 }
 
+#[macro_export]
 macro_rules! impl_binary_checked_op_test {
     { $vector:ty, $scalar:ty, $trait:ident :: $fn:ident, $trait_assign:ident :: $fn_assign:ident, $scalar_fn:expr, $check_fn:expr } => {
         mod $fn {
@@ -121,6 +124,7 @@ macro_rules! impl_binary_checked_op_test {
     };
 }
 
+#[macro_export]
 macro_rules! impl_signed_tests {
     { $vector:ident, $scalar:tt } => {
         mod $scalar {
@@ -151,6 +155,7 @@ macro_rules! impl_signed_tests {
     }
 }
 
+#[macro_export]
 macro_rules! impl_unsigned_tests {
     { $vector:ident, $scalar:tt } => {
         mod $scalar {
@@ -171,16 +176,79 @@ macro_rules! impl_unsigned_tests {
     }
 }
 
-impl_signed_tests! { SimdI8, i8 }
-impl_signed_tests! { SimdI16, i16 }
-impl_signed_tests! { SimdI32, i32 }
-impl_signed_tests! { SimdI64, i64 }
-impl_signed_tests! { SimdI128, i128 }
-impl_signed_tests! { SimdIsize, isize }
+#[macro_export]
+macro_rules! impl_float_tests {
+    { $vector:ident, $scalar:tt, $int_scalar:tt } => {
+        mod $scalar {
+            type Vector<const LANES: usize> = core_simd::$vector<LANES>;
+            type Scalar = $scalar;
+            type IntScalar = $int_scalar;
 
-impl_unsigned_tests! { SimdU8, u8 }
-impl_unsigned_tests! { SimdU16, u16 }
-impl_unsigned_tests! { SimdU32, u32 }
-impl_unsigned_tests! { SimdU64, u64 }
-impl_unsigned_tests! { SimdU128, u128 }
-impl_unsigned_tests! { SimdUsize, usize }
+            impl_unary_op_test!(Vector<LANES>, Scalar, Neg::neg);
+            impl_binary_op_test!(Vector<LANES>, Scalar, Add::add, AddAssign::add_assign);
+            impl_binary_op_test!(Vector<LANES>, Scalar, Sub::sub, SubAssign::sub_assign);
+            impl_binary_op_test!(Vector<LANES>, Scalar, Mul::mul, SubAssign::sub_assign);
+            impl_binary_op_test!(Vector<LANES>, Scalar, Div::div, DivAssign::div_assign);
+            impl_binary_op_test!(Vector<LANES>, Scalar, Rem::rem, RemAssign::rem_assign);
+
+            test_helpers::test_lanes! {
+                fn abs<const LANES: usize>() {
+                    test_helpers::test_unary_elementwise(
+                        &Vector::<LANES>::abs,
+                        &Scalar::abs,
+                        &|_| true,
+                    )
+                }
+
+                fn ceil<const LANES: usize>() {
+                    test_helpers::test_unary_elementwise(
+                        &Vector::<LANES>::ceil,
+                        &Scalar::ceil,
+                        &|_| true,
+                    )
+                }
+
+                fn floor<const LANES: usize>() {
+                    test_helpers::test_unary_elementwise(
+                        &Vector::<LANES>::floor,
+                        &Scalar::floor,
+                        &|_| true,
+                    )
+                }
+
+                fn round_from_int<const LANES: usize>() {
+                    test_helpers::test_unary_elementwise(
+                        &Vector::<LANES>::round_from_int,
+                        &|x| x as Scalar,
+                        &|_| true,
+                    )
+                }
+
+                fn to_int_unchecked<const LANES: usize>() {
+                    // The maximum integer that can be represented by the equivalently sized float has
+                    // all of the mantissa digits set to 1, pushed up to the MSB.
+                    const ALL_MANTISSA_BITS: IntScalar = ((1 << <Scalar>::MANTISSA_DIGITS) - 1);
+                    const MAX_REPRESENTABLE_VALUE: Scalar =
+                        (ALL_MANTISSA_BITS << (core::mem::size_of::<Scalar>() * 8 - <Scalar>::MANTISSA_DIGITS as usize - 1)) as Scalar;
+
+                    let mut runner = proptest::test_runner::TestRunner::default();
+                    runner.run(
+                        &test_helpers::array::UniformArrayStrategy::new(-MAX_REPRESENTABLE_VALUE..MAX_REPRESENTABLE_VALUE),
+                        |x| {
+                            let result_1 = unsafe { Vector::from_array(x).to_int_unchecked().to_array() };
+                            let result_2 = {
+                                let mut result = [0; LANES];
+                                for (i, o) in x.iter().zip(result.iter_mut()) {
+                                    *o = unsafe { i.to_int_unchecked() };
+                                }
+                                result
+                            };
+                            test_helpers::prop_assert_biteq!(result_1, result_2);
+                            Ok(())
+                        },
+                    ).unwrap();
+                }
+            }
+        }
+    }
+}
diff --git a/crates/core_simd/tests/u128_ops.rs b/crates/core_simd/tests/u128_ops.rs
new file mode 100644
index 000000000000..eea7e3297c66
--- /dev/null
+++ b/crates/core_simd/tests/u128_ops.rs
@@ -0,0 +1,4 @@
+#[macro_use]
+#[path = "ops_macros.rs"]
+mod macros;
+impl_unsigned_tests! { SimdU128, u128 }
diff --git a/crates/core_simd/tests/u16_ops.rs b/crates/core_simd/tests/u16_ops.rs
new file mode 100644
index 000000000000..ce9951a87c0a
--- /dev/null
+++ b/crates/core_simd/tests/u16_ops.rs
@@ -0,0 +1,4 @@
+#[macro_use]
+#[path = "ops_macros.rs"]
+mod macros;
+impl_unsigned_tests! { SimdU16, u16 }
diff --git a/crates/core_simd/tests/u32_ops.rs b/crates/core_simd/tests/u32_ops.rs
new file mode 100644
index 000000000000..87bedbd43b7c
--- /dev/null
+++ b/crates/core_simd/tests/u32_ops.rs
@@ -0,0 +1,4 @@
+#[macro_use]
+#[path = "ops_macros.rs"]
+mod macros;
+impl_unsigned_tests! { SimdU32, u32 }
diff --git a/crates/core_simd/tests/u64_ops.rs b/crates/core_simd/tests/u64_ops.rs
new file mode 100644
index 000000000000..ec76891da66e
--- /dev/null
+++ b/crates/core_simd/tests/u64_ops.rs
@@ -0,0 +1,4 @@
+#[macro_use]
+#[path = "ops_macros.rs"]
+mod macros;
+impl_unsigned_tests! { SimdU64, u64 }
diff --git a/crates/core_simd/tests/u8_ops.rs b/crates/core_simd/tests/u8_ops.rs
new file mode 100644
index 000000000000..00a63d846133
--- /dev/null
+++ b/crates/core_simd/tests/u8_ops.rs
@@ -0,0 +1,4 @@
+#[macro_use]
+#[path = "ops_macros.rs"]
+mod macros;
+impl_unsigned_tests! { SimdU8, u8 }
diff --git a/crates/core_simd/tests/usize_ops.rs b/crates/core_simd/tests/usize_ops.rs
new file mode 100644
index 000000000000..dd49c656cbe8
--- /dev/null
+++ b/crates/core_simd/tests/usize_ops.rs
@@ -0,0 +1,4 @@
+#[macro_use]
+#[path = "ops_macros.rs"]
+mod macros;
+impl_unsigned_tests! { SimdUsize, usize }

From 714ad639b39f7a35422b3fc342eb843013cc89ba Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Sat, 13 Feb 2021 15:42:04 -0500
Subject: [PATCH 093/249] Fix MulAssign typo in tests, move panic tests

---
 crates/core_simd/tests/f32_ops.rs             |   3 +-
 crates/core_simd/tests/f64_ops.rs             |   3 +-
 crates/core_simd/tests/i128_ops.rs            |   3 +-
 crates/core_simd/tests/i16_ops.rs             |   3 +-
 crates/core_simd/tests/i32_ops.rs             |   3 +-
 crates/core_simd/tests/i64_ops.rs             |   3 +-
 crates/core_simd/tests/i8_ops.rs              |   3 +-
 crates/core_simd/tests/isize_ops.rs           |   3 +-
 crates/core_simd/tests/mask_ops.rs            |   1 +
 .../{ops_impl => mask_ops_impl}/mask128.rs    |   0
 .../{ops_impl => mask_ops_impl}/mask16.rs     |   0
 .../{ops_impl => mask_ops_impl}/mask32.rs     |   0
 .../{ops_impl => mask_ops_impl}/mask64.rs     |   0
 .../{ops_impl => mask_ops_impl}/mask8.rs      |   0
 .../mask_macros.rs                            |   0
 .../{ops_impl => mask_ops_impl}/masksize.rs   |   0
 .../tests/{ops_impl => mask_ops_impl}/mod.rs  |   0
 crates/core_simd/tests/ops.rs                 |   1 -
 crates/core_simd/tests/ops_impl/int_macros.rs | 101 -----
 crates/core_simd/tests/ops_impl/u128.rs       |   4 -
 crates/core_simd/tests/ops_impl/u16.rs        |   6 -
 crates/core_simd/tests/ops_impl/u32.rs        |   6 -
 crates/core_simd/tests/ops_impl/u64.rs        |   5 -
 crates/core_simd/tests/ops_impl/u8.rs         |   6 -
 .../core_simd/tests/ops_impl/uint_macros.rs   | 428 ------------------
 crates/core_simd/tests/ops_impl/usize.rs      |   5 -
 crates/core_simd/tests/ops_macros.rs          |  57 ++-
 crates/core_simd/tests/u128_ops.rs            |   3 +-
 crates/core_simd/tests/u16_ops.rs             |   3 +-
 crates/core_simd/tests/u32_ops.rs             |   3 +-
 crates/core_simd/tests/u64_ops.rs             |   3 +-
 crates/core_simd/tests/u8_ops.rs              |   3 +-
 crates/core_simd/tests/usize_ops.rs           |   3 +-
 crates/test_helpers/src/lib.rs                |  59 ++-
 34 files changed, 129 insertions(+), 592 deletions(-)
 create mode 100644 crates/core_simd/tests/mask_ops.rs
 rename crates/core_simd/tests/{ops_impl => mask_ops_impl}/mask128.rs (100%)
 rename crates/core_simd/tests/{ops_impl => mask_ops_impl}/mask16.rs (100%)
 rename crates/core_simd/tests/{ops_impl => mask_ops_impl}/mask32.rs (100%)
 rename crates/core_simd/tests/{ops_impl => mask_ops_impl}/mask64.rs (100%)
 rename crates/core_simd/tests/{ops_impl => mask_ops_impl}/mask8.rs (100%)
 rename crates/core_simd/tests/{ops_impl => mask_ops_impl}/mask_macros.rs (100%)
 rename crates/core_simd/tests/{ops_impl => mask_ops_impl}/masksize.rs (100%)
 rename crates/core_simd/tests/{ops_impl => mask_ops_impl}/mod.rs (100%)
 delete mode 100644 crates/core_simd/tests/ops.rs
 delete mode 100644 crates/core_simd/tests/ops_impl/int_macros.rs
 delete mode 100644 crates/core_simd/tests/ops_impl/u128.rs
 delete mode 100644 crates/core_simd/tests/ops_impl/u16.rs
 delete mode 100644 crates/core_simd/tests/ops_impl/u32.rs
 delete mode 100644 crates/core_simd/tests/ops_impl/u64.rs
 delete mode 100644 crates/core_simd/tests/ops_impl/u8.rs
 delete mode 100644 crates/core_simd/tests/ops_impl/uint_macros.rs
 delete mode 100644 crates/core_simd/tests/ops_impl/usize.rs

diff --git a/crates/core_simd/tests/f32_ops.rs b/crates/core_simd/tests/f32_ops.rs
index 6e3802aae6a4..ac5499b7ffeb 100644
--- a/crates/core_simd/tests/f32_ops.rs
+++ b/crates/core_simd/tests/f32_ops.rs
@@ -1,4 +1,3 @@
 #[macro_use]
-#[path = "ops_macros.rs"]
-mod macros;
+mod ops_macros;
 impl_float_tests! { SimdF32, f32, i32 }
diff --git a/crates/core_simd/tests/f64_ops.rs b/crates/core_simd/tests/f64_ops.rs
index da31cc3161bd..dcdb2aa31522 100644
--- a/crates/core_simd/tests/f64_ops.rs
+++ b/crates/core_simd/tests/f64_ops.rs
@@ -1,4 +1,3 @@
 #[macro_use]
-#[path = "ops_macros.rs"]
-mod macros;
+mod ops_macros;
 impl_float_tests! { SimdF64, f64, i64 }
diff --git a/crates/core_simd/tests/i128_ops.rs b/crates/core_simd/tests/i128_ops.rs
index 874324463cf1..3e3fa1d20682 100644
--- a/crates/core_simd/tests/i128_ops.rs
+++ b/crates/core_simd/tests/i128_ops.rs
@@ -1,4 +1,3 @@
 #[macro_use]
-#[path = "ops_macros.rs"]
-mod macros;
+mod ops_macros;
 impl_signed_tests! { SimdI128, i128 }
diff --git a/crates/core_simd/tests/i16_ops.rs b/crates/core_simd/tests/i16_ops.rs
index ebdbf60bce49..4d2a7b053b5c 100644
--- a/crates/core_simd/tests/i16_ops.rs
+++ b/crates/core_simd/tests/i16_ops.rs
@@ -1,4 +1,3 @@
 #[macro_use]
-#[path = "ops_macros.rs"]
-mod macros;
+mod ops_macros;
 impl_signed_tests! { SimdI16, i16 }
diff --git a/crates/core_simd/tests/i32_ops.rs b/crates/core_simd/tests/i32_ops.rs
index 5c2c41cdb18d..90079d727e4b 100644
--- a/crates/core_simd/tests/i32_ops.rs
+++ b/crates/core_simd/tests/i32_ops.rs
@@ -1,4 +1,3 @@
 #[macro_use]
-#[path = "ops_macros.rs"]
-mod macros;
+mod ops_macros;
 impl_signed_tests! { SimdI32, i32 }
diff --git a/crates/core_simd/tests/i64_ops.rs b/crates/core_simd/tests/i64_ops.rs
index 9321755d6719..ebc3e194974b 100644
--- a/crates/core_simd/tests/i64_ops.rs
+++ b/crates/core_simd/tests/i64_ops.rs
@@ -1,4 +1,3 @@
 #[macro_use]
-#[path = "ops_macros.rs"]
-mod macros;
+mod ops_macros;
 impl_signed_tests! { SimdI64, i64 }
diff --git a/crates/core_simd/tests/i8_ops.rs b/crates/core_simd/tests/i8_ops.rs
index bea49c3a6468..082422b86d29 100644
--- a/crates/core_simd/tests/i8_ops.rs
+++ b/crates/core_simd/tests/i8_ops.rs
@@ -1,4 +1,3 @@
 #[macro_use]
-#[path = "ops_macros.rs"]
-mod macros;
+mod ops_macros;
 impl_signed_tests! { SimdI8, i8 }
diff --git a/crates/core_simd/tests/isize_ops.rs b/crates/core_simd/tests/isize_ops.rs
index 5ec29f232738..1509d701c29a 100644
--- a/crates/core_simd/tests/isize_ops.rs
+++ b/crates/core_simd/tests/isize_ops.rs
@@ -1,4 +1,3 @@
 #[macro_use]
-#[path = "ops_macros.rs"]
-mod macros;
+mod ops_macros;
 impl_signed_tests! { SimdIsize, isize }
diff --git a/crates/core_simd/tests/mask_ops.rs b/crates/core_simd/tests/mask_ops.rs
new file mode 100644
index 000000000000..96330550b40b
--- /dev/null
+++ b/crates/core_simd/tests/mask_ops.rs
@@ -0,0 +1 @@
+mod mask_ops_impl;
diff --git a/crates/core_simd/tests/ops_impl/mask128.rs b/crates/core_simd/tests/mask_ops_impl/mask128.rs
similarity index 100%
rename from crates/core_simd/tests/ops_impl/mask128.rs
rename to crates/core_simd/tests/mask_ops_impl/mask128.rs
diff --git a/crates/core_simd/tests/ops_impl/mask16.rs b/crates/core_simd/tests/mask_ops_impl/mask16.rs
similarity index 100%
rename from crates/core_simd/tests/ops_impl/mask16.rs
rename to crates/core_simd/tests/mask_ops_impl/mask16.rs
diff --git a/crates/core_simd/tests/ops_impl/mask32.rs b/crates/core_simd/tests/mask_ops_impl/mask32.rs
similarity index 100%
rename from crates/core_simd/tests/ops_impl/mask32.rs
rename to crates/core_simd/tests/mask_ops_impl/mask32.rs
diff --git a/crates/core_simd/tests/ops_impl/mask64.rs b/crates/core_simd/tests/mask_ops_impl/mask64.rs
similarity index 100%
rename from crates/core_simd/tests/ops_impl/mask64.rs
rename to crates/core_simd/tests/mask_ops_impl/mask64.rs
diff --git a/crates/core_simd/tests/ops_impl/mask8.rs b/crates/core_simd/tests/mask_ops_impl/mask8.rs
similarity index 100%
rename from crates/core_simd/tests/ops_impl/mask8.rs
rename to crates/core_simd/tests/mask_ops_impl/mask8.rs
diff --git a/crates/core_simd/tests/ops_impl/mask_macros.rs b/crates/core_simd/tests/mask_ops_impl/mask_macros.rs
similarity index 100%
rename from crates/core_simd/tests/ops_impl/mask_macros.rs
rename to crates/core_simd/tests/mask_ops_impl/mask_macros.rs
diff --git a/crates/core_simd/tests/ops_impl/masksize.rs b/crates/core_simd/tests/mask_ops_impl/masksize.rs
similarity index 100%
rename from crates/core_simd/tests/ops_impl/masksize.rs
rename to crates/core_simd/tests/mask_ops_impl/masksize.rs
diff --git a/crates/core_simd/tests/ops_impl/mod.rs b/crates/core_simd/tests/mask_ops_impl/mod.rs
similarity index 100%
rename from crates/core_simd/tests/ops_impl/mod.rs
rename to crates/core_simd/tests/mask_ops_impl/mod.rs
diff --git a/crates/core_simd/tests/ops.rs b/crates/core_simd/tests/ops.rs
deleted file mode 100644
index 60aff06a76a6..000000000000
--- a/crates/core_simd/tests/ops.rs
+++ /dev/null
@@ -1 +0,0 @@
-mod ops_impl;
diff --git a/crates/core_simd/tests/ops_impl/int_macros.rs b/crates/core_simd/tests/ops_impl/int_macros.rs
deleted file mode 100644
index f565ae2f04df..000000000000
--- a/crates/core_simd/tests/ops_impl/int_macros.rs
+++ /dev/null
@@ -1,101 +0,0 @@
-macro_rules! int_tests {
-    { $vector:ident, $scalar:ident } => {
-        #[cfg(test)]
-        mod $vector {
-            use super::*;
-            use helpers::lanewise::*;
-
-            #[cfg(target_arch = "wasm32")]
-            use wasm_bindgen_test::*;
-
-            #[cfg(target_arch = "wasm32")]
-            wasm_bindgen_test_configure!(run_in_browser);
-
-            // TODO impl this as an associated fn on vectors
-            fn from_slice(slice: &[$scalar]) -> core_simd::$vector {
-                let mut value = core_simd::$vector::default();
-                let value_slice: &mut [_] = value.as_mut();
-                value_slice.copy_from_slice(&slice[0..value_slice.len()]);
-                value
-            }
-
-            const A: [$scalar; 64] = [
-                7, 7, 7, 7, -7, -7, -7, -7,
-                6, 6, 6, 6, -6, -6, -6, -6,
-                5, 5, 5, 5, -5, -5, -5, -5,
-                4, 4, 4, 4, -4, -4, -4, -4,
-                3, 3, 3, 3, -3, -3, -3, -3,
-                2, 2, 2, 2, -2, -2, -2, -2,
-                1, 1, 1, 1, -1, -1, -1, -1,
-                0, 0, 0, 0,  0,  0,  0,  0,
-            ];
-            const B: [$scalar; 64] = [
-                 1,  2,  3,  4,  5,  6,  7,  8,
-                 1,  2,  3,  4,  5,  6,  7,  8,
-                 1,  2,  3,  4,  5,  6,  7,  8,
-                 1,  2,  3,  4,  5,  6,  7,  8,
-                 -1,  -2,  -3,  -4,  -5,  -6,  -7,  -8,
-                 -1,  -2,  -3,  -4,  -5,  -6,  -7,  -8,
-                 -1,  -2,  -3,  -4,  -5,  -6,  -7,  -8,
-                 -1,  -2,  -3,  -4,  -5,  -6,  -7,  -8,
-            ];
-
-            #[test]
-            #[should_panic]
-            fn div_min_panics() {
-                let a = from_slice(&vec![$scalar::MIN; 64]);
-                let b = from_slice(&vec![-1; 64]);
-                let _ = a / b;
-            }
-
-            #[test]
-            #[should_panic]
-            fn div_by_all_zeros_panics() {
-                let a = from_slice(&A);
-                let b = from_slice(&vec![0 ; 64]);
-                let _ = a / b;
-            }
-
-            #[test]
-            #[should_panic]
-            fn div_by_one_zero_panics() {
-                let a = from_slice(&A);
-                let mut b = from_slice(&B);
-                b[0] = 0 as _;
-                let _ = a / b;
-            }
-
-            #[test]
-            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
-            fn div_min_neg_one_no_panic() {
-                let a = from_slice(&A);
-                let b = from_slice(&vec![-1; 64]);
-                let _ = a / b;
-            }
-
-            #[test]
-            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
-            fn rem_min_neg_one_no_panic() {
-                let a = from_slice(&A);
-                let b = from_slice(&vec![-1; 64]);
-                let _ = a % b;
-            }
-
-            #[test]
-            #[should_panic]
-            fn rem_min_panic() {
-                let a = from_slice(&vec![$scalar::MIN; 64]);
-                let b = from_slice(&vec![-1 ; 64]);
-                let _ = a % b;
-            }
-
-            #[test]
-            #[should_panic]
-            fn rem_min_zero_panic() {
-                let a = from_slice(&A);
-                let b = from_slice(&vec![0 ; 64]);
-                let _ = a % b;
-            }
-        }
-    }
-}
diff --git a/crates/core_simd/tests/ops_impl/u128.rs b/crates/core_simd/tests/ops_impl/u128.rs
deleted file mode 100644
index cfd849640ffe..000000000000
--- a/crates/core_simd/tests/ops_impl/u128.rs
+++ /dev/null
@@ -1,4 +0,0 @@
-use super::helpers;
-
-uint_tests! { u128x2, u128 }
-uint_tests! { u128x4, u128 }
diff --git a/crates/core_simd/tests/ops_impl/u16.rs b/crates/core_simd/tests/ops_impl/u16.rs
deleted file mode 100644
index 50af4dd48b38..000000000000
--- a/crates/core_simd/tests/ops_impl/u16.rs
+++ /dev/null
@@ -1,6 +0,0 @@
-use super::helpers;
-
-uint_tests! { u16x4, u16 }
-uint_tests! { u16x8, u16 }
-uint_tests! { u16x16, u16 }
-uint_tests! { u16x32, u16 }
diff --git a/crates/core_simd/tests/ops_impl/u32.rs b/crates/core_simd/tests/ops_impl/u32.rs
deleted file mode 100644
index 8e7faa9d7403..000000000000
--- a/crates/core_simd/tests/ops_impl/u32.rs
+++ /dev/null
@@ -1,6 +0,0 @@
-use super::helpers;
-
-uint_tests! { u32x2, u32 }
-uint_tests! { u32x4, u32 }
-uint_tests! { u32x8, u32 }
-uint_tests! { u32x16, u32 }
diff --git a/crates/core_simd/tests/ops_impl/u64.rs b/crates/core_simd/tests/ops_impl/u64.rs
deleted file mode 100644
index 1a6385d37bae..000000000000
--- a/crates/core_simd/tests/ops_impl/u64.rs
+++ /dev/null
@@ -1,5 +0,0 @@
-use super::helpers;
-
-uint_tests! { u64x2, u64 }
-uint_tests! { u64x4, u64 }
-uint_tests! { u64x8, u64 }
diff --git a/crates/core_simd/tests/ops_impl/u8.rs b/crates/core_simd/tests/ops_impl/u8.rs
deleted file mode 100644
index 31568b1eacbe..000000000000
--- a/crates/core_simd/tests/ops_impl/u8.rs
+++ /dev/null
@@ -1,6 +0,0 @@
-use super::helpers;
-
-uint_tests! { u8x8, u8 }
-uint_tests! { u8x16, u8 }
-uint_tests! { u8x32, u8 }
-uint_tests! { u8x64, u8 }
diff --git a/crates/core_simd/tests/ops_impl/uint_macros.rs b/crates/core_simd/tests/ops_impl/uint_macros.rs
deleted file mode 100644
index bc8b3be74860..000000000000
--- a/crates/core_simd/tests/ops_impl/uint_macros.rs
+++ /dev/null
@@ -1,428 +0,0 @@
-macro_rules! uint_tests {
-    { $vector:ident, $scalar:ident } => {
-        #[cfg(test)]
-        mod $vector {
-            use super::*;
-            use helpers::lanewise::*;
-
-            #[cfg(target_arch = "wasm32")]
-            use wasm_bindgen_test::*;
-
-            #[cfg(target_arch = "wasm32")]
-            wasm_bindgen_test_configure!(run_in_browser);
-
-            // TODO impl this as an associated fn on vectors
-            fn from_slice(slice: &[$scalar]) -> core_simd::$vector {
-                let mut value = core_simd::$vector::default();
-                let value_slice: &mut [_] = value.as_mut();
-                value_slice.copy_from_slice(&slice[0..value_slice.len()]);
-                value
-            }
-
-            const A: [$scalar; 64] = [
-                16, 16, 16, 16, 16, 16, 16, 16,
-                14, 14, 14, 14, 14, 14, 14, 14,
-                12, 12, 12, 12, 12, 12, 12, 12,
-                10, 10, 10, 10, 10, 10, 10, 10,
-                 8,  8,  8,  8,  8,  8,  8,  8,
-                 6,  6,  6,  6,  6,  6,  7,  8,
-                 4,  4,  4,  4,  5,  6,  7,  8,
-                 1,  2,  3,  4,  5,  6,  7,  8,
-            ];
-            const B: [$scalar; 64] = [
-                1, 2, 3, 4, 1, 2, 3, 4,
-                1, 2, 3, 4, 5, 6, 7, 8,
-                1, 2, 3, 4, 5, 6, 7, 8,
-                1, 2, 3, 4, 5, 6, 7, 8,
-                1, 2, 3, 4, 5, 6, 7, 8,
-                1, 2, 3, 4, 5, 6, 7, 8,
-                1, 2, 3, 4, 5, 6, 7, 8,
-                1, 2, 3, 4, 5, 6, 7, 8,
-            ];
-
-            #[test]
-            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
-            fn add() {
-                let a = from_slice(&A);
-                let b = from_slice(&B);
-                let expected = apply_binary_lanewise(a, b, core::ops::Add::add);
-                assert_biteq!(a + b, expected);
-            }
-
-            #[test]
-            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
-            fn add_assign() {
-                let mut a = from_slice(&A);
-                let b = from_slice(&B);
-                let expected = apply_binary_lanewise(a, b, core::ops::Add::add);
-                a += b;
-                assert_biteq!(a, expected);
-            }
-
-            #[test]
-            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
-            fn add_scalar_rhs() {
-                let a = from_slice(&A);
-                let b = 5;
-                let expected = apply_binary_scalar_rhs_lanewise(a, b, core::ops::Add::add);
-                assert_biteq!(a + b, expected);
-            }
-
-            #[test]
-            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
-            fn add_scalar_lhs() {
-                let a = 5;
-                let b = from_slice(&B);
-                let expected = apply_binary_scalar_lhs_lanewise(a, b, core::ops::Add::add);
-                assert_biteq!(a + b, expected);
-            }
-
-            #[test]
-            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
-            fn add_assign_scalar() {
-                let mut a = from_slice(&A);
-                let b = 5;
-                let expected = apply_binary_scalar_rhs_lanewise(a, b, core::ops::Add::add);
-                a += b;
-                assert_biteq!(a, expected);
-            }
-
-            #[test]
-            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
-            fn sub() {
-                let a = from_slice(&A);
-                let b = from_slice(&B);
-                let expected = apply_binary_lanewise(a, b, core::ops::Sub::sub);
-                assert_biteq!(a - b, expected);
-            }
-
-            #[test]
-            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
-            fn sub_assign() {
-                let mut a = from_slice(&A);
-                let b = from_slice(&B);
-                let expected = apply_binary_lanewise(a, b, core::ops::Sub::sub);
-                a -= b;
-                assert_biteq!(a, expected);
-            }
-
-            #[test]
-            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
-            fn sub_scalar_rhs() {
-                let a = from_slice(&A);
-                let b = 1;
-                let expected = apply_binary_scalar_rhs_lanewise(a, b, core::ops::Sub::sub);
-                assert_biteq!(a - b, expected);
-            }
-
-            #[test]
-            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
-            fn sub_scalar_lhs() {
-                let a = 40;
-                let b = from_slice(&B);
-                let expected = apply_binary_scalar_lhs_lanewise(a, b, core::ops::Sub::sub);
-                assert_biteq!(a - b, expected);
-            }
-
-            #[test]
-            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
-            fn sub_assign_scalar() {
-                let mut a = from_slice(&A);
-                let b = 1;
-                let expected = apply_binary_scalar_rhs_lanewise(a, b, core::ops::Sub::sub);
-                a -= b;
-                assert_biteq!(a, expected);
-            }
-
-            #[test]
-            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
-            fn mul() {
-                let a = from_slice(&A);
-                let b = from_slice(&B);
-                let expected = apply_binary_lanewise(a, b, core::ops::Mul::mul);
-                assert_biteq!(a * b, expected);
-            }
-
-            #[test]
-            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
-            fn mul_assign() {
-                let mut a = from_slice(&A);
-                let b = from_slice(&B);
-                let expected = apply_binary_lanewise(a, b, core::ops::Mul::mul);
-                a *= b;
-                assert_biteq!(a, expected);
-            }
-
-            #[test]
-            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
-            fn mul_scalar_rhs() {
-                let a = from_slice(&A);
-                let b = 5;
-                let expected = apply_binary_scalar_rhs_lanewise(a, b, core::ops::Mul::mul);
-                assert_biteq!(a * b, expected);
-            }
-
-            #[test]
-            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
-            fn mul_scalar_lhs() {
-                let a = 5;
-                let b = from_slice(&B);
-                let expected = apply_binary_scalar_lhs_lanewise(a, b, core::ops::Mul::mul);
-                assert_biteq!(a * b, expected);
-            }
-
-            #[test]
-            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
-            fn mul_assign_scalar() {
-                let mut a = from_slice(&A);
-                let b = 5;
-                let expected = apply_binary_scalar_rhs_lanewise(a, b, core::ops::Mul::mul);
-                a *= b;
-                assert_biteq!(a, expected);
-            }
-
-            #[test]
-            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
-            fn div() {
-                let a = from_slice(&A);
-                let b = from_slice(&B);
-                let expected = apply_binary_lanewise(a, b, core::ops::Div::div);
-                assert_biteq!(a / b, expected);
-            }
-
-            #[test]
-            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
-            fn div_assign() {
-                let mut a = from_slice(&A);
-                let b = from_slice(&B);
-                let expected = apply_binary_lanewise(a, b, core::ops::Div::div);
-                a /= b;
-                assert_biteq!(a, expected);
-            }
-
-            #[test]
-            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
-            fn div_scalar_rhs() {
-                let a = from_slice(&A);
-                let b = 5;
-                let expected = apply_binary_scalar_rhs_lanewise(a, b, core::ops::Div::div);
-                assert_biteq!(a / b, expected);
-            }
-
-            #[test]
-            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
-            fn div_scalar_lhs() {
-                let a = 5;
-                let b = from_slice(&B);
-                let expected = apply_binary_scalar_lhs_lanewise(a, b, core::ops::Div::div);
-                assert_biteq!(a / b, expected);
-            }
-
-            #[test]
-            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
-            fn div_assign_scalar() {
-                let mut a = from_slice(&A);
-                let b = 5;
-                let expected = apply_binary_scalar_rhs_lanewise(a, b, core::ops::Div::div);
-                a /= b;
-                assert_biteq!(a, expected);
-            }
-
-            #[test]
-            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
-            fn rem() {
-                let a = from_slice(&A);
-                let b = from_slice(&B);
-                let expected = apply_binary_lanewise(a, b, core::ops::Rem::rem);
-                assert_biteq!(a % b, expected);
-            }
-
-            #[test]
-            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
-            fn rem_assign() {
-                let mut a = from_slice(&A);
-                let b = from_slice(&B);
-                let expected = apply_binary_lanewise(a, b, core::ops::Rem::rem);
-                a %= b;
-                assert_biteq!(a, expected);
-            }
-
-            #[test]
-            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
-            fn rem_scalar_rhs() {
-                let a = from_slice(&A);
-                let b = 5;
-                let expected = apply_binary_scalar_rhs_lanewise(a, b, core::ops::Rem::rem);
-                assert_biteq!(a % b, expected);
-            }
-
-            #[test]
-            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
-            fn rem_scalar_lhs() {
-                let a = 5;
-                let b = from_slice(&B);
-                let expected = apply_binary_scalar_lhs_lanewise(a, b, core::ops::Rem::rem);
-                assert_biteq!(a % b, expected);
-            }
-
-            #[test]
-            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
-            fn rem_assign_scalar() {
-                let mut a = from_slice(&A);
-                let b = 5;
-                let expected = apply_binary_scalar_rhs_lanewise(a, b, core::ops::Rem::rem);
-                a %= b;
-                assert_biteq!(a, expected);
-            }
-
-            #[test]
-            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
-            fn bitand() {
-                let a = from_slice(&A);
-                let b = from_slice(&B);
-                let expected = apply_binary_lanewise(a, b, core::ops::BitAnd::bitand);
-                assert_biteq!(a & b, expected);
-            }
-
-            #[test]
-            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
-            fn bitand_assign() {
-                let mut a = from_slice(&A);
-                let b = from_slice(&B);
-                let expected = apply_binary_lanewise(a, b, core::ops::BitAnd::bitand);
-                a &= b;
-                assert_biteq!(a, expected);
-            }
-
-            #[test]
-            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
-            fn bitand_scalar_rhs() {
-                let a = from_slice(&A);
-                let b = 5;
-                let expected = apply_binary_scalar_rhs_lanewise(a, b, core::ops::BitAnd::bitand);
-                assert_biteq!(a & b, expected);
-            }
-
-            #[test]
-            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
-            fn bitand_scalar_lhs() {
-                let a = 5;
-                let b = from_slice(&B);
-                let expected = apply_binary_scalar_lhs_lanewise(a, b, core::ops::BitAnd::bitand);
-                assert_biteq!(a & b, expected);
-            }
-
-            #[test]
-            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
-            fn bitand_assign_scalar() {
-                let mut a = from_slice(&A);
-                let b = 5;
-                let expected = apply_binary_scalar_rhs_lanewise(a, b, core::ops::BitAnd::bitand);
-                a &= b;
-                assert_biteq!(a, expected);
-            }
-
-            #[test]
-            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
-            fn bitor() {
-                let a = from_slice(&A);
-                let b = from_slice(&B);
-                let expected = apply_binary_lanewise(a, b, core::ops::BitOr::bitor);
-                assert_biteq!(a | b, expected);
-            }
-
-            #[test]
-            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
-            fn bitor_assign() {
-                let mut a = from_slice(&A);
-                let b = from_slice(&B);
-                let expected = apply_binary_lanewise(a, b, core::ops::BitOr::bitor);
-                a |= b;
-                assert_biteq!(a, expected);
-            }
-
-            #[test]
-            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
-            fn bitor_scalar_rhs() {
-                let a = from_slice(&A);
-                let b = 5;
-                let expected = apply_binary_scalar_rhs_lanewise(a, b, core::ops::BitOr::bitor);
-                assert_biteq!(a | b, expected);
-            }
-
-            #[test]
-            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
-            fn bitor_scalar_lhs() {
-                let a = 5;
-                let b = from_slice(&B);
-                let expected = apply_binary_scalar_lhs_lanewise(a, b, core::ops::BitOr::bitor);
-                assert_biteq!(a | b, expected);
-            }
-
-            #[test]
-            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
-            fn bitor_assign_scalar() {
-                let mut a = from_slice(&A);
-                let b = 5;
-                let expected = apply_binary_scalar_rhs_lanewise(a, b, core::ops::BitOr::bitor);
-                a |= b;
-                assert_biteq!(a, expected);
-            }
-
-            #[test]
-            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
-            fn bitxor() {
-                let a = from_slice(&A);
-                let b = from_slice(&B);
-                let expected = apply_binary_lanewise(a, b, core::ops::BitXor::bitxor);
-                assert_biteq!(a ^ b, expected);
-            }
-
-            #[test]
-            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
-            fn bitxor_assign() {
-                let mut a = from_slice(&A);
-                let b = from_slice(&B);
-                let expected = apply_binary_lanewise(a, b, core::ops::BitXor::bitxor);
-                a ^= b;
-                assert_biteq!(a, expected);
-            }
-
-            #[test]
-            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
-            fn bitxor_scalar_rhs() {
-                let a = from_slice(&A);
-                let b = 5;
-                let expected = apply_binary_scalar_rhs_lanewise(a, b, core::ops::BitXor::bitxor);
-                assert_biteq!(a ^ b, expected);
-            }
-
-            #[test]
-            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
-            fn bitxor_scalar_lhs() {
-                let a = 5;
-                let b = from_slice(&B);
-                let expected = apply_binary_scalar_lhs_lanewise(a, b, core::ops::BitXor::bitxor);
-                assert_biteq!(a ^ b, expected);
-            }
-
-            #[test]
-            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
-            fn bitxor_assign_scalar() {
-                let mut a = from_slice(&A);
-                let b = 5;
-                let expected = apply_binary_scalar_rhs_lanewise(a, b, core::ops::BitXor::bitxor);
-                a ^= b;
-                assert_biteq!(a, expected);
-            }
-
-            #[test]
-            #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
-            fn not() {
-                let v = from_slice(&A);
-                let expected = apply_unary_lanewise(v, core::ops::Not::not);
-                assert_biteq!(!v, expected);
-            }
-        }
-    }
-}
diff --git a/crates/core_simd/tests/ops_impl/usize.rs b/crates/core_simd/tests/ops_impl/usize.rs
deleted file mode 100644
index 13da57f15869..000000000000
--- a/crates/core_simd/tests/ops_impl/usize.rs
+++ /dev/null
@@ -1,5 +0,0 @@
-use super::helpers;
-
-uint_tests! { usizex2, usize }
-uint_tests! { usizex4, usize }
-uint_tests! { usizex8, usize }
diff --git a/crates/core_simd/tests/ops_macros.rs b/crates/core_simd/tests/ops_macros.rs
index dc920c649d55..a4b26a1777af 100644
--- a/crates/core_simd/tests/ops_macros.rs
+++ b/crates/core_simd/tests/ops_macros.rs
@@ -141,6 +141,53 @@ macro_rules! impl_signed_tests {
                 }
             }
 
+            test_helpers::test_lanes_panic! {
+                fn div_min_overflow_panics<const LANES: usize>() {
+                    let a = Vector::<LANES>::splat(Scalar::MIN);
+                    let b = Vector::<LANES>::splat(-1);
+                    let _ = a / b;
+                }
+
+                fn div_by_all_zeros_panics<const LANES: usize>() {
+                    let a = Vector::<LANES>::splat(42);
+                    let b = Vector::<LANES>::splat(0);
+                    let _ = a / b;
+                }
+
+                fn div_by_one_zero_panics<const LANES: usize>() {
+                    let a = Vector::<LANES>::splat(42);
+                    let mut b = Vector::<LANES>::splat(21);
+                    b[0] = 0 as _;
+                    let _ = a / b;
+                }
+
+                fn rem_min_overflow_panic<const LANES: usize>() {
+                    let a = Vector::<LANES>::splat(Scalar::MIN);
+                    let b = Vector::<LANES>::splat(-1);
+                    let _ = a % b;
+                }
+
+                fn rem_zero_panic<const LANES: usize>() {
+                    let a = Vector::<LANES>::splat(42);
+                    let b = Vector::<LANES>::splat(0);
+                    let _ = a % b;
+                }
+            }
+
+            test_helpers::test_lanes! {
+                fn div_neg_one_no_panic<const LANES: usize>() {
+                    let a = Vector::<LANES>::splat(42);
+                    let b = Vector::<LANES>::splat(-1);
+                    let _ = a / b;
+                }
+
+                fn rem_neg_one_no_panic<const LANES: usize>() {
+                    let a = Vector::<LANES>::splat(42);
+                    let b = Vector::<LANES>::splat(-1);
+                    let _ = a % b;
+                }
+            }
+
             impl_binary_op_test!(Vector<LANES>, Scalar, Add::add, AddAssign::add_assign, Scalar::wrapping_add);
             impl_binary_op_test!(Vector<LANES>, Scalar, Sub::sub, SubAssign::sub_assign, Scalar::wrapping_sub);
             impl_binary_op_test!(Vector<LANES>, Scalar, Mul::mul, MulAssign::mul_assign, Scalar::wrapping_mul);
@@ -162,6 +209,14 @@ macro_rules! impl_unsigned_tests {
             type Vector<const LANES: usize> = core_simd::$vector<LANES>;
             type Scalar = $scalar;
 
+            test_helpers::test_lanes_panic! {
+                fn rem_zero_panic<const LANES: usize>() {
+                    let a = Vector::<LANES>::splat(42);
+                    let b = Vector::<LANES>::splat(0);
+                    let _ = a % b;
+                }
+            }
+
             impl_binary_op_test!(Vector<LANES>, Scalar, Add::add, AddAssign::add_assign, Scalar::wrapping_add);
             impl_binary_op_test!(Vector<LANES>, Scalar, Sub::sub, SubAssign::sub_assign, Scalar::wrapping_sub);
             impl_binary_op_test!(Vector<LANES>, Scalar, Mul::mul, MulAssign::mul_assign, Scalar::wrapping_mul);
@@ -187,7 +242,7 @@ macro_rules! impl_float_tests {
             impl_unary_op_test!(Vector<LANES>, Scalar, Neg::neg);
             impl_binary_op_test!(Vector<LANES>, Scalar, Add::add, AddAssign::add_assign);
             impl_binary_op_test!(Vector<LANES>, Scalar, Sub::sub, SubAssign::sub_assign);
-            impl_binary_op_test!(Vector<LANES>, Scalar, Mul::mul, SubAssign::sub_assign);
+            impl_binary_op_test!(Vector<LANES>, Scalar, Mul::mul, MulAssign::mul_assign);
             impl_binary_op_test!(Vector<LANES>, Scalar, Div::div, DivAssign::div_assign);
             impl_binary_op_test!(Vector<LANES>, Scalar, Rem::rem, RemAssign::rem_assign);
 
diff --git a/crates/core_simd/tests/u128_ops.rs b/crates/core_simd/tests/u128_ops.rs
index eea7e3297c66..4be7d751ffd8 100644
--- a/crates/core_simd/tests/u128_ops.rs
+++ b/crates/core_simd/tests/u128_ops.rs
@@ -1,4 +1,3 @@
 #[macro_use]
-#[path = "ops_macros.rs"]
-mod macros;
+mod ops_macros;
 impl_unsigned_tests! { SimdU128, u128 }
diff --git a/crates/core_simd/tests/u16_ops.rs b/crates/core_simd/tests/u16_ops.rs
index ce9951a87c0a..488e703d54fb 100644
--- a/crates/core_simd/tests/u16_ops.rs
+++ b/crates/core_simd/tests/u16_ops.rs
@@ -1,4 +1,3 @@
 #[macro_use]
-#[path = "ops_macros.rs"]
-mod macros;
+mod ops_macros;
 impl_unsigned_tests! { SimdU16, u16 }
diff --git a/crates/core_simd/tests/u32_ops.rs b/crates/core_simd/tests/u32_ops.rs
index 87bedbd43b7c..bf0631029e37 100644
--- a/crates/core_simd/tests/u32_ops.rs
+++ b/crates/core_simd/tests/u32_ops.rs
@@ -1,4 +1,3 @@
 #[macro_use]
-#[path = "ops_macros.rs"]
-mod macros;
+mod ops_macros;
 impl_unsigned_tests! { SimdU32, u32 }
diff --git a/crates/core_simd/tests/u64_ops.rs b/crates/core_simd/tests/u64_ops.rs
index ec76891da66e..e52fc3cfce1f 100644
--- a/crates/core_simd/tests/u64_ops.rs
+++ b/crates/core_simd/tests/u64_ops.rs
@@ -1,4 +1,3 @@
 #[macro_use]
-#[path = "ops_macros.rs"]
-mod macros;
+mod ops_macros;
 impl_unsigned_tests! { SimdU64, u64 }
diff --git a/crates/core_simd/tests/u8_ops.rs b/crates/core_simd/tests/u8_ops.rs
index 00a63d846133..45be3580ec39 100644
--- a/crates/core_simd/tests/u8_ops.rs
+++ b/crates/core_simd/tests/u8_ops.rs
@@ -1,4 +1,3 @@
 #[macro_use]
-#[path = "ops_macros.rs"]
-mod macros;
+mod ops_macros;
 impl_unsigned_tests! { SimdU8, u8 }
diff --git a/crates/core_simd/tests/usize_ops.rs b/crates/core_simd/tests/usize_ops.rs
index dd49c656cbe8..1ce6e7180045 100644
--- a/crates/core_simd/tests/usize_ops.rs
+++ b/crates/core_simd/tests/usize_ops.rs
@@ -1,4 +1,3 @@
 #[macro_use]
-#[path = "ops_macros.rs"]
-mod macros;
+mod ops_macros;
 impl_unsigned_tests! { SimdUsize, usize }
diff --git a/crates/test_helpers/src/lib.rs b/crates/test_helpers/src/lib.rs
index e1832bf63779..253435dea336 100644
--- a/crates/test_helpers/src/lib.rs
+++ b/crates/test_helpers/src/lib.rs
@@ -280,4 +280,61 @@ macro_rules! test_lanes {
             }
         )*
     }
-}
+}  
+
+#[macro_export]
+macro_rules! test_lanes_panic {
+    {
+        $(fn $test:ident<const $lanes:ident: usize>() $body:tt)*
+    } => {
+        $(
+            mod $test {
+                use super::*;
+
+                fn implementation<const $lanes: usize>() $body
+
+                #[test]
+                #[should_panic]
+                fn lanes_1() {
+                    implementation::<1>();
+                }
+
+                #[test]
+                #[should_panic]
+                fn lanes_2() {
+                    implementation::<2>();
+                }
+
+                #[test]
+                #[should_panic]
+                fn lanes_4() {
+                    implementation::<4>();
+                }
+
+                #[test]
+                #[should_panic]
+                fn lanes_8() {
+                    implementation::<8>();
+                }
+
+                #[test]
+                #[should_panic]
+                fn lanes_16() {
+                    implementation::<16>();
+                }
+
+                #[test]
+                #[should_panic]
+                fn lanes_32() {
+                    implementation::<32>();
+                }
+
+                #[test]
+                #[should_panic]
+                fn lanes_64() {
+                    implementation::<64>();
+                }
+            }
+        )*
+    }
+}  

From 8c378d30271de8cb9e09a4b70f28db71f282f413 Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Sat, 13 Feb 2021 15:58:44 -0500
Subject: [PATCH 094/249] Add documentation

---
 crates/core_simd/tests/helpers/biteq.rs     | 120 --------------------
 crates/core_simd/tests/helpers/mod.rs       |   2 -
 crates/core_simd/tests/mask_ops_impl/mod.rs |   4 -
 crates/core_simd/tests/ops_macros.rs        |  19 ++++
 crates/test_helpers/src/array.rs            |   2 +
 crates/test_helpers/src/biteq.rs            |   2 +
 crates/test_helpers/src/lib.rs              |  11 ++
 crates/test_helpers/src/wasm.rs             |   2 +
 8 files changed, 36 insertions(+), 126 deletions(-)
 delete mode 100644 crates/core_simd/tests/helpers/biteq.rs
 delete mode 100644 crates/core_simd/tests/helpers/mod.rs

diff --git a/crates/core_simd/tests/helpers/biteq.rs b/crates/core_simd/tests/helpers/biteq.rs
deleted file mode 100644
index 9da2bdfce42e..000000000000
--- a/crates/core_simd/tests/helpers/biteq.rs
+++ /dev/null
@@ -1,120 +0,0 @@
-pub(crate) trait BitEq {
-    fn biteq(&self, other: &Self) -> bool;
-    fn fmt(&self, f: &mut core::fmt::Formatter) -> core::fmt::Result;
-}
-
-macro_rules! impl_biteq {
-    { integer impl BitEq for $($type:ty,)* } => {
-        $(
-        impl BitEq for $type {
-            fn biteq(&self, other: &Self) -> bool {
-                self == other
-            }
-
-            fn fmt(&self, f: &mut core::fmt::Formatter) -> core::fmt::Result {
-                write!(f, "{:?} ({:x})", self, self)
-            }
-        }
-        )*
-    };
-    { float impl BitEq for $($type:ty,)* } => {
-        $(
-        impl BitEq for $type {
-            fn biteq(&self, other: &Self) -> bool {
-                self.to_bits() == other.to_bits()
-            }
-
-            fn fmt(&self, f: &mut core::fmt::Formatter) -> core::fmt::Result {
-                write!(f, "{:?} ({:x})", self, self.to_bits())
-            }
-        }
-        )*
-    };
-    { vector impl BitEq for $($type:ty,)* } => {
-        $(
-        impl BitEq for $type {
-            fn biteq(&self, other: &Self) -> bool {
-                let a: &[_] = self.as_ref();
-                let b: &[_] = other.as_ref();
-                if a.len() == b.len() {
-                    a.iter().zip(b.iter()).fold(true, |value, (left, right)| {
-                        value && left.biteq(right)
-                    })
-                } else {
-                    false
-                }
-            }
-
-            fn fmt(&self, f: &mut core::fmt::Formatter) -> core::fmt::Result {
-                #[repr(transparent)]
-                struct Wrapper<'a, T: BitEq>(&'a T);
-
-                impl<T: BitEq> core::fmt::Debug for Wrapper<'_, T> {
-                    fn fmt(&self, f: &mut core::fmt::Formatter) -> core::fmt::Result {
-                        self.0.fmt(f)
-                    }
-                }
-
-                let slice: &[_] = self.as_ref();
-
-                f.debug_list()
-                    .entries(slice.iter().map(|x| Wrapper(x)))
-                    .finish()
-            }
-        }
-        )*
-    };
-}
-
-impl_biteq! {
-    integer impl BitEq for
-        u8, u16, u32, u64, u128, usize,
-        i8, i16, i32, i64, i128, isize,
-}
-
-impl_biteq! {
-    float impl BitEq for f32, f64,
-}
-
-impl_biteq! {
-    vector impl BitEq for
-        core_simd::u8x8,    core_simd::u8x16,   core_simd::u8x32,   core_simd::u8x64,
-        core_simd::i8x8,    core_simd::i8x16,   core_simd::i8x32,   core_simd::i8x64,
-        core_simd::u16x4,   core_simd::u16x8,   core_simd::u16x16,  core_simd::u16x32,
-        core_simd::i16x4,   core_simd::i16x8,   core_simd::i16x16,  core_simd::i16x32,
-        core_simd::u32x2,   core_simd::u32x4,   core_simd::u32x8,   core_simd::u32x16,
-        core_simd::i32x2,   core_simd::i32x4,   core_simd::i32x8,   core_simd::i32x16,
-        core_simd::u64x2,   core_simd::u64x4,   core_simd::u64x8,
-        core_simd::i64x2,   core_simd::i64x4,   core_simd::i64x8,
-        core_simd::u128x2,  core_simd::u128x4,
-        core_simd::i128x2,  core_simd::i128x4,
-        core_simd::usizex2, core_simd::usizex4, core_simd::usizex8,
-        core_simd::isizex2, core_simd::isizex4, core_simd::isizex8,
-        core_simd::f32x2, core_simd::f32x4, core_simd::f32x8, core_simd::f32x16,
-        core_simd::f64x2, core_simd::f64x4, core_simd::f64x8,
-}
-
-pub(crate) struct BitEqWrapper<'a, T>(pub(crate) &'a T);
-
-impl<T: BitEq> PartialEq for BitEqWrapper<'_, T> {
-    fn eq(&self, other: &Self) -> bool {
-        self.0.biteq(other.0)
-    }
-}
-
-impl<T: BitEq> core::fmt::Debug for BitEqWrapper<'_, T> {
-    fn fmt(&self, f: &mut core::fmt::Formatter) -> core::fmt::Result {
-        self.0.fmt(f)
-    }
-}
-
-macro_rules! assert_biteq {
-    { $a:expr, $b:expr } => {
-        {
-            use helpers::biteq::BitEqWrapper;
-            let a = $a;
-            let b = $b;
-            assert_eq!(BitEqWrapper(&a), BitEqWrapper(&b));
-        }
-    }
-}
diff --git a/crates/core_simd/tests/helpers/mod.rs b/crates/core_simd/tests/helpers/mod.rs
deleted file mode 100644
index 41b4fea9d95d..000000000000
--- a/crates/core_simd/tests/helpers/mod.rs
+++ /dev/null
@@ -1,2 +0,0 @@
-#[macro_use]
-pub mod biteq;
diff --git a/crates/core_simd/tests/mask_ops_impl/mod.rs b/crates/core_simd/tests/mask_ops_impl/mod.rs
index 89f5e1b0b52a..99d735be2935 100644
--- a/crates/core_simd/tests/mask_ops_impl/mod.rs
+++ b/crates/core_simd/tests/mask_ops_impl/mod.rs
@@ -1,7 +1,3 @@
-#[macro_use]
-#[path = "../helpers/mod.rs"]
-mod helpers;
-
 #[macro_use]
 mod mask_macros;
 
diff --git a/crates/core_simd/tests/ops_macros.rs b/crates/core_simd/tests/ops_macros.rs
index a4b26a1777af..58e80a8f277e 100644
--- a/crates/core_simd/tests/ops_macros.rs
+++ b/crates/core_simd/tests/ops_macros.rs
@@ -1,3 +1,6 @@
+/// Implements a test on a unary operation using proptest.
+///
+/// Compares the vector operation to the equivalent scalar operation.
 #[macro_export]
 macro_rules! impl_unary_op_test {
     { $vector:ty, $scalar:ty, $trait:ident :: $fn:ident, $scalar_fn:expr } => {
@@ -16,6 +19,9 @@ macro_rules! impl_unary_op_test {
     };
 }
 
+/// Implements a test on a binary operation using proptest.
+///
+/// Compares the vector operation to the equivalent scalar operation.
 #[macro_export]
 macro_rules! impl_binary_op_test {
     { $vector:ty, $scalar:ty, $trait:ident :: $fn:ident, $trait_assign:ident :: $fn_assign:ident, $scalar_fn:expr } => {
@@ -70,6 +76,12 @@ macro_rules! impl_binary_op_test {
     };
 }
 
+/// Implements a test on a binary operation using proptest.
+///
+/// Like `impl_binary_op_test`, but allows providing a function for rejecting particular inputs
+/// (like the `proptest_assume` macro).
+///
+/// Compares the vector operation to the equivalent scalar operation.
 #[macro_export]
 macro_rules! impl_binary_checked_op_test {
     { $vector:ty, $scalar:ty, $trait:ident :: $fn:ident, $trait_assign:ident :: $fn_assign:ident, $scalar_fn:expr, $check_fn:expr } => {
@@ -124,6 +136,7 @@ macro_rules! impl_binary_checked_op_test {
     };
 }
 
+/// Implement tests for signed integers.
 #[macro_export]
 macro_rules! impl_signed_tests {
     { $vector:ident, $scalar:tt } => {
@@ -191,6 +204,8 @@ macro_rules! impl_signed_tests {
             impl_binary_op_test!(Vector<LANES>, Scalar, Add::add, AddAssign::add_assign, Scalar::wrapping_add);
             impl_binary_op_test!(Vector<LANES>, Scalar, Sub::sub, SubAssign::sub_assign, Scalar::wrapping_sub);
             impl_binary_op_test!(Vector<LANES>, Scalar, Mul::mul, MulAssign::mul_assign, Scalar::wrapping_mul);
+
+            // Exclude Div and Rem panicking cases
             impl_binary_checked_op_test!(Vector<LANES>, Scalar, Div::div, DivAssign::div_assign, Scalar::wrapping_div, |x, y| y != 0 && !(x == Scalar::MIN && y == -1));
             impl_binary_checked_op_test!(Vector<LANES>, Scalar, Rem::rem, RemAssign::rem_assign, Scalar::wrapping_rem, |x, y| y != 0 && !(x == Scalar::MIN && y == -1));
 
@@ -202,6 +217,7 @@ macro_rules! impl_signed_tests {
     }
 }
 
+/// Implement tests for unsigned integers.
 #[macro_export]
 macro_rules! impl_unsigned_tests {
     { $vector:ident, $scalar:tt } => {
@@ -220,6 +236,8 @@ macro_rules! impl_unsigned_tests {
             impl_binary_op_test!(Vector<LANES>, Scalar, Add::add, AddAssign::add_assign, Scalar::wrapping_add);
             impl_binary_op_test!(Vector<LANES>, Scalar, Sub::sub, SubAssign::sub_assign, Scalar::wrapping_sub);
             impl_binary_op_test!(Vector<LANES>, Scalar, Mul::mul, MulAssign::mul_assign, Scalar::wrapping_mul);
+
+            // Exclude Div and Rem panicking cases
             impl_binary_checked_op_test!(Vector<LANES>, Scalar, Div::div, DivAssign::div_assign, Scalar::wrapping_div, |_, y| y != 0);
             impl_binary_checked_op_test!(Vector<LANES>, Scalar, Rem::rem, RemAssign::rem_assign, Scalar::wrapping_rem, |_, y| y != 0);
 
@@ -231,6 +249,7 @@ macro_rules! impl_unsigned_tests {
     }
 }
 
+/// Implement tests for floating point numbers.
 #[macro_export]
 macro_rules! impl_float_tests {
     { $vector:ident, $scalar:tt, $int_scalar:tt } => {
diff --git a/crates/test_helpers/src/array.rs b/crates/test_helpers/src/array.rs
index 3953d0bbea5e..c64bfee4f2d1 100644
--- a/crates/test_helpers/src/array.rs
+++ b/crates/test_helpers/src/array.rs
@@ -1,3 +1,5 @@
+//! Generic-length array strategy.
+
 // Adapted from proptest's array code
 // Copyright 2017 Jason Lingle
 
diff --git a/crates/test_helpers/src/biteq.rs b/crates/test_helpers/src/biteq.rs
index 23aa7d4d908e..8c6280643803 100644
--- a/crates/test_helpers/src/biteq.rs
+++ b/crates/test_helpers/src/biteq.rs
@@ -1,3 +1,5 @@
+//! Compare numeric types by exact bit value.
+
 pub trait BitEq {
     fn biteq(&self, other: &Self) -> bool;
     fn fmt(&self, f: &mut core::fmt::Formatter) -> core::fmt::Result;
diff --git a/crates/test_helpers/src/lib.rs b/crates/test_helpers/src/lib.rs
index 253435dea336..a81713e865b7 100644
--- a/crates/test_helpers/src/lib.rs
+++ b/crates/test_helpers/src/lib.rs
@@ -6,6 +6,9 @@ pub mod wasm;
 #[macro_use]
 pub mod biteq;
 
+/// Specifies the default strategy for testing a type.
+///
+/// This strategy should be what "makes sense" to test.
 pub trait DefaultStrategy {
     type Strategy: proptest::strategy::Strategy<Value = Self>;
     fn default_strategy() -> Self::Strategy;
@@ -74,6 +77,7 @@ impl<T: core::fmt::Debug + DefaultStrategy, const LANES: usize> DefaultStrategy
     }
 }
 
+/// Test a function that takes a single value.
 pub fn test_1<A: core::fmt::Debug + DefaultStrategy>(
     f: &dyn Fn(A) -> proptest::test_runner::TestCaseResult,
 ) {
@@ -81,6 +85,7 @@ pub fn test_1<A: core::fmt::Debug + DefaultStrategy>(
     runner.run(&A::default_strategy(), f).unwrap();
 }
 
+/// Test a function that takes two values.
 pub fn test_2<A: core::fmt::Debug + DefaultStrategy, B: core::fmt::Debug + DefaultStrategy>(
     f: &dyn Fn(A, B) -> proptest::test_runner::TestCaseResult,
 ) {
@@ -92,6 +97,7 @@ pub fn test_2<A: core::fmt::Debug + DefaultStrategy, B: core::fmt::Debug + Defau
         .unwrap();
 }
 
+/// Test a unary vector function against a unary scalar function, applied elementwise.
 #[inline(never)]
 pub fn test_unary_elementwise<Scalar, ScalarResult, Vector, VectorResult, const LANES: usize>(
     fv: &dyn Fn(Vector) -> VectorResult,
@@ -118,6 +124,7 @@ pub fn test_unary_elementwise<Scalar, ScalarResult, Vector, VectorResult, const
     });
 }
 
+/// Test a binary vector function against a binary scalar function, applied elementwise.
 #[inline(never)]
 pub fn test_binary_elementwise<
     Scalar1,
@@ -154,6 +161,7 @@ pub fn test_binary_elementwise<
     });
 }
 
+/// Test a binary vector-scalar function against a binary scalar function, applied elementwise.
 #[inline(never)]
 pub fn test_binary_scalar_rhs_elementwise<
     Scalar1,
@@ -188,6 +196,7 @@ pub fn test_binary_scalar_rhs_elementwise<
     });
 }
 
+/// Test a binary vector-scalar function against a binary scalar function, applied elementwise.
 #[inline(never)]
 pub fn test_binary_scalar_lhs_elementwise<
     Scalar1,
@@ -222,6 +231,7 @@ pub fn test_binary_scalar_lhs_elementwise<
     });
 }
 
+/// Expand a const-generic test into separate tests for each possible lane count.
 #[macro_export]
 macro_rules! test_lanes {
     {
@@ -282,6 +292,7 @@ macro_rules! test_lanes {
     }
 }  
 
+/// Expand a const-generic `#[should_panic]` test into separate tests for each possible lane count.
 #[macro_export]
 macro_rules! test_lanes_panic {
     {
diff --git a/crates/test_helpers/src/wasm.rs b/crates/test_helpers/src/wasm.rs
index 02cb9264d7c7..3f11d67cbf39 100644
--- a/crates/test_helpers/src/wasm.rs
+++ b/crates/test_helpers/src/wasm.rs
@@ -1,3 +1,5 @@
+//! Strategies for `u128` and `i128`, since proptest doesn't provide them for the wasm target.
+
 macro_rules! impl_num {
     { $name:ident } => {
         pub(crate) mod $name {

From 15dd0ae14f01bd6f12e2323c1cd36e1a5ed2b048 Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Sat, 13 Feb 2021 17:13:55 -0500
Subject: [PATCH 095/249] Disable CI on branches without PR

---
 .travis.yml | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/.travis.yml b/.travis.yml
index c1fda2e4f170..a06fa62d50fd 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -1,3 +1,7 @@
+branches:
+  only:
+    - master
+
 language: rust
 rust:
   - nightly

From 2b3f4b258ce59285d8cd556ac8803eefd716ba49 Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Mon, 15 Feb 2021 18:38:35 -0500
Subject: [PATCH 096/249] Add LanesAtMost64 bounds

---
 crates/core_simd/src/macros.rs |  2 +-
 crates/test_helpers/src/lib.rs | 38 ++++++++++++++++++++++++++++++++--
 2 files changed, 37 insertions(+), 3 deletions(-)

diff --git a/crates/core_simd/src/macros.rs b/crates/core_simd/src/macros.rs
index 75104dc50e45..db90c4e3149b 100644
--- a/crates/core_simd/src/macros.rs
+++ b/crates/core_simd/src/macros.rs
@@ -141,7 +141,7 @@ macro_rules! impl_vector {
             }
         }
 
-        impl <const LANES: usize> From<$name<LANES>> for [$type; LANES] {
+        impl <const LANES: usize> From<$name<LANES>> for [$type; LANES] where $name<LANES>: crate::LanesAtMost64 {
             fn from(vector: $name<LANES>) -> Self {
                 vector.to_array()
             }
diff --git a/crates/test_helpers/src/lib.rs b/crates/test_helpers/src/lib.rs
index a81713e865b7..2c74c02d48a0 100644
--- a/crates/test_helpers/src/lib.rs
+++ b/crates/test_helpers/src/lib.rs
@@ -241,7 +241,24 @@ macro_rules! test_lanes {
             mod $test {
                 use super::*;
 
-                fn implementation<const $lanes: usize>() $body
+                fn implementation<const $lanes: usize>()
+                where
+                    core_simd::SimdU8<$lanes>: core_simd::LanesAtMost64,
+                    core_simd::SimdU16<$lanes>: core_simd::LanesAtMost64,
+                    core_simd::SimdU32<$lanes>: core_simd::LanesAtMost64,
+                    core_simd::SimdU64<$lanes>: core_simd::LanesAtMost64,
+                    core_simd::SimdU128<$lanes>: core_simd::LanesAtMost64,
+                    core_simd::SimdUsize<$lanes>: core_simd::LanesAtMost64,
+                    core_simd::SimdI8<$lanes>: core_simd::LanesAtMost64,
+                    core_simd::SimdI16<$lanes>: core_simd::LanesAtMost64,
+                    core_simd::SimdI32<$lanes>: core_simd::LanesAtMost64,
+                    core_simd::SimdI64<$lanes>: core_simd::LanesAtMost64,
+                    core_simd::SimdI128<$lanes>: core_simd::LanesAtMost64,
+                    core_simd::SimdIsize<$lanes>: core_simd::LanesAtMost64,
+                    core_simd::SimdF32<$lanes>: core_simd::LanesAtMost64,
+                    core_simd::SimdF64<$lanes>: core_simd::LanesAtMost64,
+                    core_simd::BitMask<$lanes>: core_simd::LanesAtMost64,
+                $body
 
                 #[cfg(target_arch = "wasm32")]
                 wasm_bindgen_test::wasm_bindgen_test_configure!(run_in_browser);
@@ -302,7 +319,24 @@ macro_rules! test_lanes_panic {
             mod $test {
                 use super::*;
 
-                fn implementation<const $lanes: usize>() $body
+                fn implementation<const $lanes: usize>()
+                where
+                    core_simd::SimdU8<$lanes>: core_simd::LanesAtMost64,
+                    core_simd::SimdU16<$lanes>: core_simd::LanesAtMost64,
+                    core_simd::SimdU32<$lanes>: core_simd::LanesAtMost64,
+                    core_simd::SimdU64<$lanes>: core_simd::LanesAtMost64,
+                    core_simd::SimdU128<$lanes>: core_simd::LanesAtMost64,
+                    core_simd::SimdUsize<$lanes>: core_simd::LanesAtMost64,
+                    core_simd::SimdI8<$lanes>: core_simd::LanesAtMost64,
+                    core_simd::SimdI16<$lanes>: core_simd::LanesAtMost64,
+                    core_simd::SimdI32<$lanes>: core_simd::LanesAtMost64,
+                    core_simd::SimdI64<$lanes>: core_simd::LanesAtMost64,
+                    core_simd::SimdI128<$lanes>: core_simd::LanesAtMost64,
+                    core_simd::SimdIsize<$lanes>: core_simd::LanesAtMost64,
+                    core_simd::SimdF32<$lanes>: core_simd::LanesAtMost64,
+                    core_simd::SimdF64<$lanes>: core_simd::LanesAtMost64,
+                    core_simd::BitMask<$lanes>: core_simd::LanesAtMost64,
+                $body
 
                 #[test]
                 #[should_panic]

From 2f2a463c0ddd4f22e29c3ad7e4a8ed8f1090e7a8 Mon Sep 17 00:00:00 2001
From: Jubilee Young <workingjubilee@gmail.com>
Date: Tue, 16 Feb 2021 10:02:02 -0800
Subject: [PATCH 097/249] Remove From<Scalar> for SimdTy impl

0. It was not being tested.
1. The possible conversions are ambiguous between splatting
   and setting a single value but zero-initializing the rest.
2. Splat works fine.
---
 crates/core_simd/src/macros.rs | 8 --------
 1 file changed, 8 deletions(-)

diff --git a/crates/core_simd/src/macros.rs b/crates/core_simd/src/macros.rs
index db90c4e3149b..d78403ddb93d 100644
--- a/crates/core_simd/src/macros.rs
+++ b/crates/core_simd/src/macros.rs
@@ -147,14 +147,6 @@ macro_rules! impl_vector {
             }
         }
 
-        // splat
-        impl<const LANES: usize> From<$type> for $name<LANES> where Self: crate::LanesAtMost64 {
-            #[inline]
-            fn from(value: $type) -> Self {
-                Self::splat(value)
-            }
-        }
-
         impl_shuffle_2pow_lanes!{ $name }
     }
 }

From 1a19ad4fb035bba35edff46bd58f953f6de34eec Mon Sep 17 00:00:00 2001
From: Jubilee Young <workingjubilee@gmail.com>
Date: Tue, 16 Feb 2021 16:37:24 -0800
Subject: [PATCH 098/249] Reorg vectors into crate::vector::*;

---
 crates/core_simd/src/lib.rs                   | 32 ++-----------------
 crates/core_simd/src/vector.rs                | 29 +++++++++++++++++
 .../core_simd/src/{ => vector}/vectors_f32.rs |  0
 .../core_simd/src/{ => vector}/vectors_f64.rs |  0
 .../src/{ => vector}/vectors_i128.rs          |  0
 .../core_simd/src/{ => vector}/vectors_i16.rs |  0
 .../core_simd/src/{ => vector}/vectors_i32.rs |  0
 .../core_simd/src/{ => vector}/vectors_i64.rs |  0
 .../core_simd/src/{ => vector}/vectors_i8.rs  |  0
 .../src/{ => vector}/vectors_isize.rs         |  0
 .../src/{ => vector}/vectors_u128.rs          |  0
 .../core_simd/src/{ => vector}/vectors_u16.rs |  0
 .../core_simd/src/{ => vector}/vectors_u32.rs |  0
 .../core_simd/src/{ => vector}/vectors_u64.rs |  0
 .../core_simd/src/{ => vector}/vectors_u8.rs  |  0
 .../src/{ => vector}/vectors_usize.rs         |  0
 16 files changed, 31 insertions(+), 30 deletions(-)
 create mode 100644 crates/core_simd/src/vector.rs
 rename crates/core_simd/src/{ => vector}/vectors_f32.rs (100%)
 rename crates/core_simd/src/{ => vector}/vectors_f64.rs (100%)
 rename crates/core_simd/src/{ => vector}/vectors_i128.rs (100%)
 rename crates/core_simd/src/{ => vector}/vectors_i16.rs (100%)
 rename crates/core_simd/src/{ => vector}/vectors_i32.rs (100%)
 rename crates/core_simd/src/{ => vector}/vectors_i64.rs (100%)
 rename crates/core_simd/src/{ => vector}/vectors_i8.rs (100%)
 rename crates/core_simd/src/{ => vector}/vectors_isize.rs (100%)
 rename crates/core_simd/src/{ => vector}/vectors_u128.rs (100%)
 rename crates/core_simd/src/{ => vector}/vectors_u16.rs (100%)
 rename crates/core_simd/src/{ => vector}/vectors_u32.rs (100%)
 rename crates/core_simd/src/{ => vector}/vectors_u64.rs (100%)
 rename crates/core_simd/src/{ => vector}/vectors_u8.rs (100%)
 rename crates/core_simd/src/{ => vector}/vectors_usize.rs (100%)

diff --git a/crates/core_simd/src/lib.rs b/crates/core_simd/src/lib.rs
index 1f216bacd9af..6c499a197b14 100644
--- a/crates/core_simd/src/lib.rs
+++ b/crates/core_simd/src/lib.rs
@@ -20,33 +20,5 @@ pub use lanes_at_most_64::LanesAtMost64;
 mod masks;
 pub use masks::*;
 
-mod vectors_u8;
-pub use vectors_u8::*;
-mod vectors_u16;
-pub use vectors_u16::*;
-mod vectors_u32;
-pub use vectors_u32::*;
-mod vectors_u64;
-pub use vectors_u64::*;
-mod vectors_u128;
-pub use vectors_u128::*;
-mod vectors_usize;
-pub use vectors_usize::*;
-
-mod vectors_i8;
-pub use vectors_i8::*;
-mod vectors_i16;
-pub use vectors_i16::*;
-mod vectors_i32;
-pub use vectors_i32::*;
-mod vectors_i64;
-pub use vectors_i64::*;
-mod vectors_i128;
-pub use vectors_i128::*;
-mod vectors_isize;
-pub use vectors_isize::*;
-
-mod vectors_f32;
-pub use vectors_f32::*;
-mod vectors_f64;
-pub use vectors_f64::*;
+mod vector;
+pub use vector::*;
diff --git a/crates/core_simd/src/vector.rs b/crates/core_simd/src/vector.rs
new file mode 100644
index 000000000000..592a809bb47d
--- /dev/null
+++ b/crates/core_simd/src/vector.rs
@@ -0,0 +1,29 @@
+mod vectors_f32;
+mod vectors_f64;
+mod vectors_i128;
+mod vectors_i16;
+mod vectors_i32;
+mod vectors_i64;
+mod vectors_i8;
+mod vectors_isize;
+mod vectors_u128;
+mod vectors_u16;
+mod vectors_u32;
+mod vectors_u64;
+mod vectors_u8;
+mod vectors_usize;
+
+pub use vectors_f32::*;
+pub use vectors_f64::*;
+pub use vectors_i128::*;
+pub use vectors_i16::*;
+pub use vectors_i32::*;
+pub use vectors_i64::*;
+pub use vectors_i8::*;
+pub use vectors_isize::*;
+pub use vectors_u128::*;
+pub use vectors_u16::*;
+pub use vectors_u32::*;
+pub use vectors_u64::*;
+pub use vectors_u8::*;
+pub use vectors_usize::*;
diff --git a/crates/core_simd/src/vectors_f32.rs b/crates/core_simd/src/vector/vectors_f32.rs
similarity index 100%
rename from crates/core_simd/src/vectors_f32.rs
rename to crates/core_simd/src/vector/vectors_f32.rs
diff --git a/crates/core_simd/src/vectors_f64.rs b/crates/core_simd/src/vector/vectors_f64.rs
similarity index 100%
rename from crates/core_simd/src/vectors_f64.rs
rename to crates/core_simd/src/vector/vectors_f64.rs
diff --git a/crates/core_simd/src/vectors_i128.rs b/crates/core_simd/src/vector/vectors_i128.rs
similarity index 100%
rename from crates/core_simd/src/vectors_i128.rs
rename to crates/core_simd/src/vector/vectors_i128.rs
diff --git a/crates/core_simd/src/vectors_i16.rs b/crates/core_simd/src/vector/vectors_i16.rs
similarity index 100%
rename from crates/core_simd/src/vectors_i16.rs
rename to crates/core_simd/src/vector/vectors_i16.rs
diff --git a/crates/core_simd/src/vectors_i32.rs b/crates/core_simd/src/vector/vectors_i32.rs
similarity index 100%
rename from crates/core_simd/src/vectors_i32.rs
rename to crates/core_simd/src/vector/vectors_i32.rs
diff --git a/crates/core_simd/src/vectors_i64.rs b/crates/core_simd/src/vector/vectors_i64.rs
similarity index 100%
rename from crates/core_simd/src/vectors_i64.rs
rename to crates/core_simd/src/vector/vectors_i64.rs
diff --git a/crates/core_simd/src/vectors_i8.rs b/crates/core_simd/src/vector/vectors_i8.rs
similarity index 100%
rename from crates/core_simd/src/vectors_i8.rs
rename to crates/core_simd/src/vector/vectors_i8.rs
diff --git a/crates/core_simd/src/vectors_isize.rs b/crates/core_simd/src/vector/vectors_isize.rs
similarity index 100%
rename from crates/core_simd/src/vectors_isize.rs
rename to crates/core_simd/src/vector/vectors_isize.rs
diff --git a/crates/core_simd/src/vectors_u128.rs b/crates/core_simd/src/vector/vectors_u128.rs
similarity index 100%
rename from crates/core_simd/src/vectors_u128.rs
rename to crates/core_simd/src/vector/vectors_u128.rs
diff --git a/crates/core_simd/src/vectors_u16.rs b/crates/core_simd/src/vector/vectors_u16.rs
similarity index 100%
rename from crates/core_simd/src/vectors_u16.rs
rename to crates/core_simd/src/vector/vectors_u16.rs
diff --git a/crates/core_simd/src/vectors_u32.rs b/crates/core_simd/src/vector/vectors_u32.rs
similarity index 100%
rename from crates/core_simd/src/vectors_u32.rs
rename to crates/core_simd/src/vector/vectors_u32.rs
diff --git a/crates/core_simd/src/vectors_u64.rs b/crates/core_simd/src/vector/vectors_u64.rs
similarity index 100%
rename from crates/core_simd/src/vectors_u64.rs
rename to crates/core_simd/src/vector/vectors_u64.rs
diff --git a/crates/core_simd/src/vectors_u8.rs b/crates/core_simd/src/vector/vectors_u8.rs
similarity index 100%
rename from crates/core_simd/src/vectors_u8.rs
rename to crates/core_simd/src/vector/vectors_u8.rs
diff --git a/crates/core_simd/src/vectors_usize.rs b/crates/core_simd/src/vector/vectors_usize.rs
similarity index 100%
rename from crates/core_simd/src/vectors_usize.rs
rename to crates/core_simd/src/vector/vectors_usize.rs

From 27f094f5ee216b499ee9e666a7f6bdba361443f9 Mon Sep 17 00:00:00 2001
From: Jubilee Young <workingjubilee@gmail.com>
Date: Tue, 16 Feb 2021 17:31:44 -0800
Subject: [PATCH 099/249] Nominate base files

---
 crates/core_simd/src/vector.rs                     | 14 ++++++++------
 .../src/vector/{vectors_f32.rs => float.rs}        |  0
 .../src/vector/{vectors_isize.rs => int.rs}        |  0
 .../src/vector/{vectors_usize.rs => uint.rs}       |  0
 4 files changed, 8 insertions(+), 6 deletions(-)
 rename crates/core_simd/src/vector/{vectors_f32.rs => float.rs} (100%)
 rename crates/core_simd/src/vector/{vectors_isize.rs => int.rs} (100%)
 rename crates/core_simd/src/vector/{vectors_usize.rs => uint.rs} (100%)

diff --git a/crates/core_simd/src/vector.rs b/crates/core_simd/src/vector.rs
index 592a809bb47d..33b45a003239 100644
--- a/crates/core_simd/src/vector.rs
+++ b/crates/core_simd/src/vector.rs
@@ -1,29 +1,31 @@
-mod vectors_f32;
 mod vectors_f64;
 mod vectors_i128;
 mod vectors_i16;
 mod vectors_i32;
 mod vectors_i64;
 mod vectors_i8;
-mod vectors_isize;
 mod vectors_u128;
 mod vectors_u16;
 mod vectors_u32;
 mod vectors_u64;
 mod vectors_u8;
-mod vectors_usize;
 
-pub use vectors_f32::*;
 pub use vectors_f64::*;
 pub use vectors_i128::*;
 pub use vectors_i16::*;
 pub use vectors_i32::*;
 pub use vectors_i64::*;
 pub use vectors_i8::*;
-pub use vectors_isize::*;
 pub use vectors_u128::*;
 pub use vectors_u16::*;
 pub use vectors_u32::*;
 pub use vectors_u64::*;
 pub use vectors_u8::*;
-pub use vectors_usize::*;
+
+mod float;
+mod int;
+mod uint;
+
+pub use float::*;
+pub use int::*;
+pub use uint::*;
diff --git a/crates/core_simd/src/vector/vectors_f32.rs b/crates/core_simd/src/vector/float.rs
similarity index 100%
rename from crates/core_simd/src/vector/vectors_f32.rs
rename to crates/core_simd/src/vector/float.rs
diff --git a/crates/core_simd/src/vector/vectors_isize.rs b/crates/core_simd/src/vector/int.rs
similarity index 100%
rename from crates/core_simd/src/vector/vectors_isize.rs
rename to crates/core_simd/src/vector/int.rs
diff --git a/crates/core_simd/src/vector/vectors_usize.rs b/crates/core_simd/src/vector/uint.rs
similarity index 100%
rename from crates/core_simd/src/vector/vectors_usize.rs
rename to crates/core_simd/src/vector/uint.rs

From ca15e4fcd8dbf2a5eaa33e7b548b7b3f61cd590f Mon Sep 17 00:00:00 2001
From: Jubilee Young <workingjubilee@gmail.com>
Date: Tue, 16 Feb 2021 17:42:42 -0800
Subject: [PATCH 100/249] cat vector types by kind

Rearrange aliases to bottom of files
---
 crates/core_simd/src/vector.rs              |  24 ----
 crates/core_simd/src/vector/float.rs        |  27 ++++-
 crates/core_simd/src/vector/int.rs          | 128 ++++++++++++++++++--
 crates/core_simd/src/vector/uint.rs         | 128 ++++++++++++++++++--
 crates/core_simd/src/vector/vectors_f64.rs  |  22 ----
 crates/core_simd/src/vector/vectors_i128.rs |  18 ---
 crates/core_simd/src/vector/vectors_i16.rs  |  25 ----
 crates/core_simd/src/vector/vectors_i32.rs  |  25 ----
 crates/core_simd/src/vector/vectors_i64.rs  |  22 ----
 crates/core_simd/src/vector/vectors_i8.rs   |  25 ----
 crates/core_simd/src/vector/vectors_u128.rs |  18 ---
 crates/core_simd/src/vector/vectors_u16.rs  |  25 ----
 crates/core_simd/src/vector/vectors_u32.rs  |  25 ----
 crates/core_simd/src/vector/vectors_u64.rs  |  22 ----
 crates/core_simd/src/vector/vectors_u8.rs   |  23 ----
 15 files changed, 262 insertions(+), 295 deletions(-)
 delete mode 100644 crates/core_simd/src/vector/vectors_f64.rs
 delete mode 100644 crates/core_simd/src/vector/vectors_i128.rs
 delete mode 100644 crates/core_simd/src/vector/vectors_i16.rs
 delete mode 100644 crates/core_simd/src/vector/vectors_i32.rs
 delete mode 100644 crates/core_simd/src/vector/vectors_i64.rs
 delete mode 100644 crates/core_simd/src/vector/vectors_i8.rs
 delete mode 100644 crates/core_simd/src/vector/vectors_u128.rs
 delete mode 100644 crates/core_simd/src/vector/vectors_u16.rs
 delete mode 100644 crates/core_simd/src/vector/vectors_u32.rs
 delete mode 100644 crates/core_simd/src/vector/vectors_u64.rs
 delete mode 100644 crates/core_simd/src/vector/vectors_u8.rs

diff --git a/crates/core_simd/src/vector.rs b/crates/core_simd/src/vector.rs
index 33b45a003239..95534384b705 100644
--- a/crates/core_simd/src/vector.rs
+++ b/crates/core_simd/src/vector.rs
@@ -1,27 +1,3 @@
-mod vectors_f64;
-mod vectors_i128;
-mod vectors_i16;
-mod vectors_i32;
-mod vectors_i64;
-mod vectors_i8;
-mod vectors_u128;
-mod vectors_u16;
-mod vectors_u32;
-mod vectors_u64;
-mod vectors_u8;
-
-pub use vectors_f64::*;
-pub use vectors_i128::*;
-pub use vectors_i16::*;
-pub use vectors_i32::*;
-pub use vectors_i64::*;
-pub use vectors_i8::*;
-pub use vectors_u128::*;
-pub use vectors_u16::*;
-pub use vectors_u32::*;
-pub use vectors_u64::*;
-pub use vectors_u8::*;
-
 mod float;
 mod int;
 mod uint;
diff --git a/crates/core_simd/src/vector/float.rs b/crates/core_simd/src/vector/float.rs
index 5bb8f3a1c34d..56f43d0a18cf 100644
--- a/crates/core_simd/src/vector/float.rs
+++ b/crates/core_simd/src/vector/float.rs
@@ -8,6 +8,22 @@ where
 
 impl_float_vector! { SimdF32, f32, SimdU32 }
 
+from_transmute_x86! { unsafe f32x4 => __m128 }
+from_transmute_x86! { unsafe f32x8 => __m256 }
+//from_transmute_x86! { unsafe f32x16 => __m512 }
+
+/// A SIMD vector of containing `LANES` `f64` values.
+#[repr(simd)]
+pub struct SimdF64<const LANES: usize>([f64; LANES])
+where
+    Self: crate::LanesAtMost64;
+
+impl_float_vector! { SimdF64, f64, SimdU64 }
+
+from_transmute_x86! { unsafe f64x2 => __m128d }
+from_transmute_x86! { unsafe f64x4 => __m256d }
+//from_transmute_x86! { unsafe f64x8 => __m512d }
+
 /// Vector of two `f32` values
 pub type f32x2 = SimdF32<2>;
 
@@ -20,6 +36,11 @@ pub type f32x8 = SimdF32<8>;
 /// Vector of 16 `f32` values
 pub type f32x16 = SimdF32<16>;
 
-from_transmute_x86! { unsafe f32x4 => __m128 }
-from_transmute_x86! { unsafe f32x8 => __m256 }
-//from_transmute_x86! { unsafe f32x16 => __m512 }
+/// Vector of two `f64` values
+pub type f64x2 = SimdF64<2>;
+
+/// Vector of four `f64` values
+pub type f64x4 = SimdF64<4>;
+
+/// Vector of eight `f64` values
+pub type f64x8 = SimdF64<8>;
diff --git a/crates/core_simd/src/vector/int.rs b/crates/core_simd/src/vector/int.rs
index ee23dfe7d865..100e1d5afac6 100644
--- a/crates/core_simd/src/vector/int.rs
+++ b/crates/core_simd/src/vector/int.rs
@@ -8,15 +8,6 @@ where
 
 impl_integer_vector! { SimdIsize, isize }
 
-/// Vector of two `isize` values
-pub type isizex2 = SimdIsize<2>;
-
-/// Vector of four `isize` values
-pub type isizex4 = SimdIsize<4>;
-
-/// Vector of eight `isize` values
-pub type isizex8 = SimdIsize<8>;
-
 #[cfg(target_pointer_width = "32")]
 from_transmute_x86! { unsafe isizex4 => __m128i }
 #[cfg(target_pointer_width = "32")]
@@ -28,3 +19,122 @@ from_transmute_x86! { unsafe isizex2 => __m128i }
 from_transmute_x86! { unsafe isizex4 => __m256i }
 //#[cfg(target_pointer_width = "64")]
 //from_transmute_x86! { unsafe isizex8 => __m512i }
+
+/// A SIMD vector of containing `LANES` `i128` values.
+#[repr(simd)]
+pub struct SimdI128<const LANES: usize>([i128; LANES])
+where
+    Self: crate::LanesAtMost64;
+
+impl_integer_vector! { SimdI128, i128 }
+
+from_transmute_x86! { unsafe i128x2 => __m256i }
+//from_transmute_x86! { unsafe i128x4 => __m512i }
+
+/// A SIMD vector of containing `LANES` `i16` values.
+#[repr(simd)]
+pub struct SimdI16<const LANES: usize>([i16; LANES])
+where
+    Self: crate::LanesAtMost64;
+
+impl_integer_vector! { SimdI16, i16 }
+
+from_transmute_x86! { unsafe i16x8 => __m128i }
+from_transmute_x86! { unsafe i16x16 => __m256i }
+//from_transmute_x86! { unsafe i16x32 => __m512i }
+
+/// A SIMD vector of containing `LANES` `i32` values.
+#[repr(simd)]
+pub struct SimdI32<const LANES: usize>([i32; LANES])
+where
+    Self: crate::LanesAtMost64;
+
+impl_integer_vector! { SimdI32, i32 }
+
+from_transmute_x86! { unsafe i32x4 => __m128i }
+from_transmute_x86! { unsafe i32x8 => __m256i }
+//from_transmute_x86! { unsafe i32x16 => __m512i }
+
+/// A SIMD vector of containing `LANES` `i64` values.
+#[repr(simd)]
+pub struct SimdI64<const LANES: usize>([i64; LANES])
+where
+    Self: crate::LanesAtMost64;
+
+impl_integer_vector! { SimdI64, i64 }
+
+from_transmute_x86! { unsafe i64x2 => __m128i }
+from_transmute_x86! { unsafe i64x4 => __m256i }
+//from_transmute_x86! { unsafe i64x8 => __m512i }
+
+/// A SIMD vector of containing `LANES` `i8` values.
+#[repr(simd)]
+pub struct SimdI8<const LANES: usize>([i8; LANES])
+where
+    Self: crate::LanesAtMost64;
+
+impl_integer_vector! { SimdI8, i8 }
+
+from_transmute_x86! { unsafe i8x16 => __m128i }
+from_transmute_x86! { unsafe i8x32 => __m256i }
+//from_transmute_x86! { unsafe i8x64 => __m512i }
+
+/// Vector of two `isize` values
+pub type isizex2 = SimdIsize<2>;
+
+/// Vector of four `isize` values
+pub type isizex4 = SimdIsize<4>;
+
+/// Vector of eight `isize` values
+pub type isizex8 = SimdIsize<8>;
+
+/// Vector of two `i128` values
+pub type i128x2 = SimdI128<2>;
+
+/// Vector of four `i128` values
+pub type i128x4 = SimdI128<4>;
+
+/// Vector of four `i16` values
+pub type i16x4 = SimdI16<4>;
+
+/// Vector of eight `i16` values
+pub type i16x8 = SimdI16<8>;
+
+/// Vector of 16 `i16` values
+pub type i16x16 = SimdI16<16>;
+
+/// Vector of 32 `i16` values
+pub type i16x32 = SimdI16<32>;
+
+/// Vector of two `i32` values
+pub type i32x2 = SimdI32<2>;
+
+/// Vector of four `i32` values
+pub type i32x4 = SimdI32<4>;
+
+/// Vector of eight `i32` values
+pub type i32x8 = SimdI32<8>;
+
+/// Vector of 16 `i32` values
+pub type i32x16 = SimdI32<16>;
+
+/// Vector of two `i64` values
+pub type i64x2 = SimdI64<2>;
+
+/// Vector of four `i64` values
+pub type i64x4 = SimdI64<4>;
+
+/// Vector of eight `i64` values
+pub type i64x8 = SimdI64<8>;
+
+/// Vector of eight `i8` values
+pub type i8x8 = SimdI8<8>;
+
+/// Vector of 16 `i8` values
+pub type i8x16 = SimdI8<16>;
+
+/// Vector of 32 `i8` values
+pub type i8x32 = SimdI8<32>;
+
+/// Vector of 64 `i8` values
+pub type i8x64 = SimdI8<64>;
diff --git a/crates/core_simd/src/vector/uint.rs b/crates/core_simd/src/vector/uint.rs
index ea089aeb9d3c..6a16f06e50f2 100644
--- a/crates/core_simd/src/vector/uint.rs
+++ b/crates/core_simd/src/vector/uint.rs
@@ -8,15 +8,6 @@ where
 
 impl_integer_vector! { SimdUsize, usize }
 
-/// Vector of two `usize` values
-pub type usizex2 = SimdUsize<2>;
-
-/// Vector of four `usize` values
-pub type usizex4 = SimdUsize<4>;
-
-/// Vector of eight `usize` values
-pub type usizex8 = SimdUsize<8>;
-
 #[cfg(target_pointer_width = "32")]
 from_transmute_x86! { unsafe usizex4 => __m128i }
 #[cfg(target_pointer_width = "32")]
@@ -28,3 +19,122 @@ from_transmute_x86! { unsafe usizex2 => __m128i }
 from_transmute_x86! { unsafe usizex4 => __m256i }
 //#[cfg(target_pointer_width = "64")]
 //from_transmute_x86! { unsafe usizex8 => __m512i }
+
+/// A SIMD vector of containing `LANES` `u128` values.
+#[repr(simd)]
+pub struct SimdU128<const LANES: usize>([u128; LANES])
+where
+    Self: crate::LanesAtMost64;
+
+impl_integer_vector! { SimdU128, u128 }
+
+from_transmute_x86! { unsafe u128x2 => __m256i }
+//from_transmute_x86! { unsafe u128x4 => __m512i }
+
+/// A SIMD vector of containing `LANES` `u16` values.
+#[repr(simd)]
+pub struct SimdU16<const LANES: usize>([u16; LANES])
+where
+    Self: crate::LanesAtMost64;
+
+impl_integer_vector! { SimdU16, u16 }
+
+from_transmute_x86! { unsafe u16x8 => __m128i }
+from_transmute_x86! { unsafe u16x16 => __m256i }
+//from_transmute_x86! { unsafe u16x32 => __m512i }
+
+/// A SIMD vector of containing `LANES` `u32` values.
+#[repr(simd)]
+pub struct SimdU32<const LANES: usize>([u32; LANES])
+where
+    Self: crate::LanesAtMost64;
+
+impl_integer_vector! { SimdU32, u32 }
+
+from_transmute_x86! { unsafe u32x4 => __m128i }
+from_transmute_x86! { unsafe u32x8 => __m256i }
+//from_transmute_x86! { unsafe u32x16 => __m512i }
+
+/// A SIMD vector of containing `LANES` `u64` values.
+#[repr(simd)]
+pub struct SimdU64<const LANES: usize>([u64; LANES])
+where
+    Self: crate::LanesAtMost64;
+
+impl_integer_vector! { SimdU64, u64 }
+
+from_transmute_x86! { unsafe u64x2 => __m128i }
+from_transmute_x86! { unsafe u64x4 => __m256i }
+//from_transmute_x86! { unsafe u64x8 => __m512i }
+
+/// A SIMD vector of containing `LANES` `u8` values.
+#[repr(simd)]
+pub struct SimdU8<const LANES: usize>([u8; LANES])
+where
+    Self: crate::LanesAtMost64;
+
+impl_integer_vector! { SimdU8, u8 }
+
+from_transmute_x86! { unsafe u8x16 => __m128i }
+from_transmute_x86! { unsafe u8x32 => __m256i }
+//from_transmute_x86! { unsafe u8x64 => __m512i }
+
+/// Vector of two `usize` values
+pub type usizex2 = SimdUsize<2>;
+
+/// Vector of four `usize` values
+pub type usizex4 = SimdUsize<4>;
+
+/// Vector of eight `usize` values
+pub type usizex8 = SimdUsize<8>;
+
+/// Vector of two `u128` values
+pub type u128x2 = SimdU128<2>;
+
+/// Vector of four `u128` values
+pub type u128x4 = SimdU128<4>;
+
+/// Vector of four `u16` values
+pub type u16x4 = SimdU16<4>;
+
+/// Vector of eight `u16` values
+pub type u16x8 = SimdU16<8>;
+
+/// Vector of 16 `u16` values
+pub type u16x16 = SimdU16<16>;
+
+/// Vector of 32 `u16` values
+pub type u16x32 = SimdU16<32>;
+
+/// Vector of two `u32` values
+pub type u32x2 = SimdU32<2>;
+
+/// Vector of four `u32` values
+pub type u32x4 = SimdU32<4>;
+
+/// Vector of eight `u32` values
+pub type u32x8 = SimdU32<8>;
+
+/// Vector of 16 `u32` values
+pub type u32x16 = SimdU32<16>;
+
+/// Vector of two `u64` values
+pub type u64x2 = SimdU64<2>;
+
+/// Vector of four `u64` values
+pub type u64x4 = SimdU64<4>;
+
+/// Vector of eight `u64` values
+pub type u64x8 = SimdU64<8>;
+
+/// Vector of eight `u8` values
+pub type u8x8 = SimdU8<8>;
+
+/// Vector of 16 `u8` values
+pub type u8x16 = SimdU8<16>;
+
+/// Vector of 32 `u8` values
+pub type u8x32 = SimdU8<32>;
+
+/// Vector of 64 `u8` values
+pub type u8x64 = SimdU8<64>;
diff --git a/crates/core_simd/src/vector/vectors_f64.rs b/crates/core_simd/src/vector/vectors_f64.rs
deleted file mode 100644
index c0dca6a52ac6..000000000000
--- a/crates/core_simd/src/vector/vectors_f64.rs
+++ /dev/null
@@ -1,22 +0,0 @@
-#![allow(non_camel_case_types)]
-
-/// A SIMD vector of containing `LANES` `f64` values.
-#[repr(simd)]
-pub struct SimdF64<const LANES: usize>([f64; LANES])
-where
-    Self: crate::LanesAtMost64;
-
-impl_float_vector! { SimdF64, f64, SimdU64 }
-
-/// Vector of two `f64` values
-pub type f64x2 = SimdF64<2>;
-
-/// Vector of four `f64` values
-pub type f64x4 = SimdF64<4>;
-
-/// Vector of eight `f64` values
-pub type f64x8 = SimdF64<8>;
-
-from_transmute_x86! { unsafe f64x2 => __m128d }
-from_transmute_x86! { unsafe f64x4 => __m256d }
-//from_transmute_x86! { unsafe f64x8 => __m512d }
diff --git a/crates/core_simd/src/vector/vectors_i128.rs b/crates/core_simd/src/vector/vectors_i128.rs
deleted file mode 100644
index 568fa81da80e..000000000000
--- a/crates/core_simd/src/vector/vectors_i128.rs
+++ /dev/null
@@ -1,18 +0,0 @@
-#![allow(non_camel_case_types)]
-
-/// A SIMD vector of containing `LANES` `i128` values.
-#[repr(simd)]
-pub struct SimdI128<const LANES: usize>([i128; LANES])
-where
-    Self: crate::LanesAtMost64;
-
-impl_integer_vector! { SimdI128, i128 }
-
-/// Vector of two `i128` values
-pub type i128x2 = SimdI128<2>;
-
-/// Vector of four `i128` values
-pub type i128x4 = SimdI128<4>;
-
-from_transmute_x86! { unsafe i128x2 => __m256i }
-//from_transmute_x86! { unsafe i128x4 => __m512i }
diff --git a/crates/core_simd/src/vector/vectors_i16.rs b/crates/core_simd/src/vector/vectors_i16.rs
deleted file mode 100644
index d77e593a2edc..000000000000
--- a/crates/core_simd/src/vector/vectors_i16.rs
+++ /dev/null
@@ -1,25 +0,0 @@
-#![allow(non_camel_case_types)]
-
-/// A SIMD vector of containing `LANES` `i16` values.
-#[repr(simd)]
-pub struct SimdI16<const LANES: usize>([i16; LANES])
-where
-    Self: crate::LanesAtMost64;
-
-impl_integer_vector! { SimdI16, i16 }
-
-/// Vector of four `i16` values
-pub type i16x4 = SimdI16<4>;
-
-/// Vector of eight `i16` values
-pub type i16x8 = SimdI16<8>;
-
-/// Vector of 16 `i16` values
-pub type i16x16 = SimdI16<16>;
-
-/// Vector of 32 `i16` values
-pub type i16x32 = SimdI16<32>;
-
-from_transmute_x86! { unsafe i16x8 => __m128i }
-from_transmute_x86! { unsafe i16x16 => __m256i }
-//from_transmute_x86! { unsafe i16x32 => __m512i }
diff --git a/crates/core_simd/src/vector/vectors_i32.rs b/crates/core_simd/src/vector/vectors_i32.rs
deleted file mode 100644
index 0a89eeda3b2f..000000000000
--- a/crates/core_simd/src/vector/vectors_i32.rs
+++ /dev/null
@@ -1,25 +0,0 @@
-#![allow(non_camel_case_types)]
-
-/// A SIMD vector of containing `LANES` `i32` values.
-#[repr(simd)]
-pub struct SimdI32<const LANES: usize>([i32; LANES])
-where
-    Self: crate::LanesAtMost64;
-
-impl_integer_vector! { SimdI32, i32 }
-
-/// Vector of two `i32` values
-pub type i32x2 = SimdI32<2>;
-
-/// Vector of four `i32` values
-pub type i32x4 = SimdI32<4>;
-
-/// Vector of eight `i32` values
-pub type i32x8 = SimdI32<8>;
-
-/// Vector of 16 `i32` values
-pub type i32x16 = SimdI32<16>;
-
-from_transmute_x86! { unsafe i32x4 => __m128i }
-from_transmute_x86! { unsafe i32x8 => __m256i }
-//from_transmute_x86! { unsafe i32x16 => __m512i }
diff --git a/crates/core_simd/src/vector/vectors_i64.rs b/crates/core_simd/src/vector/vectors_i64.rs
deleted file mode 100644
index 017140654a51..000000000000
--- a/crates/core_simd/src/vector/vectors_i64.rs
+++ /dev/null
@@ -1,22 +0,0 @@
-#![allow(non_camel_case_types)]
-
-/// A SIMD vector of containing `LANES` `i64` values.
-#[repr(simd)]
-pub struct SimdI64<const LANES: usize>([i64; LANES])
-where
-    Self: crate::LanesAtMost64;
-
-impl_integer_vector! { SimdI64, i64 }
-
-/// Vector of two `i64` values
-pub type i64x2 = SimdI64<2>;
-
-/// Vector of four `i64` values
-pub type i64x4 = SimdI64<4>;
-
-/// Vector of eight `i64` values
-pub type i64x8 = SimdI64<8>;
-
-from_transmute_x86! { unsafe i64x2 => __m128i }
-from_transmute_x86! { unsafe i64x4 => __m256i }
-//from_transmute_x86! { unsafe i64x8 => __m512i }
diff --git a/crates/core_simd/src/vector/vectors_i8.rs b/crates/core_simd/src/vector/vectors_i8.rs
deleted file mode 100644
index e21126533b88..000000000000
--- a/crates/core_simd/src/vector/vectors_i8.rs
+++ /dev/null
@@ -1,25 +0,0 @@
-#![allow(non_camel_case_types)]
-
-/// A SIMD vector of containing `LANES` `i8` values.
-#[repr(simd)]
-pub struct SimdI8<const LANES: usize>([i8; LANES])
-where
-    Self: crate::LanesAtMost64;
-
-impl_integer_vector! { SimdI8, i8 }
-
-/// Vector of eight `i8` values
-pub type i8x8 = SimdI8<8>;
-
-/// Vector of 16 `i8` values
-pub type i8x16 = SimdI8<16>;
-
-/// Vector of 32 `i8` values
-pub type i8x32 = SimdI8<32>;
-
-/// Vector of 64 `i8` values
-pub type i8x64 = SimdI8<64>;
-
-from_transmute_x86! { unsafe i8x16 => __m128i }
-from_transmute_x86! { unsafe i8x32 => __m256i }
-//from_transmute_x86! { unsafe i8x64 => __m512i }
diff --git a/crates/core_simd/src/vector/vectors_u128.rs b/crates/core_simd/src/vector/vectors_u128.rs
deleted file mode 100644
index 7931b9e088f6..000000000000
--- a/crates/core_simd/src/vector/vectors_u128.rs
+++ /dev/null
@@ -1,18 +0,0 @@
-#![allow(non_camel_case_types)]
-
-/// A SIMD vector of containing `LANES` `u128` values.
-#[repr(simd)]
-pub struct SimdU128<const LANES: usize>([u128; LANES])
-where
-    Self: crate::LanesAtMost64;
-
-impl_integer_vector! { SimdU128, u128 }
-
-/// Vector of two `u128` values
-pub type u128x2 = SimdU128<2>;
-
-/// Vector of four `u128` values
-pub type u128x4 = SimdU128<4>;
-
-from_transmute_x86! { unsafe u128x2 => __m256i }
-//from_transmute_x86! { unsafe u128x4 => __m512i }
diff --git a/crates/core_simd/src/vector/vectors_u16.rs b/crates/core_simd/src/vector/vectors_u16.rs
deleted file mode 100644
index 91c0e6168089..000000000000
--- a/crates/core_simd/src/vector/vectors_u16.rs
+++ /dev/null
@@ -1,25 +0,0 @@
-#![allow(non_camel_case_types)]
-
-/// A SIMD vector of containing `LANES` `u16` values.
-#[repr(simd)]
-pub struct SimdU16<const LANES: usize>([u16; LANES])
-where
-    Self: crate::LanesAtMost64;
-
-impl_integer_vector! { SimdU16, u16 }
-
-/// Vector of four `u16` values
-pub type u16x4 = SimdU16<4>;
-
-/// Vector of eight `u16` values
-pub type u16x8 = SimdU16<8>;
-
-/// Vector of 16 `u16` values
-pub type u16x16 = SimdU16<16>;
-
-/// Vector of 32 `u16` values
-pub type u16x32 = SimdU16<32>;
-
-from_transmute_x86! { unsafe u16x8 => __m128i }
-from_transmute_x86! { unsafe u16x16 => __m256i }
-//from_transmute_x86! { unsafe u16x32 => __m512i }
diff --git a/crates/core_simd/src/vector/vectors_u32.rs b/crates/core_simd/src/vector/vectors_u32.rs
deleted file mode 100644
index b0400b5ba3a9..000000000000
--- a/crates/core_simd/src/vector/vectors_u32.rs
+++ /dev/null
@@ -1,25 +0,0 @@
-#![allow(non_camel_case_types)]
-
-/// A SIMD vector of containing `LANES` `u32` values.
-#[repr(simd)]
-pub struct SimdU32<const LANES: usize>([u32; LANES])
-where
-    Self: crate::LanesAtMost64;
-
-impl_integer_vector! { SimdU32, u32 }
-
-/// Vector of two `u32` values
-pub type u32x2 = SimdU32<2>;
-
-/// Vector of four `u32` values
-pub type u32x4 = SimdU32<4>;
-
-/// Vector of eight `u32` values
-pub type u32x8 = SimdU32<8>;
-
-/// Vector of 16 `u32` values
-pub type u32x16 = SimdU32<16>;
-
-from_transmute_x86! { unsafe u32x4 => __m128i }
-from_transmute_x86! { unsafe u32x8 => __m256i }
-//from_transmute_x86! { unsafe u32x16 => __m512i }
diff --git a/crates/core_simd/src/vector/vectors_u64.rs b/crates/core_simd/src/vector/vectors_u64.rs
deleted file mode 100644
index 0f3712241fe7..000000000000
--- a/crates/core_simd/src/vector/vectors_u64.rs
+++ /dev/null
@@ -1,22 +0,0 @@
-#![allow(non_camel_case_types)]
-
-/// A SIMD vector of containing `LANES` `u64` values.
-#[repr(simd)]
-pub struct SimdU64<const LANES: usize>([u64; LANES])
-where
-    Self: crate::LanesAtMost64;
-
-impl_integer_vector! { SimdU64, u64 }
-
-/// Vector of two `u64` values
-pub type u64x2 = SimdU64<2>;
-
-/// Vector of four `u64` values
-pub type u64x4 = SimdU64<4>;
-
-/// Vector of eight `u64` values
-pub type u64x8 = SimdU64<8>;
-
-from_transmute_x86! { unsafe u64x2 => __m128i }
-from_transmute_x86! { unsafe u64x4 => __m256i }
-//from_transmute_x86! { unsafe u64x8 => __m512i }
diff --git a/crates/core_simd/src/vector/vectors_u8.rs b/crates/core_simd/src/vector/vectors_u8.rs
deleted file mode 100644
index 6cf623f68013..000000000000
--- a/crates/core_simd/src/vector/vectors_u8.rs
+++ /dev/null
@@ -1,23 +0,0 @@
-#![allow(non_camel_case_types)]
-
-/// A SIMD vector of containing `LANES` `u8` values.
-#[repr(simd)]
-pub struct SimdU8<const LANES: usize>([u8; LANES]) where Self: crate::LanesAtMost64;
-
-impl_integer_vector! { SimdU8, u8 }
-
-/// Vector of eight `u8` values
-pub type u8x8 = SimdU8<8>;
-
-/// Vector of 16 `u8` values
-pub type u8x16 = SimdU8<16>;
-
-/// Vector of 32 `u8` values
-pub type u8x32 = SimdU8<32>;
-
-/// Vector of 64 `u8` values
-pub type u8x64 = SimdU8<64>;
-
-from_transmute_x86! { unsafe u8x16 => __m128i }
-from_transmute_x86! { unsafe u8x32 => __m256i }
-//from_transmute_x86! { unsafe u8x64 => __m512i }

From 39fb2233855ff54f3c0479459328cecf1b6185f2 Mon Sep 17 00:00:00 2001
From: Jubilee Young <workingjubilee@gmail.com>
Date: Sun, 21 Feb 2021 14:38:35 -0800
Subject: [PATCH 101/249] Partially carve macros.rs into other files

The base impl_vector! in macros.rs is staying put for the moment as
it must go first before everything in order to work.

Everything else, like transmutes, specific type impls, etc. have
been moved into appropriate files elsewhere to subdivide concerns.
---
 crates/core_simd/src/lib.rs          |  2 +
 crates/core_simd/src/macros.rs       | 94 ----------------------------
 crates/core_simd/src/transmute.rs    | 27 ++++++++
 crates/core_simd/src/vector/float.rs | 40 ++++++++++++
 crates/core_simd/src/vector/int.rs   | 27 ++++++++
 crates/core_simd/src/vector/uint.rs  | 40 ++++++++++--
 6 files changed, 130 insertions(+), 100 deletions(-)
 create mode 100644 crates/core_simd/src/transmute.rs

diff --git a/crates/core_simd/src/lib.rs b/crates/core_simd/src/lib.rs
index 6c499a197b14..5ed142197f8c 100644
--- a/crates/core_simd/src/lib.rs
+++ b/crates/core_simd/src/lib.rs
@@ -8,6 +8,8 @@
 mod macros;
 #[macro_use]
 mod permute;
+#[macro_use]
+mod transmute;
 
 mod fmt;
 mod intrinsics;
diff --git a/crates/core_simd/src/macros.rs b/crates/core_simd/src/macros.rs
index d78403ddb93d..b18fe5213a3d 100644
--- a/crates/core_simd/src/macros.rs
+++ b/crates/core_simd/src/macros.rs
@@ -1,31 +1,3 @@
-/// Provides implementations of `From<$a> for $b` and `From<$b> for $a` that transmutes the value.
-macro_rules! from_transmute {
-    { unsafe $a:ty => $b:ty } => {
-        from_transmute!{ @impl $a => $b }
-        from_transmute!{ @impl $b => $a }
-    };
-    { @impl $from:ty => $to:ty } => {
-        impl core::convert::From<$from> for $to {
-            #[inline]
-            fn from(value: $from) -> $to {
-                unsafe { core::mem::transmute(value) }
-            }
-        }
-    };
-}
-
-/// Provides implementations of `From<$generic> for core::arch::{x86, x86_64}::$intel` and
-/// vice-versa that transmutes the value.
-macro_rules! from_transmute_x86 {
-    { unsafe $generic:ty => $intel:ident } => {
-        #[cfg(target_arch = "x86")]
-        from_transmute! { unsafe $generic => core::arch::x86::$intel }
-
-        #[cfg(target_arch = "x86_64")]
-        from_transmute! { unsafe $generic => core::arch::x86_64::$intel }
-    }
-}
-
 /// Implements common traits on the specified vector `$name`, holding multiple `$lanes` of `$type`.
 macro_rules! impl_vector {
     { $name:ident, $type:ty } => {
@@ -150,69 +122,3 @@ macro_rules! impl_vector {
         impl_shuffle_2pow_lanes!{ $name }
     }
 }
-
-/// Implements additional integer traits (Eq, Ord, Hash) on the specified vector `$name`, holding multiple `$lanes` of `$type`.
-macro_rules! impl_integer_vector {
-    { $name:ident, $type:ty } => {
-        impl_vector! { $name, $type }
-
-        impl<const LANES: usize> Eq for $name<LANES> where Self: crate::LanesAtMost64 {}
-
-        impl<const LANES: usize> Ord for $name<LANES> where Self: crate::LanesAtMost64 {
-            #[inline]
-            fn cmp(&self, other: &Self) -> core::cmp::Ordering {
-                // TODO use SIMD cmp
-                self.to_array().cmp(other.as_ref())
-            }
-        }
-
-        impl<const LANES: usize> core::hash::Hash for $name<LANES> where Self: crate::LanesAtMost64 {
-            #[inline]
-            fn hash<H>(&self, state: &mut H)
-            where
-                H: core::hash::Hasher
-            {
-                self.as_slice().hash(state)
-            }
-        }
-    }
-}
-
-/// Implements inherent methods for a float vector `$name` containing multiple
-/// `$lanes` of float `$type`, which uses `$bits_ty` as its binary
-/// representation. Called from `define_float_vector!`.
-macro_rules! impl_float_vector {
-    { $name:ident, $type:ty, $bits_ty:ident } => {
-        impl_vector! { $name, $type }
-
-        impl<const LANES: usize> $name<LANES>
-        where
-            Self: crate::LanesAtMost64,
-            crate::$bits_ty<LANES>: crate::LanesAtMost64,
-        {
-            /// Raw transmutation to an unsigned integer vector type with the
-            /// same size and number of lanes.
-            #[inline]
-            pub fn to_bits(self) -> crate::$bits_ty<LANES> {
-                assert_eq!(core::mem::size_of::<Self>(), core::mem::size_of::<crate::$bits_ty<LANES>>());
-                unsafe { core::mem::transmute_copy(&self) }
-            }
-
-            /// Raw transmutation from an unsigned integer vector type with the
-            /// same size and number of lanes.
-            #[inline]
-            pub fn from_bits(bits: crate::$bits_ty<LANES>) -> Self {
-                assert_eq!(core::mem::size_of::<Self>(), core::mem::size_of::<crate::$bits_ty<LANES>>());
-                unsafe { core::mem::transmute_copy(&bits) }
-            }
-
-            /// Produces a vector where every lane has the absolute value of the
-            /// equivalently-indexed lane in `self`.
-            #[inline]
-            pub fn abs(self) -> Self {
-                let no_sign = crate::$bits_ty::splat(!0 >> 1);
-                Self::from_bits(self.to_bits() & no_sign)
-            }
-        }
-    };
-}
diff --git a/crates/core_simd/src/transmute.rs b/crates/core_simd/src/transmute.rs
new file mode 100644
index 000000000000..835d863029c5
--- /dev/null
+++ b/crates/core_simd/src/transmute.rs
@@ -0,0 +1,27 @@
+/// Provides implementations of `From<$a> for $b` and `From<$b> for $a` that transmutes the value.
+macro_rules! from_transmute {
+    { unsafe $a:ty => $b:ty } => {
+        from_transmute!{ @impl $a => $b }
+        from_transmute!{ @impl $b => $a }
+    };
+    { @impl $from:ty => $to:ty } => {
+        impl core::convert::From<$from> for $to {
+            #[inline]
+            fn from(value: $from) -> $to {
+                unsafe { core::mem::transmute(value) }
+            }
+        }
+    };
+}
+
+/// Provides implementations of `From<$generic> for core::arch::{x86, x86_64}::$intel` and
+/// vice-versa that transmutes the value.
+macro_rules! from_transmute_x86 {
+    { unsafe $generic:ty => $intel:ident } => {
+        #[cfg(target_arch = "x86")]
+        from_transmute! { unsafe $generic => core::arch::x86::$intel }
+
+        #[cfg(target_arch = "x86_64")]
+        from_transmute! { unsafe $generic => core::arch::x86_64::$intel }
+    }
+}
diff --git a/crates/core_simd/src/vector/float.rs b/crates/core_simd/src/vector/float.rs
index 56f43d0a18cf..9031e12b604f 100644
--- a/crates/core_simd/src/vector/float.rs
+++ b/crates/core_simd/src/vector/float.rs
@@ -1,5 +1,45 @@
 #![allow(non_camel_case_types)]
 
+/// Implements inherent methods for a float vector `$name` containing multiple
+/// `$lanes` of float `$type`, which uses `$bits_ty` as its binary
+/// representation. Called from `define_float_vector!`.
+macro_rules! impl_float_vector {
+    { $name:ident, $type:ty, $bits_ty:ident } => {
+        impl_vector! { $name, $type }
+
+        impl<const LANES: usize> $name<LANES>
+        where
+            Self: crate::LanesAtMost64,
+            crate::$bits_ty<LANES>: crate::LanesAtMost64,
+        {
+            /// Raw transmutation to an unsigned integer vector type with the
+            /// same size and number of lanes.
+            #[inline]
+            pub fn to_bits(self) -> crate::$bits_ty<LANES> {
+                assert_eq!(core::mem::size_of::<Self>(), core::mem::size_of::<crate::$bits_ty<LANES>>());
+                unsafe { core::mem::transmute_copy(&self) }
+            }
+
+            /// Raw transmutation from an unsigned integer vector type with the
+            /// same size and number of lanes.
+            #[inline]
+            pub fn from_bits(bits: crate::$bits_ty<LANES>) -> Self {
+                assert_eq!(core::mem::size_of::<Self>(), core::mem::size_of::<crate::$bits_ty<LANES>>());
+                unsafe { core::mem::transmute_copy(&bits) }
+            }
+
+            /// Produces a vector where every lane has the absolute value of the
+            /// equivalently-indexed lane in `self`.
+            #[inline]
+            pub fn abs(self) -> Self {
+                let no_sign = crate::$bits_ty::splat(!0 >> 1);
+                Self::from_bits(self.to_bits() & no_sign)
+            }
+        }
+    };
+}
+
+
 /// A SIMD vector of containing `LANES` `f32` values.
 #[repr(simd)]
 pub struct SimdF32<const LANES: usize>([f32; LANES])
diff --git a/crates/core_simd/src/vector/int.rs b/crates/core_simd/src/vector/int.rs
index 100e1d5afac6..86762f74ff47 100644
--- a/crates/core_simd/src/vector/int.rs
+++ b/crates/core_simd/src/vector/int.rs
@@ -1,5 +1,32 @@
 #![allow(non_camel_case_types)]
 
+/// Implements additional integer traits (Eq, Ord, Hash) on the specified vector `$name`, holding multiple `$lanes` of `$type`.
+macro_rules! impl_integer_vector {
+    { $name:ident, $type:ty } => {
+        impl_vector! { $name, $type }
+
+        impl<const LANES: usize> Eq for $name<LANES> where Self: crate::LanesAtMost64 {}
+
+        impl<const LANES: usize> Ord for $name<LANES> where Self: crate::LanesAtMost64 {
+            #[inline]
+            fn cmp(&self, other: &Self) -> core::cmp::Ordering {
+                // TODO use SIMD cmp
+                self.to_array().cmp(other.as_ref())
+            }
+        }
+
+        impl<const LANES: usize> core::hash::Hash for $name<LANES> where Self: crate::LanesAtMost64 {
+            #[inline]
+            fn hash<H>(&self, state: &mut H)
+            where
+                H: core::hash::Hasher
+            {
+                self.as_slice().hash(state)
+            }
+        }
+    }
+}
+
 /// A SIMD vector of containing `LANES` `isize` values.
 #[repr(simd)]
 pub struct SimdIsize<const LANES: usize>([isize; LANES])
diff --git a/crates/core_simd/src/vector/uint.rs b/crates/core_simd/src/vector/uint.rs
index 6a16f06e50f2..0f7a47eee30e 100644
--- a/crates/core_simd/src/vector/uint.rs
+++ b/crates/core_simd/src/vector/uint.rs
@@ -1,12 +1,40 @@
 #![allow(non_camel_case_types)]
 
+
+/// Implements additional integer traits (Eq, Ord, Hash) on the specified vector `$name`, holding multiple `$lanes` of `$type`.
+macro_rules! impl_unsigned_vector {
+    { $name:ident, $type:ty } => {
+        impl_vector! { $name, $type }
+
+        impl<const LANES: usize> Eq for $name<LANES> where Self: crate::LanesAtMost64 {}
+
+        impl<const LANES: usize> Ord for $name<LANES> where Self: crate::LanesAtMost64 {
+            #[inline]
+            fn cmp(&self, other: &Self) -> core::cmp::Ordering {
+                // TODO use SIMD cmp
+                self.to_array().cmp(other.as_ref())
+            }
+        }
+
+        impl<const LANES: usize> core::hash::Hash for $name<LANES> where Self: crate::LanesAtMost64 {
+            #[inline]
+            fn hash<H>(&self, state: &mut H)
+            where
+                H: core::hash::Hasher
+            {
+                self.as_slice().hash(state)
+            }
+        }
+    }
+}
+
 /// A SIMD vector of containing `LANES` `usize` values.
 #[repr(simd)]
 pub struct SimdUsize<const LANES: usize>([usize; LANES])
 where
     Self: crate::LanesAtMost64;
 
-impl_integer_vector! { SimdUsize, usize }
+impl_unsigned_vector! { SimdUsize, usize }
 
 #[cfg(target_pointer_width = "32")]
 from_transmute_x86! { unsafe usizex4 => __m128i }
@@ -26,7 +54,7 @@ pub struct SimdU128<const LANES: usize>([u128; LANES])
 where
     Self: crate::LanesAtMost64;
 
-impl_integer_vector! { SimdU128, u128 }
+impl_unsigned_vector! { SimdU128, u128 }
 
 from_transmute_x86! { unsafe u128x2 => __m256i }
 //from_transmute_x86! { unsafe u128x4 => __m512i }
@@ -37,7 +65,7 @@ pub struct SimdU16<const LANES: usize>([u16; LANES])
 where
     Self: crate::LanesAtMost64;
 
-impl_integer_vector! { SimdU16, u16 }
+impl_unsigned_vector! { SimdU16, u16 }
 
 from_transmute_x86! { unsafe u16x8 => __m128i }
 from_transmute_x86! { unsafe u16x16 => __m256i }
@@ -49,7 +77,7 @@ pub struct SimdU32<const LANES: usize>([u32; LANES])
 where
     Self: crate::LanesAtMost64;
 
-impl_integer_vector! { SimdU32, u32 }
+impl_unsigned_vector! { SimdU32, u32 }
 
 from_transmute_x86! { unsafe u32x4 => __m128i }
 from_transmute_x86! { unsafe u32x8 => __m256i }
@@ -61,7 +89,7 @@ pub struct SimdU64<const LANES: usize>([u64; LANES])
 where
     Self: crate::LanesAtMost64;
 
-impl_integer_vector! { SimdU64, u64 }
+impl_unsigned_vector! { SimdU64, u64 }
 
 from_transmute_x86! { unsafe u64x2 => __m128i }
 from_transmute_x86! { unsafe u64x4 => __m256i }
@@ -73,7 +101,7 @@ pub struct SimdU8<const LANES: usize>([u8; LANES])
 where
     Self: crate::LanesAtMost64;
 
-impl_integer_vector! { SimdU8, u8 }
+impl_unsigned_vector! { SimdU8, u8 }
 
 from_transmute_x86! { unsafe u8x16 => __m128i }
 from_transmute_x86! { unsafe u8x32 => __m256i }

From a2302da5b26fd5456c676a57c702c260cd977f95 Mon Sep 17 00:00:00 2001
From: Jubilee Young <workingjubilee@gmail.com>
Date: Sun, 21 Feb 2021 16:23:19 -0800
Subject: [PATCH 102/249] Move macros.rs to first.rs

This awkwardly has to go first right now, and we don't want more
macros to go into this ambiguously named mod, so let's rename it
to be more literal.
---
 crates/core_simd/src/{macros.rs => first.rs} | 0
 crates/core_simd/src/lib.rs                  | 2 +-
 2 files changed, 1 insertion(+), 1 deletion(-)
 rename crates/core_simd/src/{macros.rs => first.rs} (100%)

diff --git a/crates/core_simd/src/macros.rs b/crates/core_simd/src/first.rs
similarity index 100%
rename from crates/core_simd/src/macros.rs
rename to crates/core_simd/src/first.rs
diff --git a/crates/core_simd/src/lib.rs b/crates/core_simd/src/lib.rs
index 5ed142197f8c..489996ae15ed 100644
--- a/crates/core_simd/src/lib.rs
+++ b/crates/core_simd/src/lib.rs
@@ -5,7 +5,7 @@
 //! Portable SIMD module.
 
 #[macro_use]
-mod macros;
+mod first;
 #[macro_use]
 mod permute;
 #[macro_use]

From 8cb1fe0c0e88811c12cce245b5437e6b4b003702 Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Sat, 6 Mar 2021 22:00:39 -0500
Subject: [PATCH 103/249] Fix wasm-bindgen dependency

---
 crates/core_simd/Cargo.toml | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/crates/core_simd/Cargo.toml b/crates/core_simd/Cargo.toml
index d76bd547cde6..26cc65a86876 100644
--- a/crates/core_simd/Cargo.toml
+++ b/crates/core_simd/Cargo.toml
@@ -9,7 +9,7 @@ keywords = ["core", "simd", "intrinsics"]
 categories = ["hardware-support", "no-std"]
 license = "MIT OR Apache-2.0"
 
-[target.'cfg(target_arch = "wasm32")'.dependencies.wasm-bindgen]
+[target.'cfg(target_arch = "wasm32")'.dev-dependencies.wasm-bindgen]
 version = "0.2"
 
 [dev-dependencies.wasm-bindgen-test]

From fa77b196c8f4cca27009c334f09466b80aca2f8a Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Sat, 6 Mar 2021 21:56:01 -0500
Subject: [PATCH 104/249] Add std cargo feature

---
 crates/core_simd/Cargo.toml          |  4 ++++
 crates/core_simd/src/intrinsics.rs   |  2 ++
 crates/core_simd/src/round.rs        |  2 ++
 crates/core_simd/tests/ops_macros.rs | 19 +++++++++++--------
 4 files changed, 19 insertions(+), 8 deletions(-)

diff --git a/crates/core_simd/Cargo.toml b/crates/core_simd/Cargo.toml
index 26cc65a86876..6044eabcd140 100644
--- a/crates/core_simd/Cargo.toml
+++ b/crates/core_simd/Cargo.toml
@@ -9,6 +9,10 @@ keywords = ["core", "simd", "intrinsics"]
 categories = ["hardware-support", "no-std"]
 license = "MIT OR Apache-2.0"
 
+[features]
+default = ["std"]
+std = []
+
 [target.'cfg(target_arch = "wasm32")'.dev-dependencies.wasm-bindgen]
 version = "0.2"
 
diff --git a/crates/core_simd/src/intrinsics.rs b/crates/core_simd/src/intrinsics.rs
index 4ea383763487..213ebff3df45 100644
--- a/crates/core_simd/src/intrinsics.rs
+++ b/crates/core_simd/src/intrinsics.rs
@@ -41,9 +41,11 @@ extern "platform-intrinsic" {
     pub(crate) fn simd_cast<T, U>(x: T) -> U;
 
     // floor
+    #[cfg(feature = "std")]
     pub(crate) fn simd_floor<T>(x: T) -> T;
 
     // ceil
+    #[cfg(feature = "std")]
     pub(crate) fn simd_ceil<T>(x: T) -> T;
 
     pub(crate) fn simd_eq<T, U>(x: T, y: T) -> U;
diff --git a/crates/core_simd/src/round.rs b/crates/core_simd/src/round.rs
index ee232e2b2225..dc37130a8ce6 100644
--- a/crates/core_simd/src/round.rs
+++ b/crates/core_simd/src/round.rs
@@ -7,6 +7,7 @@ macro_rules! implement {
             Self: crate::LanesAtMost64,
         {
             /// Returns the largest integer less than or equal to each lane.
+            #[cfg(feature = "std")]
             #[must_use = "method returns a new vector and does not mutate the original value"]
             #[inline]
             pub fn floor(self) -> Self {
@@ -14,6 +15,7 @@ macro_rules! implement {
             }
 
             /// Returns the smallest integer greater than or equal to each lane.
+            #[cfg(feature = "std")]
             #[must_use = "method returns a new vector and does not mutate the original value"]
             #[inline]
             pub fn ceil(self) -> Self {
diff --git a/crates/core_simd/tests/ops_macros.rs b/crates/core_simd/tests/ops_macros.rs
index 58e80a8f277e..8e0b9626861f 100644
--- a/crates/core_simd/tests/ops_macros.rs
+++ b/crates/core_simd/tests/ops_macros.rs
@@ -265,15 +265,8 @@ macro_rules! impl_float_tests {
             impl_binary_op_test!(Vector<LANES>, Scalar, Div::div, DivAssign::div_assign);
             impl_binary_op_test!(Vector<LANES>, Scalar, Rem::rem, RemAssign::rem_assign);
 
+            #[cfg(feature = "std")]
             test_helpers::test_lanes! {
-                fn abs<const LANES: usize>() {
-                    test_helpers::test_unary_elementwise(
-                        &Vector::<LANES>::abs,
-                        &Scalar::abs,
-                        &|_| true,
-                    )
-                }
-
                 fn ceil<const LANES: usize>() {
                     test_helpers::test_unary_elementwise(
                         &Vector::<LANES>::ceil,
@@ -289,6 +282,16 @@ macro_rules! impl_float_tests {
                         &|_| true,
                     )
                 }
+            }
+
+            test_helpers::test_lanes! {
+                fn abs<const LANES: usize>() {
+                    test_helpers::test_unary_elementwise(
+                        &Vector::<LANES>::abs,
+                        &Scalar::abs,
+                        &|_| true,
+                    )
+                }
 
                 fn round_from_int<const LANES: usize>() {
                     test_helpers::test_unary_elementwise(

From 4a6b4c0a2e245e3484196f2533600c29a8b8ae1a Mon Sep 17 00:00:00 2001
From: Jubilee Young <workingjubilee@gmail.com>
Date: Sun, 28 Mar 2021 18:44:16 -0700
Subject: [PATCH 105/249] Introduce saturating math

---
 crates/core_simd/src/intrinsics.rs |  6 ++
 crates/core_simd/src/lib.rs        |  3 +
 crates/core_simd/src/math.rs       | 88 ++++++++++++++++++++++++++++++
 3 files changed, 97 insertions(+)
 create mode 100644 crates/core_simd/src/math.rs

diff --git a/crates/core_simd/src/intrinsics.rs b/crates/core_simd/src/intrinsics.rs
index 213ebff3df45..93c97cfed8ec 100644
--- a/crates/core_simd/src/intrinsics.rs
+++ b/crates/core_simd/src/intrinsics.rs
@@ -62,4 +62,10 @@ extern "platform-intrinsic" {
     pub(crate) fn simd_shuffle16<T, U>(x: T, y: T, idx: [u32; 16]) -> U;
     pub(crate) fn simd_shuffle32<T, U>(x: T, y: T, idx: [u32; 32]) -> U;
     pub(crate) fn simd_shuffle64<T, U>(x: T, y: T, idx: [u32; 64]) -> U;
+
+    // {s,u}add.sat
+    pub(crate) fn simd_saturating_add<T>(x: T, y: T) -> T;
+
+    // {s,u}sub.sat
+    pub(crate) fn simd_saturating_sub<T>(x: T, y: T) -> T;
 }
diff --git a/crates/core_simd/src/lib.rs b/crates/core_simd/src/lib.rs
index 489996ae15ed..8ff08223598a 100644
--- a/crates/core_simd/src/lib.rs
+++ b/crates/core_simd/src/lib.rs
@@ -1,6 +1,7 @@
 #![no_std]
 #![allow(incomplete_features)]
 #![feature(repr_simd, platform_intrinsics, simd_ffi, const_generics)]
+#![feature(extended_key_value_attributes)]
 #![warn(missing_docs)]
 //! Portable SIMD module.
 
@@ -16,6 +17,8 @@ mod intrinsics;
 mod ops;
 mod round;
 
+mod math;
+
 mod lanes_at_most_64;
 pub use lanes_at_most_64::LanesAtMost64;
 
diff --git a/crates/core_simd/src/math.rs b/crates/core_simd/src/math.rs
new file mode 100644
index 000000000000..6fabf35e3da3
--- /dev/null
+++ b/crates/core_simd/src/math.rs
@@ -0,0 +1,88 @@
+macro_rules! impl_uint_arith {
+    ($(($name:ident, $n:ty)),+) => {
+        $( impl<const LANES: usize> $name<LANES> where Self: crate::LanesAtMost64 {
+
+            /// Lanewise saturating add.
+            ///
+            /// # Examples
+            /// ```
+            /// # use core_simd::*;
+            #[doc = concat!("# use core::", stringify!($n), "::MAX;")]
+            #[doc = concat!("let x = ", stringify!($name), "::from_array([2, 1, 0, MAX]);")]
+            #[doc = concat!("let max = ", stringify!($name), "::splat(MAX);")]
+            /// let unsat = x + max;
+            /// let sat = x.saturating_add(max);
+            /// assert_eq!(x - 1, unsat);
+            /// assert_eq!(sat, max);
+            /// ```
+            #[inline]
+            pub fn saturating_add(self, second: Self) -> Self {
+                unsafe { crate::intrinsics::simd_saturating_add(self, second) }
+            }
+
+            /// Lanewise saturating subtract.
+            ///
+            /// # Examples
+            /// ```
+            /// # use core_simd::*;
+            #[doc = concat!("# use core::", stringify!($n), "::MAX;")]
+            #[doc = concat!("let x = ", stringify!($name), "::from_array([2, 1, 0, MAX]);")]
+            #[doc = concat!("let max = ", stringify!($name), "::splat(MAX);")]
+            /// let unsat = x - max;
+            /// let sat = x.saturating_sub(max);
+            /// assert_eq!(unsat, x + 1);
+            #[doc = concat!("assert_eq!(sat, ", stringify!($name), "::splat(0));")]
+            #[inline]
+            pub fn saturating_sub(self, second: Self) -> Self {
+                unsafe { crate::intrinsics::simd_saturating_sub(self, second) }
+            }
+        })+
+    }
+}
+
+macro_rules! impl_int_arith {
+    ($(($name:ident, $n:ty)),+) => {
+        $( impl<const LANES: usize> $name<LANES> where Self: crate::LanesAtMost64 {
+
+            /// Lanewise saturating add.
+            ///
+            /// # Examples
+            /// ```
+            /// # use core_simd::*;
+            #[doc = concat!("# use core::", stringify!($n), "::{MIN, MAX};")]
+            #[doc = concat!("let x = ", stringify!($name), "::from_array([MIN, 0, 1, MAX]);")]
+            #[doc = concat!("let max = ", stringify!($name), "::splat(MAX);")]
+            /// let unsat = x + max;
+            /// let sat = x.saturating_add(max);
+            #[doc = concat!("assert_eq!(unsat, ", stringify!($name), "::from_array([-1, MAX, MIN, -2]));")]
+            #[doc = concat!("assert_eq!(sat, ", stringify!($name), "::from_array([-1, MAX, MAX, MAX]));")]
+            /// ```
+            #[inline]
+            pub fn saturating_add(self, second: Self) -> Self {
+                unsafe { crate::intrinsics::simd_saturating_add(self, second) }
+            }
+
+            /// Lanewise saturating subtract.
+            ///
+            /// # Examples
+            /// ```
+            /// # use core_simd::*;
+            #[doc = concat!("# use core::", stringify!($n), "::{MIN, MAX};")]
+            #[doc = concat!("let x = ", stringify!($name), "::from_array([MIN, -2, -1, MAX]);")]
+            #[doc = concat!("let max = ", stringify!($name), "::splat(MAX);")]
+            /// let unsat = x - max;
+            /// let sat = x.saturating_sub(max);
+            #[doc = concat!("assert_eq!(unsat, ", stringify!($name), "::from_array([1, MAX, MIN, 0]));")]
+            #[doc = concat!("assert_eq!(sat, ", stringify!($name), "::from_array([MIN, MIN, MIN, 0]));")]
+            #[inline]
+            pub fn saturating_sub(self, second: Self) -> Self {
+                unsafe { crate::intrinsics::simd_saturating_sub(self, second) }
+            }
+        })+
+    }
+}
+
+use crate::vector::*;
+
+impl_uint_arith! { (SimdU8, u8), (SimdU16, u16), (SimdU32, u32), (SimdU64, u64), (SimdUsize, usize) }
+impl_int_arith! { (SimdI8, i8), (SimdI16, i16), (SimdI32, i32), (SimdI64, i64), (SimdIsize, isize) }

From dd1a5e41ad0763cf8f57eba2a92528b895dc9378 Mon Sep 17 00:00:00 2001
From: Jubilee Young <workingjubilee@gmail.com>
Date: Fri, 2 Apr 2021 08:28:48 -0700
Subject: [PATCH 106/249] Add saturating abs/neg

---
 crates/core_simd/src/math.rs | 35 +++++++++++++++++++++++++++++++++++
 1 file changed, 35 insertions(+)

diff --git a/crates/core_simd/src/math.rs b/crates/core_simd/src/math.rs
index 6fabf35e3da3..0d9d62353568 100644
--- a/crates/core_simd/src/math.rs
+++ b/crates/core_simd/src/math.rs
@@ -78,6 +78,41 @@ macro_rules! impl_int_arith {
             pub fn saturating_sub(self, second: Self) -> Self {
                 unsafe { crate::intrinsics::simd_saturating_sub(self, second) }
             }
+
+            /// Lanewise saturating absolute value, implemented in Rust.
+            ///
+            /// # Examples
+            /// # use core_simd::*;
+            #[doc = concat!("# use core::", stringify!($n), "::{MIN, MAX};")]
+            #[doc = concat!("let x = ", stringify!($name), "::splat([MIN, -2, 0, 3]);")]
+            /// let unsat = x.abs();
+            /// let sat = x.saturating_abs();
+            #[doc = concat!("assert_eq!(unsat, ", stringify!($name), "::from_array([MIN, 2, 0, 3]);")]
+            #[doc = concat!("assert_eq!(sat, ", stringify!($name), "::from_array([MAX, 2, 0, 3]));")]
+            /// ```
+            #[inline]
+            pub fn saturating_abs(self) -> Self {
+                // arith shift for -1 or 0 mask based on sign bit, giving 2s complement
+                const SHR: $n = <$n>::BITS as $n - 1;
+                let m = self >> SHR;
+                (self^m).saturating_sub(m)
+            }
+
+            /// Lanewise saturating negation, implemented in Rust.
+            ///
+            /// # Examples
+            /// # use core_simd::*;
+            #[doc = concat!("# use core::", stringify!($n), "::{MIN, MAX};")]
+            #[doc = concat!("let x = ", stringify!($name), "::splat([MIN, -2, 3, MAX]);")]
+            /// let unsat = -x;
+            /// let sat = x.saturating_neg();
+            #[doc = concat!("assert_eq!(unsat, ", stringify!($name), "::from_array([MIN, 2, -3, MIN + 1]);")]
+            #[doc = concat!("assert_eq!(sat, ", stringify!($name), "::from_array([MAX, 2, -3, MIN + 1]));")]
+            /// ```
+            #[inline]
+            pub fn saturating_neg(self) -> Self {
+                Self::splat(0).saturating_sub(self)
+            }
         })+
     }
 }

From 331230fabff28d2c883967e2ac4ef02eaea4db5c Mon Sep 17 00:00:00 2001
From: Jubilee Young <workingjubilee@gmail.com>
Date: Fri, 2 Apr 2021 09:11:24 -0700
Subject: [PATCH 107/249] Explain why to use saturation

---
 crates/core_simd/src/math.rs | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/crates/core_simd/src/math.rs b/crates/core_simd/src/math.rs
index 0d9d62353568..eb46feb5c4b4 100644
--- a/crates/core_simd/src/math.rs
+++ b/crates/core_simd/src/math.rs
@@ -80,6 +80,7 @@ macro_rules! impl_int_arith {
             }
 
             /// Lanewise saturating absolute value, implemented in Rust.
+            /// As abs(), except the MIN value becomes MAX instead of itself.
             ///
             /// # Examples
             /// # use core_simd::*;
@@ -99,6 +100,7 @@ macro_rules! impl_int_arith {
             }
 
             /// Lanewise saturating negation, implemented in Rust.
+            /// As neg(), except the MIN value becomes MAX instead of itself.
             ///
             /// # Examples
             /// # use core_simd::*;

From 93ce1c1a597aba7049fe2a39737c8155dd4e2a50 Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Sun, 14 Feb 2021 23:35:24 -0500
Subject: [PATCH 108/249] Add floating-point classification functions

---
 crates/core_simd/src/comparisons.rs      | 86 ++++++++++++++++++++++++
 crates/core_simd/src/lib.rs              |  1 +
 crates/core_simd/src/masks/full_masks.rs | 19 ++++++
 crates/core_simd/src/masks/mod.rs        | 67 ------------------
 crates/core_simd/src/vector/float.rs     | 60 ++++++++++++++++-
 5 files changed, 163 insertions(+), 70 deletions(-)
 create mode 100644 crates/core_simd/src/comparisons.rs

diff --git a/crates/core_simd/src/comparisons.rs b/crates/core_simd/src/comparisons.rs
new file mode 100644
index 000000000000..eb901a89ca33
--- /dev/null
+++ b/crates/core_simd/src/comparisons.rs
@@ -0,0 +1,86 @@
+use crate::LanesAtMost64;
+
+macro_rules! implement_mask_ops {
+    { $($vector:ident => $mask:ident ($inner_mask_ty:ident, $inner_ty:ident),)* } => {
+        $(
+            impl<const LANES: usize> crate::$vector<LANES>
+            where
+                crate::$vector<LANES>: LanesAtMost64,
+                crate::$inner_ty<LANES>: LanesAtMost64,
+            {
+                /// Test if each lane is equal to the corresponding lane in `other`.
+                #[inline]
+                pub fn lanes_eq(self, other: Self) -> crate::$mask<LANES> {
+                    unsafe {
+                        crate::$inner_mask_ty::from_int_unchecked(crate::intrinsics::simd_eq(self, other))
+                            .into()
+                    }
+                }
+
+                /// Test if each lane is not equal to the corresponding lane in `other`.
+                #[inline]
+                pub fn lanes_ne(self, other: Self) -> crate::$mask<LANES> {
+                    unsafe {
+                        crate::$inner_mask_ty::from_int_unchecked(crate::intrinsics::simd_ne(self, other))
+                            .into()
+                    }
+                }
+
+                /// Test if each lane is less than the corresponding lane in `other`.
+                #[inline]
+                pub fn lanes_lt(self, other: Self) -> crate::$mask<LANES> {
+                    unsafe {
+                        crate::$inner_mask_ty::from_int_unchecked(crate::intrinsics::simd_lt(self, other))
+                            .into()
+                    }
+                }
+
+                /// Test if each lane is greater than the corresponding lane in `other`.
+                #[inline]
+                pub fn lanes_gt(self, other: Self) -> crate::$mask<LANES> {
+                    unsafe {
+                        crate::$inner_mask_ty::from_int_unchecked(crate::intrinsics::simd_gt(self, other))
+                            .into()
+                    }
+                }
+
+                /// Test if each lane is less than or equal to the corresponding lane in `other`.
+                #[inline]
+                pub fn lanes_le(self, other: Self) -> crate::$mask<LANES> {
+                    unsafe {
+                        crate::$inner_mask_ty::from_int_unchecked(crate::intrinsics::simd_le(self, other))
+                            .into()
+                    }
+                }
+
+                /// Test if each lane is greater than or equal to the corresponding lane in `other`.
+                #[inline]
+                pub fn lanes_ge(self, other: Self) -> crate::$mask<LANES> {
+                    unsafe {
+                        crate::$inner_mask_ty::from_int_unchecked(crate::intrinsics::simd_ge(self, other))
+                            .into()
+                    }
+                }
+            }
+        )*
+    }
+}
+
+implement_mask_ops! {
+    SimdI8 => Mask8 (SimdMask8, SimdI8),
+    SimdI16 => Mask16 (SimdMask16, SimdI16),
+    SimdI32 => Mask32 (SimdMask32, SimdI32),
+    SimdI64 => Mask64 (SimdMask64, SimdI64),
+    SimdI128 => Mask128 (SimdMask128, SimdI128),
+    SimdIsize => MaskSize (SimdMaskSize, SimdIsize),
+
+    SimdU8 => Mask8 (SimdMask8, SimdI8),
+    SimdU16 => Mask16 (SimdMask16, SimdI16),
+    SimdU32 => Mask32 (SimdMask32, SimdI32),
+    SimdU64 => Mask64 (SimdMask64, SimdI64),
+    SimdU128 => Mask128 (SimdMask128, SimdI128),
+    SimdUsize => MaskSize (SimdMaskSize, SimdIsize),
+
+    SimdF32 => Mask32 (SimdMask32, SimdI32),
+    SimdF64 => Mask64 (SimdMask64, SimdI64),
+}
diff --git a/crates/core_simd/src/lib.rs b/crates/core_simd/src/lib.rs
index 8ff08223598a..db3b9d0e4091 100644
--- a/crates/core_simd/src/lib.rs
+++ b/crates/core_simd/src/lib.rs
@@ -16,6 +16,7 @@ mod fmt;
 mod intrinsics;
 mod ops;
 mod round;
+mod comparisons;
 
 mod math;
 
diff --git a/crates/core_simd/src/masks/full_masks.rs b/crates/core_simd/src/masks/full_masks.rs
index fa93d252df46..d88875deacac 100644
--- a/crates/core_simd/src/masks/full_masks.rs
+++ b/crates/core_simd/src/masks/full_masks.rs
@@ -75,6 +75,25 @@ macro_rules! define_mask {
                     0
                 }
             }
+
+            /// Creates a mask from an integer vector.
+            ///
+            /// # Safety
+            /// All lanes must be either 0 or -1.
+            #[inline]
+            pub unsafe fn from_int_unchecked(value: $type) -> Self {
+                Self(value)
+            }
+
+            /// Creates a mask from an integer vector.
+            ///
+            /// # Panics
+            /// Panics if any lane is not 0 or -1.
+            #[inline]
+            pub fn from_int(value: $type) -> Self {
+                use core::convert::TryInto;
+                value.try_into().unwrap()
+            }
         }
 
         impl<const $lanes: usize> core::convert::From<bool> for $name<$lanes>
diff --git a/crates/core_simd/src/masks/mod.rs b/crates/core_simd/src/masks/mod.rs
index 7d7f7af627df..945ee9377f16 100644
--- a/crates/core_simd/src/masks/mod.rs
+++ b/crates/core_simd/src/masks/mod.rs
@@ -360,73 +360,6 @@ define_opaque_mask! {
     @bits crate::SimdIsize<LANES>
 }
 
-macro_rules! implement_mask_ops {
-    { $($vector:ident => $mask:ident ($inner_ty:ident),)* } => {
-        $(
-            impl<const LANES: usize> crate::$vector<LANES>
-            where
-                crate::$vector<LANES>: LanesAtMost64,
-                crate::$inner_ty<LANES>: LanesAtMost64,
-            {
-                /// Test if each lane is equal to the corresponding lane in `other`.
-                #[inline]
-                pub fn lanes_eq(&self, other: &Self) -> $mask<LANES> {
-                    unsafe { $mask(crate::intrinsics::simd_eq(self, other)) }
-                }
-
-                /// Test if each lane is not equal to the corresponding lane in `other`.
-                #[inline]
-                pub fn lanes_ne(&self, other: &Self) -> $mask<LANES> {
-                    unsafe { $mask(crate::intrinsics::simd_ne(self, other)) }
-                }
-
-                /// Test if each lane is less than the corresponding lane in `other`.
-                #[inline]
-                pub fn lanes_lt(&self, other: &Self) -> $mask<LANES> {
-                    unsafe { $mask(crate::intrinsics::simd_lt(self, other)) }
-                }
-
-                /// Test if each lane is greater than the corresponding lane in `other`.
-                #[inline]
-                pub fn lanes_gt(&self, other: &Self) -> $mask<LANES> {
-                    unsafe { $mask(crate::intrinsics::simd_gt(self, other)) }
-                }
-
-                /// Test if each lane is less than or equal to the corresponding lane in `other`.
-                #[inline]
-                pub fn lanes_le(&self, other: &Self) -> $mask<LANES> {
-                    unsafe { $mask(crate::intrinsics::simd_le(self, other)) }
-                }
-
-                /// Test if each lane is greater than or equal to the corresponding lane in `other`.
-                #[inline]
-                pub fn lanes_ge(&self, other: &Self) -> $mask<LANES> {
-                    unsafe { $mask(crate::intrinsics::simd_ge(self, other)) }
-                }
-            }
-        )*
-    }
-}
-
-implement_mask_ops! {
-    SimdI8 => Mask8 (SimdI8),
-    SimdI16 => Mask16 (SimdI16),
-    SimdI32 => Mask32 (SimdI32),
-    SimdI64 => Mask64 (SimdI64),
-    SimdI128 => Mask128 (SimdI128),
-    SimdIsize => MaskSize (SimdIsize),
-
-    SimdU8 => Mask8 (SimdI8),
-    SimdU16 => Mask16 (SimdI16),
-    SimdU32 => Mask32 (SimdI32),
-    SimdU64 => Mask64 (SimdI64),
-    SimdU128 => Mask128 (SimdI128),
-    SimdUsize => MaskSize (SimdIsize),
-
-    SimdF32 => Mask32 (SimdI32),
-    SimdF64 => Mask64 (SimdI64),
-}
-
 /// Vector of eight 8-bit masks
 pub type mask8x8 = Mask8<8>;
 
diff --git a/crates/core_simd/src/vector/float.rs b/crates/core_simd/src/vector/float.rs
index 9031e12b604f..9d72e69ec47c 100644
--- a/crates/core_simd/src/vector/float.rs
+++ b/crates/core_simd/src/vector/float.rs
@@ -4,7 +4,7 @@
 /// `$lanes` of float `$type`, which uses `$bits_ty` as its binary
 /// representation. Called from `define_float_vector!`.
 macro_rules! impl_float_vector {
-    { $name:ident, $type:ty, $bits_ty:ident } => {
+    { $name:ident, $type:ty, $bits_ty:ident, $mask_ty:ident, $mask_impl_ty:ident } => {
         impl_vector! { $name, $type }
 
         impl<const LANES: usize> $name<LANES>
@@ -36,6 +36,60 @@ macro_rules! impl_float_vector {
                 Self::from_bits(self.to_bits() & no_sign)
             }
         }
+
+        impl<const LANES: usize> $name<LANES>
+        where
+            Self: crate::LanesAtMost64,
+            crate::$bits_ty<LANES>: crate::LanesAtMost64,
+            crate::$mask_impl_ty<LANES>: crate::LanesAtMost64,
+        {
+            /// Returns true for each lane if it has a positive sign, including
+            /// `+0.0`, `NaN`s with positive sign bit and positive infinity.
+            #[inline]
+            pub fn is_sign_positive(self) -> crate::$mask_ty<LANES> {
+                let sign_bits = self.to_bits() & crate::$bits_ty::splat((!0 >> 1) + 1);
+                sign_bits.lanes_gt(crate::$bits_ty::splat(0))
+            }
+
+            /// Returns true for each lane if it has a negative sign, including
+            /// `-0.0`, `NaN`s with negative sign bit and negative infinity.
+            #[inline]
+            pub fn is_sign_negative(self) -> crate::$mask_ty<LANES> {
+                !self.is_sign_positive()
+            }
+
+            /// Returns true for each lane if its value is `NaN`.
+            #[inline]
+            pub fn is_nan(self) -> crate::$mask_ty<LANES> {
+                self.lanes_eq(self)
+            }
+
+            /// Returns true for each lane if its value is positive infinity or negative infinity.
+            #[inline]
+            pub fn is_infinite(self) -> crate::$mask_ty<LANES> {
+                self.abs().lanes_eq(Self::splat(<$type>::INFINITY))
+            }
+
+            /// Returns true for each lane if its value is neither infinite nor `NaN`.
+            #[inline]
+            pub fn is_finite(self) -> crate::$mask_ty<LANES> {
+                self.abs().lanes_lt(Self::splat(<$type>::INFINITY))
+            }
+
+            /// Returns true for each lane if its value is subnormal.
+            #[inline]
+            pub fn is_subnormal(self) -> crate::$mask_ty<LANES> {
+                let mantissa_mask = crate::$bits_ty::splat((1 << (<$type>::MANTISSA_DIGITS - 1)) - 1);
+                self.abs().lanes_ne(Self::splat(0.0)) & (self.to_bits() & mantissa_mask).lanes_eq(crate::$bits_ty::splat(0))
+            }
+
+            /// Returns true for each lane if its value is neither neither zero, infinite,
+            /// subnormal, or `NaN`.
+            #[inline]
+            pub fn is_normal(self) -> crate::$mask_ty<LANES> {
+                !(self.abs().lanes_eq(Self::splat(0.0)) | self.is_nan() | self.is_subnormal())
+            }
+        }
     };
 }
 
@@ -46,7 +100,7 @@ pub struct SimdF32<const LANES: usize>([f32; LANES])
 where
     Self: crate::LanesAtMost64;
 
-impl_float_vector! { SimdF32, f32, SimdU32 }
+impl_float_vector! { SimdF32, f32, SimdU32, Mask32, SimdI32 }
 
 from_transmute_x86! { unsafe f32x4 => __m128 }
 from_transmute_x86! { unsafe f32x8 => __m256 }
@@ -58,7 +112,7 @@ pub struct SimdF64<const LANES: usize>([f64; LANES])
 where
     Self: crate::LanesAtMost64;
 
-impl_float_vector! { SimdF64, f64, SimdU64 }
+impl_float_vector! { SimdF64, f64, SimdU64, Mask64, SimdI64 }
 
 from_transmute_x86! { unsafe f64x2 => __m128d }
 from_transmute_x86! { unsafe f64x4 => __m256d }

From 07247a001f43ddc35235202fc203159fbf081449 Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Sat, 6 Mar 2021 02:14:58 -0500
Subject: [PATCH 109/249] Various bug fixes

---
 crates/core_simd/src/lib.rs          |  2 +-
 crates/core_simd/src/masks/mod.rs    | 35 +++++++++++++
 crates/core_simd/src/vector/float.rs | 13 +++--
 crates/core_simd/src/vector/int.rs   | 30 ++++++++---
 crates/core_simd/tests/f32_ops.rs    |  2 +
 crates/core_simd/tests/f64_ops.rs    |  2 +
 crates/core_simd/tests/ops_macros.rs | 74 +++++++++++++++++++++++++++-
 crates/test_helpers/src/biteq.rs     | 10 ++++
 crates/test_helpers/src/lib.rs       | 30 ++++++++++-
 9 files changed, 180 insertions(+), 18 deletions(-)

diff --git a/crates/core_simd/src/lib.rs b/crates/core_simd/src/lib.rs
index db3b9d0e4091..747096306ff7 100644
--- a/crates/core_simd/src/lib.rs
+++ b/crates/core_simd/src/lib.rs
@@ -12,11 +12,11 @@ mod permute;
 #[macro_use]
 mod transmute;
 
+mod comparisons;
 mod fmt;
 mod intrinsics;
 mod ops;
 mod round;
-mod comparisons;
 
 mod math;
 
diff --git a/crates/core_simd/src/masks/mod.rs b/crates/core_simd/src/masks/mod.rs
index 945ee9377f16..d30399fb5ad1 100644
--- a/crates/core_simd/src/masks/mod.rs
+++ b/crates/core_simd/src/masks/mod.rs
@@ -28,6 +28,28 @@ macro_rules! define_opaque_mask {
                 Self(<$inner_ty>::splat(value))
             }
 
+            /// Converts an array to a SIMD vector.
+            pub fn from_array(array: [bool; LANES]) -> Self {
+                let mut vector = Self::splat(false);
+                let mut i = 0;
+                while i < $lanes {
+                    vector.set(i, array[i]);
+                    i += 1;
+                }
+                vector
+            }
+
+            /// Converts a SIMD vector to an array.
+            pub fn to_array(self) -> [bool; LANES] {
+                let mut array = [false; LANES];
+                let mut i = 0;
+                while i < $lanes {
+                    array[i] = self.test(i);
+                    i += 1;
+                }
+                array
+            }
+
             /// Tests the value of the specified lane.
             ///
             /// # Panics
@@ -85,6 +107,19 @@ macro_rules! define_opaque_mask {
             }
         }
 
+        // vector/array conversion
+        impl<const $lanes: usize> From<[bool; $lanes]> for $name<$lanes> where $bits_ty: crate::LanesAtMost64 {
+            fn from(array: [bool; $lanes]) -> Self {
+                Self::from_array(array)
+            }
+        }
+
+        impl <const $lanes: usize> From<$name<$lanes>> for [bool; $lanes] where $bits_ty: crate::LanesAtMost64 {
+            fn from(vector: $name<$lanes>) -> Self {
+                vector.to_array()
+            }
+        }
+
         impl<const $lanes: usize> Copy for $name<$lanes>
         where
             $inner_ty: Copy,
diff --git a/crates/core_simd/src/vector/float.rs b/crates/core_simd/src/vector/float.rs
index 9d72e69ec47c..85456e7bf1ec 100644
--- a/crates/core_simd/src/vector/float.rs
+++ b/crates/core_simd/src/vector/float.rs
@@ -47,21 +47,21 @@ macro_rules! impl_float_vector {
             /// `+0.0`, `NaN`s with positive sign bit and positive infinity.
             #[inline]
             pub fn is_sign_positive(self) -> crate::$mask_ty<LANES> {
-                let sign_bits = self.to_bits() & crate::$bits_ty::splat((!0 >> 1) + 1);
-                sign_bits.lanes_gt(crate::$bits_ty::splat(0))
+                !self.is_sign_negative()
             }
 
             /// Returns true for each lane if it has a negative sign, including
             /// `-0.0`, `NaN`s with negative sign bit and negative infinity.
             #[inline]
             pub fn is_sign_negative(self) -> crate::$mask_ty<LANES> {
-                !self.is_sign_positive()
+                let sign_bits = self.to_bits() & crate::$bits_ty::splat((!0 >> 1) + 1);
+                sign_bits.lanes_gt(crate::$bits_ty::splat(0))
             }
 
             /// Returns true for each lane if its value is `NaN`.
             #[inline]
             pub fn is_nan(self) -> crate::$mask_ty<LANES> {
-                self.lanes_eq(self)
+                self.lanes_ne(self)
             }
 
             /// Returns true for each lane if its value is positive infinity or negative infinity.
@@ -79,8 +79,8 @@ macro_rules! impl_float_vector {
             /// Returns true for each lane if its value is subnormal.
             #[inline]
             pub fn is_subnormal(self) -> crate::$mask_ty<LANES> {
-                let mantissa_mask = crate::$bits_ty::splat((1 << (<$type>::MANTISSA_DIGITS - 1)) - 1);
-                self.abs().lanes_ne(Self::splat(0.0)) & (self.to_bits() & mantissa_mask).lanes_eq(crate::$bits_ty::splat(0))
+                let exponent_mask = crate::$bits_ty::splat(!0 << <$type>::MANTISSA_DIGITS);
+                self.abs().lanes_ne(Self::splat(0.0)) & (self.to_bits() & exponent_mask).lanes_eq(crate::$bits_ty::splat(0))
             }
 
             /// Returns true for each lane if its value is neither neither zero, infinite,
@@ -93,7 +93,6 @@ macro_rules! impl_float_vector {
     };
 }
 
-
 /// A SIMD vector of containing `LANES` `f32` values.
 #[repr(simd)]
 pub struct SimdF32<const LANES: usize>([f32; LANES])
diff --git a/crates/core_simd/src/vector/int.rs b/crates/core_simd/src/vector/int.rs
index 86762f74ff47..364a2ed935fb 100644
--- a/crates/core_simd/src/vector/int.rs
+++ b/crates/core_simd/src/vector/int.rs
@@ -2,7 +2,7 @@
 
 /// Implements additional integer traits (Eq, Ord, Hash) on the specified vector `$name`, holding multiple `$lanes` of `$type`.
 macro_rules! impl_integer_vector {
-    { $name:ident, $type:ty } => {
+    { $name:ident, $type:ty, $mask_ty:ident, $mask_impl_ty:ident } => {
         impl_vector! { $name, $type }
 
         impl<const LANES: usize> Eq for $name<LANES> where Self: crate::LanesAtMost64 {}
@@ -24,6 +24,22 @@ macro_rules! impl_integer_vector {
                 self.as_slice().hash(state)
             }
         }
+
+        impl<const LANES: usize> $name<LANES>
+        where
+            Self: crate::LanesAtMost64,
+            crate::$mask_impl_ty<LANES>: crate::LanesAtMost64,
+        {
+            /// Returns true for each positive lane and false if it is zero or negative.
+            pub fn is_positive(self) -> crate::$mask_ty<LANES> {
+                self.lanes_gt(Self::splat(0))
+            }
+
+            /// Returns true for each negative lane and false if it is zero or positive.
+            pub fn is_negative(self) -> crate::$mask_ty<LANES> {
+                self.lanes_lt(Self::splat(0))
+            }
+        }
     }
 }
 
@@ -33,7 +49,7 @@ pub struct SimdIsize<const LANES: usize>([isize; LANES])
 where
     Self: crate::LanesAtMost64;
 
-impl_integer_vector! { SimdIsize, isize }
+impl_integer_vector! { SimdIsize, isize, MaskSize, SimdIsize }
 
 #[cfg(target_pointer_width = "32")]
 from_transmute_x86! { unsafe isizex4 => __m128i }
@@ -53,7 +69,7 @@ pub struct SimdI128<const LANES: usize>([i128; LANES])
 where
     Self: crate::LanesAtMost64;
 
-impl_integer_vector! { SimdI128, i128 }
+impl_integer_vector! { SimdI128, i128, Mask128, SimdI128 }
 
 from_transmute_x86! { unsafe i128x2 => __m256i }
 //from_transmute_x86! { unsafe i128x4 => __m512i }
@@ -64,7 +80,7 @@ pub struct SimdI16<const LANES: usize>([i16; LANES])
 where
     Self: crate::LanesAtMost64;
 
-impl_integer_vector! { SimdI16, i16 }
+impl_integer_vector! { SimdI16, i16, Mask16, SimdI16 }
 
 from_transmute_x86! { unsafe i16x8 => __m128i }
 from_transmute_x86! { unsafe i16x16 => __m256i }
@@ -76,7 +92,7 @@ pub struct SimdI32<const LANES: usize>([i32; LANES])
 where
     Self: crate::LanesAtMost64;
 
-impl_integer_vector! { SimdI32, i32 }
+impl_integer_vector! { SimdI32, i32, Mask32, SimdI32 }
 
 from_transmute_x86! { unsafe i32x4 => __m128i }
 from_transmute_x86! { unsafe i32x8 => __m256i }
@@ -88,7 +104,7 @@ pub struct SimdI64<const LANES: usize>([i64; LANES])
 where
     Self: crate::LanesAtMost64;
 
-impl_integer_vector! { SimdI64, i64 }
+impl_integer_vector! { SimdI64, i64, Mask64, SimdI64 }
 
 from_transmute_x86! { unsafe i64x2 => __m128i }
 from_transmute_x86! { unsafe i64x4 => __m256i }
@@ -100,7 +116,7 @@ pub struct SimdI8<const LANES: usize>([i8; LANES])
 where
     Self: crate::LanesAtMost64;
 
-impl_integer_vector! { SimdI8, i8 }
+impl_integer_vector! { SimdI8, i8, Mask8, SimdI8 }
 
 from_transmute_x86! { unsafe i8x16 => __m128i }
 from_transmute_x86! { unsafe i8x32 => __m256i }
diff --git a/crates/core_simd/tests/f32_ops.rs b/crates/core_simd/tests/f32_ops.rs
index ac5499b7ffeb..048c070a391d 100644
--- a/crates/core_simd/tests/f32_ops.rs
+++ b/crates/core_simd/tests/f32_ops.rs
@@ -1,3 +1,5 @@
+#![feature(is_subnormal)]
+
 #[macro_use]
 mod ops_macros;
 impl_float_tests! { SimdF32, f32, i32 }
diff --git a/crates/core_simd/tests/f64_ops.rs b/crates/core_simd/tests/f64_ops.rs
index dcdb2aa31522..8f0dd6b73655 100644
--- a/crates/core_simd/tests/f64_ops.rs
+++ b/crates/core_simd/tests/f64_ops.rs
@@ -1,3 +1,5 @@
+#![feature(is_subnormal)]
+
 #[macro_use]
 mod ops_macros;
 impl_float_tests! { SimdF64, f64, i64 }
diff --git a/crates/core_simd/tests/ops_macros.rs b/crates/core_simd/tests/ops_macros.rs
index 8e0b9626861f..a70a8a9c48b9 100644
--- a/crates/core_simd/tests/ops_macros.rs
+++ b/crates/core_simd/tests/ops_macros.rs
@@ -147,11 +147,27 @@ macro_rules! impl_signed_tests {
             test_helpers::test_lanes! {
                 fn neg<const LANES: usize>() {
                     test_helpers::test_unary_elementwise(
-                        &<Vector<LANES> as core::ops::Neg>::neg,
+                        &<Vector::<LANES> as core::ops::Neg>::neg,
                         &<Scalar as core::ops::Neg>::neg,
                         &|x| !x.contains(&Scalar::MIN),
                     );
                 }
+
+                fn is_positive<const LANES: usize>() {
+                    test_helpers::test_unary_mask_elementwise(
+                        &Vector::<LANES>::is_positive,
+                        &Scalar::is_positive,
+                        &|_| true,
+                    );
+                }
+
+                fn is_negative<const LANES: usize>() {
+                    test_helpers::test_unary_mask_elementwise(
+                        &Vector::<LANES>::is_negative,
+                        &Scalar::is_negative,
+                        &|_| true,
+                    );
+                }
             }
 
             test_helpers::test_lanes_panic! {
@@ -285,6 +301,62 @@ macro_rules! impl_float_tests {
             }
 
             test_helpers::test_lanes! {
+                fn is_sign_positive<const LANES: usize>() {
+                    test_helpers::test_unary_mask_elementwise(
+                        &Vector::<LANES>::is_sign_positive,
+                        &Scalar::is_sign_positive,
+                        &|_| true,
+                    );
+                }
+
+                fn is_sign_negative<const LANES: usize>() {
+                    test_helpers::test_unary_mask_elementwise(
+                        &Vector::<LANES>::is_sign_negative,
+                        &Scalar::is_sign_negative,
+                        &|_| true,
+                    );
+                }
+
+                fn is_finite<const LANES: usize>() {
+                    test_helpers::test_unary_mask_elementwise(
+                        &Vector::<LANES>::is_finite,
+                        &Scalar::is_finite,
+                        &|_| true,
+                    );
+                }
+
+                fn is_infinite<const LANES: usize>() {
+                    test_helpers::test_unary_mask_elementwise(
+                        &Vector::<LANES>::is_infinite,
+                        &Scalar::is_infinite,
+                        &|_| true,
+                    );
+                }
+
+                fn is_nan<const LANES: usize>() {
+                    test_helpers::test_unary_mask_elementwise(
+                        &Vector::<LANES>::is_nan,
+                        &Scalar::is_nan,
+                        &|_| true,
+                    );
+                }
+
+                fn is_normal<const LANES: usize>() {
+                    test_helpers::test_unary_mask_elementwise(
+                        &Vector::<LANES>::is_normal,
+                        &Scalar::is_normal,
+                        &|_| true,
+                    );
+                }
+
+                fn is_subnormal<const LANES: usize>() {
+                    test_helpers::test_unary_mask_elementwise(
+                        &Vector::<LANES>::is_subnormal,
+                        &Scalar::is_subnormal,
+                        &|_| true,
+                    );
+                }
+
                 fn abs<const LANES: usize>() {
                     test_helpers::test_unary_elementwise(
                         &Vector::<LANES>::abs,
diff --git a/crates/test_helpers/src/biteq.rs b/crates/test_helpers/src/biteq.rs
index 8c6280643803..4a41fe3a16ea 100644
--- a/crates/test_helpers/src/biteq.rs
+++ b/crates/test_helpers/src/biteq.rs
@@ -5,6 +5,16 @@ pub trait BitEq {
     fn fmt(&self, f: &mut core::fmt::Formatter) -> core::fmt::Result;
 }
 
+impl BitEq for bool {
+    fn biteq(&self, other: &Self) -> bool {
+        self == other
+    }
+
+    fn fmt(&self, f: &mut core::fmt::Formatter) -> core::fmt::Result {
+        write!(f, "{:?}", self)
+    }
+}
+
 macro_rules! impl_integer_biteq {
     { $($type:ty),* } => {
         $(
diff --git a/crates/test_helpers/src/lib.rs b/crates/test_helpers/src/lib.rs
index 2c74c02d48a0..df1abcddaea1 100644
--- a/crates/test_helpers/src/lib.rs
+++ b/crates/test_helpers/src/lib.rs
@@ -124,6 +124,32 @@ pub fn test_unary_elementwise<Scalar, ScalarResult, Vector, VectorResult, const
     });
 }
 
+/// Test a unary vector function against a unary scalar function, applied elementwise.
+#[inline(never)]
+pub fn test_unary_mask_elementwise<Scalar, Vector, Mask, const LANES: usize>(
+    fv: &dyn Fn(Vector) -> Mask,
+    fs: &dyn Fn(Scalar) -> bool,
+    check: &dyn Fn([Scalar; LANES]) -> bool,
+) where
+    Scalar: Copy + Default + core::fmt::Debug + DefaultStrategy,
+    Vector: Into<[Scalar; LANES]> + From<[Scalar; LANES]> + Copy,
+    Mask: Into<[bool; LANES]> + From<[bool; LANES]> + Copy,
+{
+    test_1(&|x: [Scalar; LANES]| {
+        proptest::prop_assume!(check(x));
+        let result_1: [bool; LANES] = fv(x.into()).into();
+        let result_2: [bool; LANES] = {
+            let mut result = [false; LANES];
+            for (i, o) in x.iter().zip(result.iter_mut()) {
+                *o = fs(*i);
+            }
+            result
+        };
+        crate::prop_assert_biteq!(result_1, result_2);
+        Ok(())
+    });
+}
+
 /// Test a binary vector function against a binary scalar function, applied elementwise.
 #[inline(never)]
 pub fn test_binary_elementwise<
@@ -307,7 +333,7 @@ macro_rules! test_lanes {
             }
         )*
     }
-}  
+}
 
 /// Expand a const-generic `#[should_panic]` test into separate tests for each possible lane count.
 #[macro_export]
@@ -382,4 +408,4 @@ macro_rules! test_lanes_panic {
             }
         )*
     }
-}  
+}

From 97bbe2d86a3c35f970266dfa1a9bd9a878d629f3 Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Sat, 6 Mar 2021 09:28:55 -0500
Subject: [PATCH 110/249] Fix normal and subnormal classification

---
 crates/core_simd/src/vector/float.rs | 5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)

diff --git a/crates/core_simd/src/vector/float.rs b/crates/core_simd/src/vector/float.rs
index 85456e7bf1ec..bea711a50359 100644
--- a/crates/core_simd/src/vector/float.rs
+++ b/crates/core_simd/src/vector/float.rs
@@ -79,15 +79,14 @@ macro_rules! impl_float_vector {
             /// Returns true for each lane if its value is subnormal.
             #[inline]
             pub fn is_subnormal(self) -> crate::$mask_ty<LANES> {
-                let exponent_mask = crate::$bits_ty::splat(!0 << <$type>::MANTISSA_DIGITS);
-                self.abs().lanes_ne(Self::splat(0.0)) & (self.to_bits() & exponent_mask).lanes_eq(crate::$bits_ty::splat(0))
+                self.abs().lanes_ne(Self::splat(0.0)) & (self.to_bits() & Self::splat(<$type>::INFINITY).to_bits()).lanes_eq(crate::$bits_ty::splat(0))
             }
 
             /// Returns true for each lane if its value is neither neither zero, infinite,
             /// subnormal, or `NaN`.
             #[inline]
             pub fn is_normal(self) -> crate::$mask_ty<LANES> {
-                !(self.abs().lanes_eq(Self::splat(0.0)) | self.is_nan() | self.is_subnormal())
+                !(self.abs().lanes_eq(Self::splat(0.0)) | self.is_nan() | self.is_subnormal() | self.is_infinite())
             }
         }
     };

From e6a530907a8a6d0ab943ccd7aaebdfef9d609605 Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Sat, 3 Apr 2021 14:43:33 -0400
Subject: [PATCH 111/249] Reduce maximum lanes from 64 to 32

---
 crates/core_simd/src/comparisons.rs           |  6 +-
 crates/core_simd/src/first.rs                 | 24 +++---
 crates/core_simd/src/fmt.rs                   |  2 +-
 crates/core_simd/src/intrinsics.rs            |  1 -
 crates/core_simd/src/lanes_at_most_64.rs      | 15 ++--
 crates/core_simd/src/lib.rs                   |  2 +-
 crates/core_simd/src/masks/bitmask.rs         | 38 ++++-----
 crates/core_simd/src/masks/full_masks.rs      | 64 +++++++-------
 crates/core_simd/src/masks/mod.rs             | 66 +++++++--------
 crates/core_simd/src/math.rs                  |  4 +-
 crates/core_simd/src/ops.rs                   | 84 +++++++++----------
 crates/core_simd/src/permute.rs               |  1 -
 crates/core_simd/src/round.rs                 |  6 +-
 crates/core_simd/src/vector/float.rs          | 14 ++--
 crates/core_simd/src/vector/int.rs            | 22 ++---
 crates/core_simd/src/vector/uint.rs           | 18 ++--
 crates/core_simd/tests/mask_ops_impl/mask8.rs |  1 -
 crates/test_helpers/src/lib.rs                | 72 +++++++---------
 18 files changed, 212 insertions(+), 228 deletions(-)

diff --git a/crates/core_simd/src/comparisons.rs b/crates/core_simd/src/comparisons.rs
index eb901a89ca33..455f30dc97ee 100644
--- a/crates/core_simd/src/comparisons.rs
+++ b/crates/core_simd/src/comparisons.rs
@@ -1,12 +1,12 @@
-use crate::LanesAtMost64;
+use crate::LanesAtMost32;
 
 macro_rules! implement_mask_ops {
     { $($vector:ident => $mask:ident ($inner_mask_ty:ident, $inner_ty:ident),)* } => {
         $(
             impl<const LANES: usize> crate::$vector<LANES>
             where
-                crate::$vector<LANES>: LanesAtMost64,
-                crate::$inner_ty<LANES>: LanesAtMost64,
+                crate::$vector<LANES>: LanesAtMost32,
+                crate::$inner_ty<LANES>: LanesAtMost32,
             {
                 /// Test if each lane is equal to the corresponding lane in `other`.
                 #[inline]
diff --git a/crates/core_simd/src/first.rs b/crates/core_simd/src/first.rs
index b18fe5213a3d..50602829d482 100644
--- a/crates/core_simd/src/first.rs
+++ b/crates/core_simd/src/first.rs
@@ -1,7 +1,7 @@
 /// Implements common traits on the specified vector `$name`, holding multiple `$lanes` of `$type`.
 macro_rules! impl_vector {
     { $name:ident, $type:ty } => {
-        impl<const LANES: usize> $name<LANES> where Self: crate::LanesAtMost64 {
+        impl<const LANES: usize> $name<LANES> where Self: crate::LanesAtMost32 {
             /// Construct a SIMD vector by setting all lanes to the given value.
             pub const fn splat(value: $type) -> Self {
                 Self([value; LANES])
@@ -44,23 +44,23 @@ macro_rules! impl_vector {
             }
         }
 
-        impl<const LANES: usize> Copy for $name<LANES> where Self: crate::LanesAtMost64 {}
+        impl<const LANES: usize> Copy for $name<LANES> where Self: crate::LanesAtMost32 {}
 
-        impl<const LANES: usize> Clone for $name<LANES> where Self: crate::LanesAtMost64 {
+        impl<const LANES: usize> Clone for $name<LANES> where Self: crate::LanesAtMost32 {
             #[inline]
             fn clone(&self) -> Self {
                 *self
             }
         }
 
-        impl<const LANES: usize> Default for $name<LANES> where Self: crate::LanesAtMost64 {
+        impl<const LANES: usize> Default for $name<LANES> where Self: crate::LanesAtMost32 {
             #[inline]
             fn default() -> Self {
                 Self::splat(<$type>::default())
             }
         }
 
-        impl<const LANES: usize> PartialEq for $name<LANES> where Self: crate::LanesAtMost64 {
+        impl<const LANES: usize> PartialEq for $name<LANES> where Self: crate::LanesAtMost32 {
             #[inline]
             fn eq(&self, other: &Self) -> bool {
                 // TODO use SIMD equality
@@ -68,7 +68,7 @@ macro_rules! impl_vector {
             }
         }
 
-        impl<const LANES: usize> PartialOrd for $name<LANES> where Self: crate::LanesAtMost64 {
+        impl<const LANES: usize> PartialOrd for $name<LANES> where Self: crate::LanesAtMost32 {
             #[inline]
             fn partial_cmp(&self, other: &Self) -> Option<core::cmp::Ordering> {
                 // TODO use SIMD equalitya
@@ -77,14 +77,14 @@ macro_rules! impl_vector {
         }
 
         // array references
-        impl<const LANES: usize> AsRef<[$type; LANES]> for $name<LANES> where Self: crate::LanesAtMost64 {
+        impl<const LANES: usize> AsRef<[$type; LANES]> for $name<LANES> where Self: crate::LanesAtMost32 {
             #[inline]
             fn as_ref(&self) -> &[$type; LANES] {
                 &self.0
             }
         }
 
-        impl<const LANES: usize> AsMut<[$type; LANES]> for $name<LANES> where Self: crate::LanesAtMost64 {
+        impl<const LANES: usize> AsMut<[$type; LANES]> for $name<LANES> where Self: crate::LanesAtMost32 {
             #[inline]
             fn as_mut(&mut self) -> &mut [$type; LANES] {
                 &mut self.0
@@ -92,14 +92,14 @@ macro_rules! impl_vector {
         }
 
         // slice references
-        impl<const LANES: usize> AsRef<[$type]> for $name<LANES> where Self: crate::LanesAtMost64 {
+        impl<const LANES: usize> AsRef<[$type]> for $name<LANES> where Self: crate::LanesAtMost32 {
             #[inline]
             fn as_ref(&self) -> &[$type] {
                 &self.0
             }
         }
 
-        impl<const LANES: usize> AsMut<[$type]> for $name<LANES> where Self: crate::LanesAtMost64 {
+        impl<const LANES: usize> AsMut<[$type]> for $name<LANES> where Self: crate::LanesAtMost32 {
             #[inline]
             fn as_mut(&mut self) -> &mut [$type] {
                 &mut self.0
@@ -107,13 +107,13 @@ macro_rules! impl_vector {
         }
 
         // vector/array conversion
-        impl<const LANES: usize> From<[$type; LANES]> for $name<LANES> where Self: crate::LanesAtMost64 {
+        impl<const LANES: usize> From<[$type; LANES]> for $name<LANES> where Self: crate::LanesAtMost32 {
             fn from(array: [$type; LANES]) -> Self {
                 Self(array)
             }
         }
 
-        impl <const LANES: usize> From<$name<LANES>> for [$type; LANES] where $name<LANES>: crate::LanesAtMost64 {
+        impl <const LANES: usize> From<$name<LANES>> for [$type; LANES] where $name<LANES>: crate::LanesAtMost32 {
             fn from(vector: $name<LANES>) -> Self {
                 vector.to_array()
             }
diff --git a/crates/core_simd/src/fmt.rs b/crates/core_simd/src/fmt.rs
index 6fa238cfda6a..faf0c20e922e 100644
--- a/crates/core_simd/src/fmt.rs
+++ b/crates/core_simd/src/fmt.rs
@@ -35,7 +35,7 @@ macro_rules! impl_fmt_trait {
             $( // repeat trait
                 impl<const LANES: usize> core::fmt::$trait for crate::$type<LANES>
                 where
-                    Self: crate::LanesAtMost64,
+                    Self: crate::LanesAtMost32,
                 {
                     fn fmt(&self, f: &mut core::fmt::Formatter) -> core::fmt::Result {
                         $format(self.as_ref(), f)
diff --git a/crates/core_simd/src/intrinsics.rs b/crates/core_simd/src/intrinsics.rs
index 93c97cfed8ec..fafeed6a62a8 100644
--- a/crates/core_simd/src/intrinsics.rs
+++ b/crates/core_simd/src/intrinsics.rs
@@ -61,7 +61,6 @@ extern "platform-intrinsic" {
     pub(crate) fn simd_shuffle8<T, U>(x: T, y: T, idx: [u32; 8]) -> U;
     pub(crate) fn simd_shuffle16<T, U>(x: T, y: T, idx: [u32; 16]) -> U;
     pub(crate) fn simd_shuffle32<T, U>(x: T, y: T, idx: [u32; 32]) -> U;
-    pub(crate) fn simd_shuffle64<T, U>(x: T, y: T, idx: [u32; 64]) -> U;
 
     // {s,u}add.sat
     pub(crate) fn simd_saturating_add<T>(x: T, y: T) -> T;
diff --git a/crates/core_simd/src/lanes_at_most_64.rs b/crates/core_simd/src/lanes_at_most_64.rs
index 63882152b6d9..dc0e02c22a26 100644
--- a/crates/core_simd/src/lanes_at_most_64.rs
+++ b/crates/core_simd/src/lanes_at_most_64.rs
@@ -1,15 +1,14 @@
 /// Implemented for bitmask sizes that are supported by the implementation.
-pub trait LanesAtMost64 {}
+pub trait LanesAtMost32 {}
 
 macro_rules! impl_for {
     { $name:ident } => {
-        impl LanesAtMost64 for $name<1> {}
-        impl LanesAtMost64 for $name<2> {}
-        impl LanesAtMost64 for $name<4> {}
-        impl LanesAtMost64 for $name<8> {}
-        impl LanesAtMost64 for $name<16> {}
-        impl LanesAtMost64 for $name<32> {}
-        impl LanesAtMost64 for $name<64> {}
+        impl LanesAtMost32 for $name<1> {}
+        impl LanesAtMost32 for $name<2> {}
+        impl LanesAtMost32 for $name<4> {}
+        impl LanesAtMost32 for $name<8> {}
+        impl LanesAtMost32 for $name<16> {}
+        impl LanesAtMost32 for $name<32> {}
     }
 }
 
diff --git a/crates/core_simd/src/lib.rs b/crates/core_simd/src/lib.rs
index 747096306ff7..906ee3f06ae5 100644
--- a/crates/core_simd/src/lib.rs
+++ b/crates/core_simd/src/lib.rs
@@ -21,7 +21,7 @@ mod round;
 mod math;
 
 mod lanes_at_most_64;
-pub use lanes_at_most_64::LanesAtMost64;
+pub use lanes_at_most_64::LanesAtMost32;
 
 mod masks;
 pub use masks::*;
diff --git a/crates/core_simd/src/masks/bitmask.rs b/crates/core_simd/src/masks/bitmask.rs
index b9b1160a3f38..d7400699fde8 100644
--- a/crates/core_simd/src/masks/bitmask.rs
+++ b/crates/core_simd/src/masks/bitmask.rs
@@ -1,15 +1,15 @@
-use crate::LanesAtMost64;
+use crate::LanesAtMost32;
 
 /// A mask where each lane is represented by a single bit.
 #[derive(Copy, Clone, Debug)]
 #[repr(transparent)]
 pub struct BitMask<const LANES: usize>(u64)
 where
-    BitMask<LANES>: LanesAtMost64;
+    BitMask<LANES>: LanesAtMost32;
 
 impl<const LANES: usize> BitMask<LANES>
 where
-    Self: LanesAtMost64,
+    Self: LanesAtMost32,
 {
     /// Construct a mask by setting all lanes to the given value.
     pub fn splat(value: bool) -> Self {
@@ -43,7 +43,7 @@ where
 
 impl<const LANES: usize> core::ops::BitAnd for BitMask<LANES>
 where
-    Self: LanesAtMost64,
+    Self: LanesAtMost32,
 {
     type Output = Self;
     #[inline]
@@ -54,7 +54,7 @@ where
 
 impl<const LANES: usize> core::ops::BitAnd<bool> for BitMask<LANES>
 where
-    Self: LanesAtMost64,
+    Self: LanesAtMost32,
 {
     type Output = Self;
     #[inline]
@@ -65,7 +65,7 @@ where
 
 impl<const LANES: usize> core::ops::BitAnd<BitMask<LANES>> for bool
 where
-    BitMask<LANES>: LanesAtMost64,
+    BitMask<LANES>: LanesAtMost32,
 {
     type Output = BitMask<LANES>;
     #[inline]
@@ -76,7 +76,7 @@ where
 
 impl<const LANES: usize> core::ops::BitOr for BitMask<LANES>
 where
-    Self: LanesAtMost64,
+    Self: LanesAtMost32,
 {
     type Output = Self;
     #[inline]
@@ -87,7 +87,7 @@ where
 
 impl<const LANES: usize> core::ops::BitOr<bool> for BitMask<LANES>
 where
-    Self: LanesAtMost64,
+    Self: LanesAtMost32,
 {
     type Output = Self;
     #[inline]
@@ -98,7 +98,7 @@ where
 
 impl<const LANES: usize> core::ops::BitOr<BitMask<LANES>> for bool
 where
-    BitMask<LANES>: LanesAtMost64,
+    BitMask<LANES>: LanesAtMost32,
 {
     type Output = BitMask<LANES>;
     #[inline]
@@ -109,7 +109,7 @@ where
 
 impl<const LANES: usize> core::ops::BitXor for BitMask<LANES>
 where
-    Self: LanesAtMost64,
+    Self: LanesAtMost32,
 {
     type Output = Self;
     #[inline]
@@ -120,7 +120,7 @@ where
 
 impl<const LANES: usize> core::ops::BitXor<bool> for BitMask<LANES>
 where
-    Self: LanesAtMost64,
+    Self: LanesAtMost32,
 {
     type Output = Self;
     #[inline]
@@ -131,7 +131,7 @@ where
 
 impl<const LANES: usize> core::ops::BitXor<BitMask<LANES>> for bool
 where
-    BitMask<LANES>: LanesAtMost64,
+    BitMask<LANES>: LanesAtMost32,
 {
     type Output = BitMask<LANES>;
     #[inline]
@@ -142,7 +142,7 @@ where
 
 impl<const LANES: usize> core::ops::Not for BitMask<LANES>
 where
-    Self: LanesAtMost64,
+    Self: LanesAtMost32,
 {
     type Output = BitMask<LANES>;
     #[inline]
@@ -153,7 +153,7 @@ where
 
 impl<const LANES: usize> core::ops::BitAndAssign for BitMask<LANES>
 where
-    Self: LanesAtMost64,
+    Self: LanesAtMost32,
 {
     #[inline]
     fn bitand_assign(&mut self, rhs: Self) {
@@ -163,7 +163,7 @@ where
 
 impl<const LANES: usize> core::ops::BitAndAssign<bool> for BitMask<LANES>
 where
-    Self: LanesAtMost64,
+    Self: LanesAtMost32,
 {
     #[inline]
     fn bitand_assign(&mut self, rhs: bool) {
@@ -173,7 +173,7 @@ where
 
 impl<const LANES: usize> core::ops::BitOrAssign for BitMask<LANES>
 where
-    Self: LanesAtMost64,
+    Self: LanesAtMost32,
 {
     #[inline]
     fn bitor_assign(&mut self, rhs: Self) {
@@ -183,7 +183,7 @@ where
 
 impl<const LANES: usize> core::ops::BitOrAssign<bool> for BitMask<LANES>
 where
-    Self: LanesAtMost64,
+    Self: LanesAtMost32,
 {
     #[inline]
     fn bitor_assign(&mut self, rhs: bool) {
@@ -193,7 +193,7 @@ where
 
 impl<const LANES: usize> core::ops::BitXorAssign for BitMask<LANES>
 where
-    Self: LanesAtMost64,
+    Self: LanesAtMost32,
 {
     #[inline]
     fn bitxor_assign(&mut self, rhs: Self) {
@@ -203,7 +203,7 @@ where
 
 impl<const LANES: usize> core::ops::BitXorAssign<bool> for BitMask<LANES>
 where
-    Self: LanesAtMost64,
+    Self: LanesAtMost32,
 {
     #[inline]
     fn bitxor_assign(&mut self, rhs: bool) {
diff --git a/crates/core_simd/src/masks/full_masks.rs b/crates/core_simd/src/masks/full_masks.rs
index d88875deacac..cca077b14d04 100644
--- a/crates/core_simd/src/masks/full_masks.rs
+++ b/crates/core_simd/src/masks/full_masks.rs
@@ -20,16 +20,16 @@ macro_rules! define_mask {
         #[repr(transparent)]
         pub struct $name<const $lanes: usize>($type)
         where
-            $type: crate::LanesAtMost64;
+            $type: crate::LanesAtMost32;
 
         impl<const LANES: usize> Copy for $name<LANES>
         where
-            $type: crate::LanesAtMost64,
+            $type: crate::LanesAtMost32,
         {}
 
         impl<const LANES: usize> Clone for $name<LANES>
         where
-            $type: crate::LanesAtMost64,
+            $type: crate::LanesAtMost32,
         {
             #[inline]
             fn clone(&self) -> Self {
@@ -39,7 +39,7 @@ macro_rules! define_mask {
 
         impl<const $lanes: usize> $name<$lanes>
         where
-            $type: crate::LanesAtMost64,
+            $type: crate::LanesAtMost32,
         {
             /// Construct a mask by setting all lanes to the given value.
             pub fn splat(value: bool) -> Self {
@@ -98,7 +98,7 @@ macro_rules! define_mask {
 
         impl<const $lanes: usize> core::convert::From<bool> for $name<$lanes>
         where
-            $type: crate::LanesAtMost64,
+            $type: crate::LanesAtMost32,
         {
             fn from(value: bool) -> Self {
                 Self::splat(value)
@@ -107,7 +107,7 @@ macro_rules! define_mask {
 
         impl<const $lanes: usize> core::convert::TryFrom<$type> for $name<$lanes>
         where
-            $type: crate::LanesAtMost64,
+            $type: crate::LanesAtMost32,
         {
             type Error = TryFromMaskError;
             fn try_from(value: $type) -> Result<Self, Self::Error> {
@@ -121,7 +121,7 @@ macro_rules! define_mask {
 
         impl<const $lanes: usize> core::convert::From<$name<$lanes>> for $type
         where
-            $type: crate::LanesAtMost64,
+            $type: crate::LanesAtMost32,
         {
             fn from(value: $name<$lanes>) -> Self {
                 value.0
@@ -130,8 +130,8 @@ macro_rules! define_mask {
 
         impl<const $lanes: usize> core::convert::From<crate::BitMask<$lanes>> for $name<$lanes>
         where
-            $type: crate::LanesAtMost64,
-            crate::BitMask<$lanes>: crate::LanesAtMost64,
+            $type: crate::LanesAtMost32,
+            crate::BitMask<$lanes>: crate::LanesAtMost32,
         {
             fn from(value: crate::BitMask<$lanes>) -> Self {
                 // TODO use an intrinsic to do this efficiently (with LLVM's sext instruction)
@@ -145,8 +145,8 @@ macro_rules! define_mask {
 
         impl<const $lanes: usize> core::convert::From<$name<$lanes>> for crate::BitMask<$lanes>
         where
-            $type: crate::LanesAtMost64,
-            crate::BitMask<$lanes>: crate::LanesAtMost64,
+            $type: crate::LanesAtMost32,
+            crate::BitMask<$lanes>: crate::LanesAtMost32,
         {
             fn from(value: $name<$lanes>) -> Self {
                 // TODO use an intrinsic to do this efficiently (with LLVM's trunc instruction)
@@ -160,7 +160,7 @@ macro_rules! define_mask {
 
         impl<const $lanes: usize> core::fmt::Debug for $name<$lanes>
         where
-            $type: crate::LanesAtMost64,
+            $type: crate::LanesAtMost32,
         {
             fn fmt(&self, f: &mut core::fmt::Formatter) -> core::fmt::Result {
                 f.debug_list()
@@ -171,7 +171,7 @@ macro_rules! define_mask {
 
         impl<const $lanes: usize> core::fmt::Binary for $name<$lanes>
         where
-            $type: crate::LanesAtMost64,
+            $type: crate::LanesAtMost32,
         {
             fn fmt(&self, f: &mut core::fmt::Formatter) -> core::fmt::Result {
                 core::fmt::Binary::fmt(&self.0, f)
@@ -180,7 +180,7 @@ macro_rules! define_mask {
 
         impl<const $lanes: usize> core::fmt::Octal for $name<$lanes>
         where
-            $type: crate::LanesAtMost64,
+            $type: crate::LanesAtMost32,
         {
             fn fmt(&self, f: &mut core::fmt::Formatter) -> core::fmt::Result {
                 core::fmt::Octal::fmt(&self.0, f)
@@ -189,7 +189,7 @@ macro_rules! define_mask {
 
         impl<const $lanes: usize> core::fmt::LowerHex for $name<$lanes>
         where
-            $type: crate::LanesAtMost64,
+            $type: crate::LanesAtMost32,
         {
             fn fmt(&self, f: &mut core::fmt::Formatter) -> core::fmt::Result {
                 core::fmt::LowerHex::fmt(&self.0, f)
@@ -198,7 +198,7 @@ macro_rules! define_mask {
 
         impl<const $lanes: usize> core::fmt::UpperHex for $name<$lanes>
         where
-            $type: crate::LanesAtMost64,
+            $type: crate::LanesAtMost32,
         {
             fn fmt(&self, f: &mut core::fmt::Formatter) -> core::fmt::Result {
                 core::fmt::UpperHex::fmt(&self.0, f)
@@ -207,7 +207,7 @@ macro_rules! define_mask {
 
         impl<const LANES: usize> core::ops::BitAnd for $name<LANES>
         where
-            $type: crate::LanesAtMost64,
+            $type: crate::LanesAtMost32,
         {
             type Output = Self;
             #[inline]
@@ -218,7 +218,7 @@ macro_rules! define_mask {
 
         impl<const LANES: usize> core::ops::BitAnd<bool> for $name<LANES>
         where
-            $type: crate::LanesAtMost64,
+            $type: crate::LanesAtMost32,
         {
             type Output = Self;
             #[inline]
@@ -229,7 +229,7 @@ macro_rules! define_mask {
 
         impl<const LANES: usize> core::ops::BitAnd<$name<LANES>> for bool
         where
-            $type: crate::LanesAtMost64,
+            $type: crate::LanesAtMost32,
         {
             type Output = $name<LANES>;
             #[inline]
@@ -240,7 +240,7 @@ macro_rules! define_mask {
 
         impl<const LANES: usize> core::ops::BitOr for $name<LANES>
         where
-            $type: crate::LanesAtMost64,
+            $type: crate::LanesAtMost32,
         {
             type Output = Self;
             #[inline]
@@ -251,7 +251,7 @@ macro_rules! define_mask {
 
         impl<const LANES: usize> core::ops::BitOr<bool> for $name<LANES>
         where
-            $type: crate::LanesAtMost64,
+            $type: crate::LanesAtMost32,
         {
             type Output = Self;
             #[inline]
@@ -262,7 +262,7 @@ macro_rules! define_mask {
 
         impl<const LANES: usize> core::ops::BitOr<$name<LANES>> for bool
         where
-            $type: crate::LanesAtMost64,
+            $type: crate::LanesAtMost32,
         {
             type Output = $name<LANES>;
             #[inline]
@@ -273,7 +273,7 @@ macro_rules! define_mask {
 
         impl<const LANES: usize> core::ops::BitXor for $name<LANES>
         where
-            $type: crate::LanesAtMost64,
+            $type: crate::LanesAtMost32,
         {
             type Output = Self;
             #[inline]
@@ -284,7 +284,7 @@ macro_rules! define_mask {
 
         impl<const LANES: usize> core::ops::BitXor<bool> for $name<LANES>
         where
-            $type: crate::LanesAtMost64,
+            $type: crate::LanesAtMost32,
         {
             type Output = Self;
             #[inline]
@@ -295,7 +295,7 @@ macro_rules! define_mask {
 
         impl<const LANES: usize> core::ops::BitXor<$name<LANES>> for bool
         where
-            $type: crate::LanesAtMost64,
+            $type: crate::LanesAtMost32,
         {
             type Output = $name<LANES>;
             #[inline]
@@ -306,7 +306,7 @@ macro_rules! define_mask {
 
         impl<const LANES: usize> core::ops::Not for $name<LANES>
         where
-            $type: crate::LanesAtMost64,
+            $type: crate::LanesAtMost32,
         {
             type Output = $name<LANES>;
             #[inline]
@@ -317,7 +317,7 @@ macro_rules! define_mask {
 
         impl<const LANES: usize> core::ops::BitAndAssign for $name<LANES>
         where
-            $type: crate::LanesAtMost64,
+            $type: crate::LanesAtMost32,
         {
             #[inline]
             fn bitand_assign(&mut self, rhs: Self) {
@@ -327,7 +327,7 @@ macro_rules! define_mask {
 
         impl<const LANES: usize> core::ops::BitAndAssign<bool> for $name<LANES>
         where
-            $type: crate::LanesAtMost64,
+            $type: crate::LanesAtMost32,
         {
             #[inline]
             fn bitand_assign(&mut self, rhs: bool) {
@@ -337,7 +337,7 @@ macro_rules! define_mask {
 
         impl<const LANES: usize> core::ops::BitOrAssign for $name<LANES>
         where
-            $type: crate::LanesAtMost64,
+            $type: crate::LanesAtMost32,
         {
             #[inline]
             fn bitor_assign(&mut self, rhs: Self) {
@@ -347,7 +347,7 @@ macro_rules! define_mask {
 
         impl<const LANES: usize> core::ops::BitOrAssign<bool> for $name<LANES>
         where
-            $type: crate::LanesAtMost64,
+            $type: crate::LanesAtMost32,
         {
             #[inline]
             fn bitor_assign(&mut self, rhs: bool) {
@@ -357,7 +357,7 @@ macro_rules! define_mask {
 
         impl<const LANES: usize> core::ops::BitXorAssign for $name<LANES>
         where
-            $type: crate::LanesAtMost64,
+            $type: crate::LanesAtMost32,
         {
             #[inline]
             fn bitxor_assign(&mut self, rhs: Self) {
@@ -367,7 +367,7 @@ macro_rules! define_mask {
 
         impl<const LANES: usize> core::ops::BitXorAssign<bool> for $name<LANES>
         where
-            $type: crate::LanesAtMost64,
+            $type: crate::LanesAtMost32,
         {
             #[inline]
             fn bitxor_assign(&mut self, rhs: bool) {
diff --git a/crates/core_simd/src/masks/mod.rs b/crates/core_simd/src/masks/mod.rs
index d30399fb5ad1..0b986aaf7e17 100644
--- a/crates/core_simd/src/masks/mod.rs
+++ b/crates/core_simd/src/masks/mod.rs
@@ -7,7 +7,7 @@ pub use full_masks::*;
 mod bitmask;
 pub use bitmask::*;
 
-use crate::LanesAtMost64;
+use crate::LanesAtMost32;
 
 macro_rules! define_opaque_mask {
     {
@@ -17,11 +17,11 @@ macro_rules! define_opaque_mask {
     } => {
         $(#[$attr])*
         #[allow(non_camel_case_types)]
-        pub struct $name<const $lanes: usize>($inner_ty) where $bits_ty: LanesAtMost64;
+        pub struct $name<const $lanes: usize>($inner_ty) where $bits_ty: LanesAtMost32;
 
         impl<const $lanes: usize> $name<$lanes>
         where
-            $bits_ty: LanesAtMost64
+            $bits_ty: LanesAtMost32
         {
             /// Construct a mask by setting all lanes to the given value.
             pub fn splat(value: bool) -> Self {
@@ -71,8 +71,8 @@ macro_rules! define_opaque_mask {
 
         impl<const $lanes: usize> From<BitMask<$lanes>> for $name<$lanes>
         where
-            $bits_ty: LanesAtMost64,
-            BitMask<$lanes>: LanesAtMost64,
+            $bits_ty: LanesAtMost32,
+            BitMask<$lanes>: LanesAtMost32,
         {
             fn from(value: BitMask<$lanes>) -> Self {
                 Self(value.into())
@@ -81,8 +81,8 @@ macro_rules! define_opaque_mask {
 
         impl<const $lanes: usize> From<$name<$lanes>> for crate::BitMask<$lanes>
         where
-            $bits_ty: LanesAtMost64,
-            BitMask<$lanes>: LanesAtMost64,
+            $bits_ty: LanesAtMost32,
+            BitMask<$lanes>: LanesAtMost32,
         {
             fn from(value: $name<$lanes>) -> Self {
                 value.0.into()
@@ -91,7 +91,7 @@ macro_rules! define_opaque_mask {
 
         impl<const $lanes: usize> From<$inner_ty> for $name<$lanes>
         where
-            $bits_ty: LanesAtMost64,
+            $bits_ty: LanesAtMost32,
         {
             fn from(value: $inner_ty) -> Self {
                 Self(value)
@@ -100,7 +100,7 @@ macro_rules! define_opaque_mask {
 
         impl<const $lanes: usize> From<$name<$lanes>> for $inner_ty
         where
-            $bits_ty: LanesAtMost64,
+            $bits_ty: LanesAtMost32,
         {
             fn from(value: $name<$lanes>) -> Self {
                 value.0
@@ -108,13 +108,13 @@ macro_rules! define_opaque_mask {
         }
 
         // vector/array conversion
-        impl<const $lanes: usize> From<[bool; $lanes]> for $name<$lanes> where $bits_ty: crate::LanesAtMost64 {
+        impl<const $lanes: usize> From<[bool; $lanes]> for $name<$lanes> where $bits_ty: crate::LanesAtMost32 {
             fn from(array: [bool; $lanes]) -> Self {
                 Self::from_array(array)
             }
         }
 
-        impl <const $lanes: usize> From<$name<$lanes>> for [bool; $lanes] where $bits_ty: crate::LanesAtMost64 {
+        impl <const $lanes: usize> From<$name<$lanes>> for [bool; $lanes] where $bits_ty: crate::LanesAtMost32 {
             fn from(vector: $name<$lanes>) -> Self {
                 vector.to_array()
             }
@@ -123,12 +123,12 @@ macro_rules! define_opaque_mask {
         impl<const $lanes: usize> Copy for $name<$lanes>
         where
             $inner_ty: Copy,
-            $bits_ty: LanesAtMost64,
+            $bits_ty: LanesAtMost32,
         {}
 
         impl<const $lanes: usize> Clone for $name<$lanes>
         where
-            $bits_ty: LanesAtMost64,
+            $bits_ty: LanesAtMost32,
         {
             #[inline]
             fn clone(&self) -> Self {
@@ -138,7 +138,7 @@ macro_rules! define_opaque_mask {
 
         impl<const $lanes: usize> Default for $name<$lanes>
         where
-            $bits_ty: LanesAtMost64,
+            $bits_ty: LanesAtMost32,
         {
             #[inline]
             fn default() -> Self {
@@ -148,7 +148,7 @@ macro_rules! define_opaque_mask {
 
         impl<const $lanes: usize> PartialEq for $name<$lanes>
         where
-            $bits_ty: LanesAtMost64,
+            $bits_ty: LanesAtMost32,
         {
             #[inline]
             fn eq(&self, other: &Self) -> bool {
@@ -158,7 +158,7 @@ macro_rules! define_opaque_mask {
 
         impl<const $lanes: usize> PartialOrd for $name<$lanes>
         where
-            $bits_ty: LanesAtMost64,
+            $bits_ty: LanesAtMost32,
         {
             #[inline]
             fn partial_cmp(&self, other: &Self) -> Option<core::cmp::Ordering> {
@@ -168,7 +168,7 @@ macro_rules! define_opaque_mask {
 
         impl<const $lanes: usize> core::fmt::Debug for $name<$lanes>
         where
-            $bits_ty: LanesAtMost64,
+            $bits_ty: LanesAtMost32,
         {
             fn fmt(&self, f: &mut core::fmt::Formatter) -> core::fmt::Result {
                 core::fmt::Debug::fmt(&self.0, f)
@@ -177,7 +177,7 @@ macro_rules! define_opaque_mask {
 
         impl<const LANES: usize> core::ops::BitAnd for $name<LANES>
         where
-            $bits_ty: LanesAtMost64,
+            $bits_ty: LanesAtMost32,
         {
             type Output = Self;
             #[inline]
@@ -188,7 +188,7 @@ macro_rules! define_opaque_mask {
 
         impl<const LANES: usize> core::ops::BitAnd<bool> for $name<LANES>
         where
-            $bits_ty: LanesAtMost64,
+            $bits_ty: LanesAtMost32,
         {
             type Output = Self;
             #[inline]
@@ -199,7 +199,7 @@ macro_rules! define_opaque_mask {
 
         impl<const LANES: usize> core::ops::BitAnd<$name<LANES>> for bool
         where
-            $bits_ty: LanesAtMost64,
+            $bits_ty: LanesAtMost32,
         {
             type Output = $name<LANES>;
             #[inline]
@@ -210,7 +210,7 @@ macro_rules! define_opaque_mask {
 
         impl<const LANES: usize> core::ops::BitOr for $name<LANES>
         where
-            $bits_ty: LanesAtMost64,
+            $bits_ty: LanesAtMost32,
         {
             type Output = Self;
             #[inline]
@@ -221,7 +221,7 @@ macro_rules! define_opaque_mask {
 
         impl<const LANES: usize> core::ops::BitOr<bool> for $name<LANES>
         where
-            $bits_ty: LanesAtMost64,
+            $bits_ty: LanesAtMost32,
         {
             type Output = Self;
             #[inline]
@@ -232,7 +232,7 @@ macro_rules! define_opaque_mask {
 
         impl<const LANES: usize> core::ops::BitOr<$name<LANES>> for bool
         where
-            $bits_ty: LanesAtMost64,
+            $bits_ty: LanesAtMost32,
         {
             type Output = $name<LANES>;
             #[inline]
@@ -243,7 +243,7 @@ macro_rules! define_opaque_mask {
 
         impl<const LANES: usize> core::ops::BitXor for $name<LANES>
         where
-            $bits_ty: LanesAtMost64,
+            $bits_ty: LanesAtMost32,
         {
             type Output = Self;
             #[inline]
@@ -254,7 +254,7 @@ macro_rules! define_opaque_mask {
 
         impl<const LANES: usize> core::ops::BitXor<bool> for $name<LANES>
         where
-            $bits_ty: LanesAtMost64,
+            $bits_ty: LanesAtMost32,
         {
             type Output = Self;
             #[inline]
@@ -265,7 +265,7 @@ macro_rules! define_opaque_mask {
 
         impl<const LANES: usize> core::ops::BitXor<$name<LANES>> for bool
         where
-            $bits_ty: LanesAtMost64,
+            $bits_ty: LanesAtMost32,
         {
             type Output = $name<LANES>;
             #[inline]
@@ -276,7 +276,7 @@ macro_rules! define_opaque_mask {
 
         impl<const LANES: usize> core::ops::Not for $name<LANES>
         where
-            $bits_ty: LanesAtMost64,
+            $bits_ty: LanesAtMost32,
         {
             type Output = $name<LANES>;
             #[inline]
@@ -287,7 +287,7 @@ macro_rules! define_opaque_mask {
 
         impl<const LANES: usize> core::ops::BitAndAssign for $name<LANES>
         where
-            $bits_ty: LanesAtMost64,
+            $bits_ty: LanesAtMost32,
         {
             #[inline]
             fn bitand_assign(&mut self, rhs: Self) {
@@ -297,7 +297,7 @@ macro_rules! define_opaque_mask {
 
         impl<const LANES: usize> core::ops::BitAndAssign<bool> for $name<LANES>
         where
-            $bits_ty: LanesAtMost64,
+            $bits_ty: LanesAtMost32,
         {
             #[inline]
             fn bitand_assign(&mut self, rhs: bool) {
@@ -307,7 +307,7 @@ macro_rules! define_opaque_mask {
 
         impl<const LANES: usize> core::ops::BitOrAssign for $name<LANES>
         where
-            $bits_ty: LanesAtMost64,
+            $bits_ty: LanesAtMost32,
         {
             #[inline]
             fn bitor_assign(&mut self, rhs: Self) {
@@ -317,7 +317,7 @@ macro_rules! define_opaque_mask {
 
         impl<const LANES: usize> core::ops::BitOrAssign<bool> for $name<LANES>
         where
-            $bits_ty: LanesAtMost64,
+            $bits_ty: LanesAtMost32,
         {
             #[inline]
             fn bitor_assign(&mut self, rhs: bool) {
@@ -327,7 +327,7 @@ macro_rules! define_opaque_mask {
 
         impl<const LANES: usize> core::ops::BitXorAssign for $name<LANES>
         where
-            $bits_ty: LanesAtMost64,
+            $bits_ty: LanesAtMost32,
         {
             #[inline]
             fn bitxor_assign(&mut self, rhs: Self) {
@@ -337,7 +337,7 @@ macro_rules! define_opaque_mask {
 
         impl<const LANES: usize> core::ops::BitXorAssign<bool> for $name<LANES>
         where
-            $bits_ty: LanesAtMost64,
+            $bits_ty: LanesAtMost32,
         {
             #[inline]
             fn bitxor_assign(&mut self, rhs: bool) {
diff --git a/crates/core_simd/src/math.rs b/crates/core_simd/src/math.rs
index eb46feb5c4b4..23ff83f11a1c 100644
--- a/crates/core_simd/src/math.rs
+++ b/crates/core_simd/src/math.rs
@@ -1,6 +1,6 @@
 macro_rules! impl_uint_arith {
     ($(($name:ident, $n:ty)),+) => {
-        $( impl<const LANES: usize> $name<LANES> where Self: crate::LanesAtMost64 {
+        $( impl<const LANES: usize> $name<LANES> where Self: crate::LanesAtMost32 {
 
             /// Lanewise saturating add.
             ///
@@ -42,7 +42,7 @@ macro_rules! impl_uint_arith {
 
 macro_rules! impl_int_arith {
     ($(($name:ident, $n:ty)),+) => {
-        $( impl<const LANES: usize> $name<LANES> where Self: crate::LanesAtMost64 {
+        $( impl<const LANES: usize> $name<LANES> where Self: crate::LanesAtMost32 {
 
             /// Lanewise saturating add.
             ///
diff --git a/crates/core_simd/src/ops.rs b/crates/core_simd/src/ops.rs
index 1d9e1eeb92ce..12d675a06409 100644
--- a/crates/core_simd/src/ops.rs
+++ b/crates/core_simd/src/ops.rs
@@ -1,4 +1,4 @@
-use crate::LanesAtMost64;
+use crate::LanesAtMost32;
 
 /// Checks if the right-hand side argument of a left- or right-shift would cause overflow.
 fn invalid_shift_rhs<T>(rhs: T) -> bool
@@ -16,7 +16,7 @@ macro_rules! impl_ref_ops {
     {
         impl<const $lanes:ident: usize> core::ops::$trait:ident<$rhs:ty> for $type:ty
         where
-            $($bound:path: LanesAtMost64,)*
+            $($bound:path: LanesAtMost32,)*
         {
             type Output = $output:ty;
 
@@ -26,7 +26,7 @@ macro_rules! impl_ref_ops {
     } => {
         impl<const $lanes: usize> core::ops::$trait<$rhs> for $type
         where
-            $($bound: LanesAtMost64,)*
+            $($bound: LanesAtMost32,)*
         {
             type Output = $output;
 
@@ -36,7 +36,7 @@ macro_rules! impl_ref_ops {
 
         impl<const $lanes: usize> core::ops::$trait<&'_ $rhs> for $type
         where
-            $($bound: LanesAtMost64,)*
+            $($bound: LanesAtMost32,)*
         {
             type Output = <$type as core::ops::$trait<$rhs>>::Output;
 
@@ -48,7 +48,7 @@ macro_rules! impl_ref_ops {
 
         impl<const $lanes: usize> core::ops::$trait<$rhs> for &'_ $type
         where
-            $($bound: LanesAtMost64,)*
+            $($bound: LanesAtMost32,)*
         {
             type Output = <$type as core::ops::$trait<$rhs>>::Output;
 
@@ -60,7 +60,7 @@ macro_rules! impl_ref_ops {
 
         impl<const $lanes: usize> core::ops::$trait<&'_ $rhs> for &'_ $type
         where
-            $($bound: LanesAtMost64,)*
+            $($bound: LanesAtMost32,)*
         {
             type Output = <$type as core::ops::$trait<$rhs>>::Output;
 
@@ -75,7 +75,7 @@ macro_rules! impl_ref_ops {
     {
         impl<const $lanes:ident: usize> core::ops::$trait:ident<$rhs:ty> for $type:ty
         where
-            $($bound:path: LanesAtMost64,)*
+            $($bound:path: LanesAtMost32,)*
         {
             $(#[$attrs:meta])*
             fn $fn:ident(&mut $self_tok:ident, $rhs_arg:ident: $rhs_arg_ty:ty) $body:tt
@@ -83,7 +83,7 @@ macro_rules! impl_ref_ops {
     } => {
         impl<const $lanes: usize> core::ops::$trait<$rhs> for $type
         where
-            $($bound: LanesAtMost64,)*
+            $($bound: LanesAtMost32,)*
         {
             $(#[$attrs])*
             fn $fn(&mut $self_tok, $rhs_arg: $rhs_arg_ty) $body
@@ -91,7 +91,7 @@ macro_rules! impl_ref_ops {
 
         impl<const $lanes: usize> core::ops::$trait<&'_ $rhs> for $type
         where
-            $($bound: LanesAtMost64,)*
+            $($bound: LanesAtMost32,)*
         {
             $(#[$attrs])*
             fn $fn(&mut $self_tok, $rhs_arg: &$rhs_arg_ty) {
@@ -104,7 +104,7 @@ macro_rules! impl_ref_ops {
     {
         impl<const $lanes:ident: usize> core::ops::$trait:ident for $type:ty
         where
-            $($bound:path: LanesAtMost64,)*
+            $($bound:path: LanesAtMost32,)*
         {
             type Output = $output:ty;
             fn $fn:ident($self_tok:ident) -> Self::Output $body:tt
@@ -112,7 +112,7 @@ macro_rules! impl_ref_ops {
     } => {
         impl<const $lanes: usize> core::ops::$trait for $type
         where
-            $($bound: LanesAtMost64,)*
+            $($bound: LanesAtMost32,)*
         {
             type Output = $output;
             fn $fn($self_tok) -> Self::Output $body
@@ -120,7 +120,7 @@ macro_rules! impl_ref_ops {
 
         impl<const $lanes: usize> core::ops::$trait for &'_ $type
         where
-            $($bound: LanesAtMost64,)*
+            $($bound: LanesAtMost32,)*
         {
             type Output = <$type as core::ops::$trait>::Output;
             fn $fn($self_tok) -> Self::Output {
@@ -167,7 +167,7 @@ macro_rules! impl_op {
         impl_ref_ops! {
             impl<const LANES: usize> core::ops::Not for crate::$type<LANES>
             where
-                crate::$type<LANES>: LanesAtMost64,
+                crate::$type<LANES>: LanesAtMost32,
             {
                 type Output = Self;
                 fn not(self) -> Self::Output {
@@ -181,7 +181,7 @@ macro_rules! impl_op {
         impl_ref_ops! {
             impl<const LANES: usize> core::ops::Neg for crate::$type<LANES>
             where
-                crate::$type<LANES>: LanesAtMost64,
+                crate::$type<LANES>: LanesAtMost32,
             {
                 type Output = Self;
                 fn neg(self) -> Self::Output {
@@ -195,9 +195,9 @@ macro_rules! impl_op {
         impl_ref_ops! {
             impl<const LANES: usize> core::ops::Neg for crate::$type<LANES>
             where
-                crate::$type<LANES>: LanesAtMost64,
-                crate::SimdU32<LANES>: LanesAtMost64,
-                crate::SimdU64<LANES>: LanesAtMost64,
+                crate::$type<LANES>: LanesAtMost32,
+                crate::SimdU32<LANES>: LanesAtMost32,
+                crate::SimdU64<LANES>: LanesAtMost32,
             {
                 type Output = Self;
                 fn neg(self) -> Self::Output {
@@ -212,7 +212,7 @@ macro_rules! impl_op {
     { impl Index for $type:ident, $scalar:ty } => {
         impl<I, const LANES: usize> core::ops::Index<I> for crate::$type<LANES>
         where
-            Self: LanesAtMost64,
+            Self: LanesAtMost32,
             I: core::slice::SliceIndex<[$scalar]>,
         {
             type Output = I::Output;
@@ -224,7 +224,7 @@ macro_rules! impl_op {
 
         impl<I, const LANES: usize> core::ops::IndexMut<I> for crate::$type<LANES>
         where
-            Self: LanesAtMost64,
+            Self: LanesAtMost32,
             I: core::slice::SliceIndex<[$scalar]>,
         {
             fn index_mut(&mut self, index: I) -> &mut Self::Output {
@@ -239,7 +239,7 @@ macro_rules! impl_op {
         impl_ref_ops! {
             impl<const LANES: usize> core::ops::$trait<Self> for crate::$type<LANES>
             where
-                crate::$type<LANES>: LanesAtMost64,
+                crate::$type<LANES>: LanesAtMost32,
             {
                 type Output = Self;
 
@@ -255,7 +255,7 @@ macro_rules! impl_op {
         impl_ref_ops! {
             impl<const LANES: usize> core::ops::$trait<$scalar> for crate::$type<LANES>
             where
-                crate::$type<LANES>: LanesAtMost64,
+                crate::$type<LANES>: LanesAtMost32,
             {
                 type Output = Self;
 
@@ -269,7 +269,7 @@ macro_rules! impl_op {
         impl_ref_ops! {
             impl<const LANES: usize> core::ops::$trait<crate::$type<LANES>> for $scalar
             where
-                crate::$type<LANES>: LanesAtMost64,
+                crate::$type<LANES>: LanesAtMost32,
             {
                 type Output = crate::$type<LANES>;
 
@@ -283,7 +283,7 @@ macro_rules! impl_op {
         impl_ref_ops! {
             impl<const LANES: usize> core::ops::$assign_trait<Self> for crate::$type<LANES>
             where
-                crate::$type<LANES>: LanesAtMost64,
+                crate::$type<LANES>: LanesAtMost32,
             {
                 #[inline]
                 fn $assign_trait_fn(&mut self, rhs: Self) {
@@ -297,7 +297,7 @@ macro_rules! impl_op {
         impl_ref_ops! {
             impl<const LANES: usize> core::ops::$assign_trait<$scalar> for crate::$type<LANES>
             where
-                crate::$type<LANES>: LanesAtMost64,
+                crate::$type<LANES>: LanesAtMost32,
             {
                 #[inline]
                 fn $assign_trait_fn(&mut self, rhs: $scalar) {
@@ -343,7 +343,7 @@ macro_rules! impl_unsigned_int_ops {
                 impl_ref_ops! {
                     impl<const LANES: usize> core::ops::Div<Self> for crate::$vector<LANES>
                     where
-                        crate::$vector<LANES>: LanesAtMost64,
+                        crate::$vector<LANES>: LanesAtMost32,
                     {
                         type Output = Self;
 
@@ -371,7 +371,7 @@ macro_rules! impl_unsigned_int_ops {
                 impl_ref_ops! {
                     impl<const LANES: usize> core::ops::Div<$scalar> for crate::$vector<LANES>
                     where
-                        crate::$vector<LANES>: LanesAtMost64,
+                        crate::$vector<LANES>: LanesAtMost32,
                     {
                         type Output = Self;
 
@@ -394,7 +394,7 @@ macro_rules! impl_unsigned_int_ops {
                 impl_ref_ops! {
                     impl<const LANES: usize> core::ops::Div<crate::$vector<LANES>> for $scalar
                     where
-                        crate::$vector<LANES>: LanesAtMost64,
+                        crate::$vector<LANES>: LanesAtMost32,
                     {
                         type Output = crate::$vector<LANES>;
 
@@ -408,7 +408,7 @@ macro_rules! impl_unsigned_int_ops {
                 impl_ref_ops! {
                     impl<const LANES: usize> core::ops::DivAssign<Self> for crate::$vector<LANES>
                     where
-                        crate::$vector<LANES>: LanesAtMost64,
+                        crate::$vector<LANES>: LanesAtMost32,
                     {
                         #[inline]
                         fn div_assign(&mut self, rhs: Self) {
@@ -420,7 +420,7 @@ macro_rules! impl_unsigned_int_ops {
                 impl_ref_ops! {
                     impl<const LANES: usize> core::ops::DivAssign<$scalar> for crate::$vector<LANES>
                     where
-                        crate::$vector<LANES>: LanesAtMost64,
+                        crate::$vector<LANES>: LanesAtMost32,
                     {
                         #[inline]
                         fn div_assign(&mut self, rhs: $scalar) {
@@ -433,7 +433,7 @@ macro_rules! impl_unsigned_int_ops {
                 impl_ref_ops! {
                     impl<const LANES: usize> core::ops::Rem<Self> for crate::$vector<LANES>
                     where
-                        crate::$vector<LANES>: LanesAtMost64,
+                        crate::$vector<LANES>: LanesAtMost32,
                     {
                         type Output = Self;
 
@@ -461,7 +461,7 @@ macro_rules! impl_unsigned_int_ops {
                 impl_ref_ops! {
                     impl<const LANES: usize> core::ops::Rem<$scalar> for crate::$vector<LANES>
                     where
-                        crate::$vector<LANES>: LanesAtMost64,
+                        crate::$vector<LANES>: LanesAtMost32,
                     {
                         type Output = Self;
 
@@ -484,7 +484,7 @@ macro_rules! impl_unsigned_int_ops {
                 impl_ref_ops! {
                     impl<const LANES: usize> core::ops::Rem<crate::$vector<LANES>> for $scalar
                     where
-                        crate::$vector<LANES>: LanesAtMost64,
+                        crate::$vector<LANES>: LanesAtMost32,
                     {
                         type Output = crate::$vector<LANES>;
 
@@ -498,7 +498,7 @@ macro_rules! impl_unsigned_int_ops {
                 impl_ref_ops! {
                     impl<const LANES: usize> core::ops::RemAssign<Self> for crate::$vector<LANES>
                     where
-                        crate::$vector<LANES>: LanesAtMost64,
+                        crate::$vector<LANES>: LanesAtMost32,
                     {
                         #[inline]
                         fn rem_assign(&mut self, rhs: Self) {
@@ -510,7 +510,7 @@ macro_rules! impl_unsigned_int_ops {
                 impl_ref_ops! {
                     impl<const LANES: usize> core::ops::RemAssign<$scalar> for crate::$vector<LANES>
                     where
-                        crate::$vector<LANES>: LanesAtMost64,
+                        crate::$vector<LANES>: LanesAtMost32,
                     {
                         #[inline]
                         fn rem_assign(&mut self, rhs: $scalar) {
@@ -523,7 +523,7 @@ macro_rules! impl_unsigned_int_ops {
                 impl_ref_ops! {
                     impl<const LANES: usize> core::ops::Shl<Self> for crate::$vector<LANES>
                     where
-                        crate::$vector<LANES>: LanesAtMost64,
+                        crate::$vector<LANES>: LanesAtMost32,
                     {
                         type Output = Self;
 
@@ -545,7 +545,7 @@ macro_rules! impl_unsigned_int_ops {
                 impl_ref_ops! {
                     impl<const LANES: usize> core::ops::Shl<$scalar> for crate::$vector<LANES>
                     where
-                        crate::$vector<LANES>: LanesAtMost64,
+                        crate::$vector<LANES>: LanesAtMost32,
                     {
                         type Output = Self;
 
@@ -564,7 +564,7 @@ macro_rules! impl_unsigned_int_ops {
                 impl_ref_ops! {
                     impl<const LANES: usize> core::ops::ShlAssign<Self> for crate::$vector<LANES>
                     where
-                        crate::$vector<LANES>: LanesAtMost64,
+                        crate::$vector<LANES>: LanesAtMost32,
                     {
                         #[inline]
                         fn shl_assign(&mut self, rhs: Self) {
@@ -576,7 +576,7 @@ macro_rules! impl_unsigned_int_ops {
                 impl_ref_ops! {
                     impl<const LANES: usize> core::ops::ShlAssign<$scalar> for crate::$vector<LANES>
                     where
-                        crate::$vector<LANES>: LanesAtMost64,
+                        crate::$vector<LANES>: LanesAtMost32,
                     {
                         #[inline]
                         fn shl_assign(&mut self, rhs: $scalar) {
@@ -588,7 +588,7 @@ macro_rules! impl_unsigned_int_ops {
                 impl_ref_ops! {
                     impl<const LANES: usize> core::ops::Shr<Self> for crate::$vector<LANES>
                     where
-                        crate::$vector<LANES>: LanesAtMost64,
+                        crate::$vector<LANES>: LanesAtMost32,
                     {
                         type Output = Self;
 
@@ -610,7 +610,7 @@ macro_rules! impl_unsigned_int_ops {
                 impl_ref_ops! {
                     impl<const LANES: usize> core::ops::Shr<$scalar> for crate::$vector<LANES>
                     where
-                        crate::$vector<LANES>: LanesAtMost64,
+                        crate::$vector<LANES>: LanesAtMost32,
                     {
                         type Output = Self;
 
@@ -629,7 +629,7 @@ macro_rules! impl_unsigned_int_ops {
                 impl_ref_ops! {
                     impl<const LANES: usize> core::ops::ShrAssign<Self> for crate::$vector<LANES>
                     where
-                        crate::$vector<LANES>: LanesAtMost64,
+                        crate::$vector<LANES>: LanesAtMost32,
                     {
                         #[inline]
                         fn shr_assign(&mut self, rhs: Self) {
@@ -641,7 +641,7 @@ macro_rules! impl_unsigned_int_ops {
                 impl_ref_ops! {
                     impl<const LANES: usize> core::ops::ShrAssign<$scalar> for crate::$vector<LANES>
                     where
-                        crate::$vector<LANES>: LanesAtMost64,
+                        crate::$vector<LANES>: LanesAtMost32,
                     {
                         #[inline]
                         fn shr_assign(&mut self, rhs: $scalar) {
diff --git a/crates/core_simd/src/permute.rs b/crates/core_simd/src/permute.rs
index 05a78c3764b8..b27b0a9e1412 100644
--- a/crates/core_simd/src/permute.rs
+++ b/crates/core_simd/src/permute.rs
@@ -24,6 +24,5 @@ macro_rules! impl_shuffle_2pow_lanes {
         impl_shuffle_lane!{ $name, simd_shuffle8, 8 }
         impl_shuffle_lane!{ $name, simd_shuffle16, 16 }
         impl_shuffle_lane!{ $name, simd_shuffle32, 32 }
-        impl_shuffle_lane!{ $name, simd_shuffle64, 64 }
     }
 }
diff --git a/crates/core_simd/src/round.rs b/crates/core_simd/src/round.rs
index dc37130a8ce6..ccad1aad9c44 100644
--- a/crates/core_simd/src/round.rs
+++ b/crates/core_simd/src/round.rs
@@ -4,7 +4,7 @@ macro_rules! implement {
     } => {
         impl<const LANES: usize> crate::$type<LANES>
         where
-            Self: crate::LanesAtMost64,
+            Self: crate::LanesAtMost32,
         {
             /// Returns the largest integer less than or equal to each lane.
             #[cfg(feature = "std")]
@@ -25,8 +25,8 @@ macro_rules! implement {
 
         impl<const LANES: usize> crate::$type<LANES>
         where
-            Self: crate::LanesAtMost64,
-            crate::$int_type<LANES>: crate::LanesAtMost64,
+            Self: crate::LanesAtMost32,
+            crate::$int_type<LANES>: crate::LanesAtMost32,
         {
             /// Rounds toward zero and converts to the same-width integer type, assuming that
             /// the value is finite and fits in that type.
diff --git a/crates/core_simd/src/vector/float.rs b/crates/core_simd/src/vector/float.rs
index bea711a50359..393e39023d9c 100644
--- a/crates/core_simd/src/vector/float.rs
+++ b/crates/core_simd/src/vector/float.rs
@@ -9,8 +9,8 @@ macro_rules! impl_float_vector {
 
         impl<const LANES: usize> $name<LANES>
         where
-            Self: crate::LanesAtMost64,
-            crate::$bits_ty<LANES>: crate::LanesAtMost64,
+            Self: crate::LanesAtMost32,
+            crate::$bits_ty<LANES>: crate::LanesAtMost32,
         {
             /// Raw transmutation to an unsigned integer vector type with the
             /// same size and number of lanes.
@@ -39,9 +39,9 @@ macro_rules! impl_float_vector {
 
         impl<const LANES: usize> $name<LANES>
         where
-            Self: crate::LanesAtMost64,
-            crate::$bits_ty<LANES>: crate::LanesAtMost64,
-            crate::$mask_impl_ty<LANES>: crate::LanesAtMost64,
+            Self: crate::LanesAtMost32,
+            crate::$bits_ty<LANES>: crate::LanesAtMost32,
+            crate::$mask_impl_ty<LANES>: crate::LanesAtMost32,
         {
             /// Returns true for each lane if it has a positive sign, including
             /// `+0.0`, `NaN`s with positive sign bit and positive infinity.
@@ -96,7 +96,7 @@ macro_rules! impl_float_vector {
 #[repr(simd)]
 pub struct SimdF32<const LANES: usize>([f32; LANES])
 where
-    Self: crate::LanesAtMost64;
+    Self: crate::LanesAtMost32;
 
 impl_float_vector! { SimdF32, f32, SimdU32, Mask32, SimdI32 }
 
@@ -108,7 +108,7 @@ from_transmute_x86! { unsafe f32x8 => __m256 }
 #[repr(simd)]
 pub struct SimdF64<const LANES: usize>([f64; LANES])
 where
-    Self: crate::LanesAtMost64;
+    Self: crate::LanesAtMost32;
 
 impl_float_vector! { SimdF64, f64, SimdU64, Mask64, SimdI64 }
 
diff --git a/crates/core_simd/src/vector/int.rs b/crates/core_simd/src/vector/int.rs
index 364a2ed935fb..5304d11cd6ea 100644
--- a/crates/core_simd/src/vector/int.rs
+++ b/crates/core_simd/src/vector/int.rs
@@ -5,9 +5,9 @@ macro_rules! impl_integer_vector {
     { $name:ident, $type:ty, $mask_ty:ident, $mask_impl_ty:ident } => {
         impl_vector! { $name, $type }
 
-        impl<const LANES: usize> Eq for $name<LANES> where Self: crate::LanesAtMost64 {}
+        impl<const LANES: usize> Eq for $name<LANES> where Self: crate::LanesAtMost32 {}
 
-        impl<const LANES: usize> Ord for $name<LANES> where Self: crate::LanesAtMost64 {
+        impl<const LANES: usize> Ord for $name<LANES> where Self: crate::LanesAtMost32 {
             #[inline]
             fn cmp(&self, other: &Self) -> core::cmp::Ordering {
                 // TODO use SIMD cmp
@@ -15,7 +15,7 @@ macro_rules! impl_integer_vector {
             }
         }
 
-        impl<const LANES: usize> core::hash::Hash for $name<LANES> where Self: crate::LanesAtMost64 {
+        impl<const LANES: usize> core::hash::Hash for $name<LANES> where Self: crate::LanesAtMost32 {
             #[inline]
             fn hash<H>(&self, state: &mut H)
             where
@@ -27,8 +27,8 @@ macro_rules! impl_integer_vector {
 
         impl<const LANES: usize> $name<LANES>
         where
-            Self: crate::LanesAtMost64,
-            crate::$mask_impl_ty<LANES>: crate::LanesAtMost64,
+            Self: crate::LanesAtMost32,
+            crate::$mask_impl_ty<LANES>: crate::LanesAtMost32,
         {
             /// Returns true for each positive lane and false if it is zero or negative.
             pub fn is_positive(self) -> crate::$mask_ty<LANES> {
@@ -47,7 +47,7 @@ macro_rules! impl_integer_vector {
 #[repr(simd)]
 pub struct SimdIsize<const LANES: usize>([isize; LANES])
 where
-    Self: crate::LanesAtMost64;
+    Self: crate::LanesAtMost32;
 
 impl_integer_vector! { SimdIsize, isize, MaskSize, SimdIsize }
 
@@ -67,7 +67,7 @@ from_transmute_x86! { unsafe isizex4 => __m256i }
 #[repr(simd)]
 pub struct SimdI128<const LANES: usize>([i128; LANES])
 where
-    Self: crate::LanesAtMost64;
+    Self: crate::LanesAtMost32;
 
 impl_integer_vector! { SimdI128, i128, Mask128, SimdI128 }
 
@@ -78,7 +78,7 @@ from_transmute_x86! { unsafe i128x2 => __m256i }
 #[repr(simd)]
 pub struct SimdI16<const LANES: usize>([i16; LANES])
 where
-    Self: crate::LanesAtMost64;
+    Self: crate::LanesAtMost32;
 
 impl_integer_vector! { SimdI16, i16, Mask16, SimdI16 }
 
@@ -90,7 +90,7 @@ from_transmute_x86! { unsafe i16x16 => __m256i }
 #[repr(simd)]
 pub struct SimdI32<const LANES: usize>([i32; LANES])
 where
-    Self: crate::LanesAtMost64;
+    Self: crate::LanesAtMost32;
 
 impl_integer_vector! { SimdI32, i32, Mask32, SimdI32 }
 
@@ -102,7 +102,7 @@ from_transmute_x86! { unsafe i32x8 => __m256i }
 #[repr(simd)]
 pub struct SimdI64<const LANES: usize>([i64; LANES])
 where
-    Self: crate::LanesAtMost64;
+    Self: crate::LanesAtMost32;
 
 impl_integer_vector! { SimdI64, i64, Mask64, SimdI64 }
 
@@ -114,7 +114,7 @@ from_transmute_x86! { unsafe i64x4 => __m256i }
 #[repr(simd)]
 pub struct SimdI8<const LANES: usize>([i8; LANES])
 where
-    Self: crate::LanesAtMost64;
+    Self: crate::LanesAtMost32;
 
 impl_integer_vector! { SimdI8, i8, Mask8, SimdI8 }
 
diff --git a/crates/core_simd/src/vector/uint.rs b/crates/core_simd/src/vector/uint.rs
index 0f7a47eee30e..71b5b295112a 100644
--- a/crates/core_simd/src/vector/uint.rs
+++ b/crates/core_simd/src/vector/uint.rs
@@ -6,9 +6,9 @@ macro_rules! impl_unsigned_vector {
     { $name:ident, $type:ty } => {
         impl_vector! { $name, $type }
 
-        impl<const LANES: usize> Eq for $name<LANES> where Self: crate::LanesAtMost64 {}
+        impl<const LANES: usize> Eq for $name<LANES> where Self: crate::LanesAtMost32 {}
 
-        impl<const LANES: usize> Ord for $name<LANES> where Self: crate::LanesAtMost64 {
+        impl<const LANES: usize> Ord for $name<LANES> where Self: crate::LanesAtMost32 {
             #[inline]
             fn cmp(&self, other: &Self) -> core::cmp::Ordering {
                 // TODO use SIMD cmp
@@ -16,7 +16,7 @@ macro_rules! impl_unsigned_vector {
             }
         }
 
-        impl<const LANES: usize> core::hash::Hash for $name<LANES> where Self: crate::LanesAtMost64 {
+        impl<const LANES: usize> core::hash::Hash for $name<LANES> where Self: crate::LanesAtMost32 {
             #[inline]
             fn hash<H>(&self, state: &mut H)
             where
@@ -32,7 +32,7 @@ macro_rules! impl_unsigned_vector {
 #[repr(simd)]
 pub struct SimdUsize<const LANES: usize>([usize; LANES])
 where
-    Self: crate::LanesAtMost64;
+    Self: crate::LanesAtMost32;
 
 impl_unsigned_vector! { SimdUsize, usize }
 
@@ -52,7 +52,7 @@ from_transmute_x86! { unsafe usizex4 => __m256i }
 #[repr(simd)]
 pub struct SimdU128<const LANES: usize>([u128; LANES])
 where
-    Self: crate::LanesAtMost64;
+    Self: crate::LanesAtMost32;
 
 impl_unsigned_vector! { SimdU128, u128 }
 
@@ -63,7 +63,7 @@ from_transmute_x86! { unsafe u128x2 => __m256i }
 #[repr(simd)]
 pub struct SimdU16<const LANES: usize>([u16; LANES])
 where
-    Self: crate::LanesAtMost64;
+    Self: crate::LanesAtMost32;
 
 impl_unsigned_vector! { SimdU16, u16 }
 
@@ -75,7 +75,7 @@ from_transmute_x86! { unsafe u16x16 => __m256i }
 #[repr(simd)]
 pub struct SimdU32<const LANES: usize>([u32; LANES])
 where
-    Self: crate::LanesAtMost64;
+    Self: crate::LanesAtMost32;
 
 impl_unsigned_vector! { SimdU32, u32 }
 
@@ -87,7 +87,7 @@ from_transmute_x86! { unsafe u32x8 => __m256i }
 #[repr(simd)]
 pub struct SimdU64<const LANES: usize>([u64; LANES])
 where
-    Self: crate::LanesAtMost64;
+    Self: crate::LanesAtMost32;
 
 impl_unsigned_vector! { SimdU64, u64 }
 
@@ -99,7 +99,7 @@ from_transmute_x86! { unsafe u64x4 => __m256i }
 #[repr(simd)]
 pub struct SimdU8<const LANES: usize>([u8; LANES])
 where
-    Self: crate::LanesAtMost64;
+    Self: crate::LanesAtMost32;
 
 impl_unsigned_vector! { SimdU8, u8 }
 
diff --git a/crates/core_simd/tests/mask_ops_impl/mask8.rs b/crates/core_simd/tests/mask_ops_impl/mask8.rs
index 218fa9fe895e..9c06fbc0411d 100644
--- a/crates/core_simd/tests/mask_ops_impl/mask8.rs
+++ b/crates/core_simd/tests/mask_ops_impl/mask8.rs
@@ -1,4 +1,3 @@
 mask_tests! { mask8x8, 8 }
 mask_tests! { mask8x16, 16 }
 mask_tests! { mask8x32, 32 }
-mask_tests! { mask8x64, 64 }
diff --git a/crates/test_helpers/src/lib.rs b/crates/test_helpers/src/lib.rs
index df1abcddaea1..b5bfd96dde85 100644
--- a/crates/test_helpers/src/lib.rs
+++ b/crates/test_helpers/src/lib.rs
@@ -269,21 +269,21 @@ macro_rules! test_lanes {
 
                 fn implementation<const $lanes: usize>()
                 where
-                    core_simd::SimdU8<$lanes>: core_simd::LanesAtMost64,
-                    core_simd::SimdU16<$lanes>: core_simd::LanesAtMost64,
-                    core_simd::SimdU32<$lanes>: core_simd::LanesAtMost64,
-                    core_simd::SimdU64<$lanes>: core_simd::LanesAtMost64,
-                    core_simd::SimdU128<$lanes>: core_simd::LanesAtMost64,
-                    core_simd::SimdUsize<$lanes>: core_simd::LanesAtMost64,
-                    core_simd::SimdI8<$lanes>: core_simd::LanesAtMost64,
-                    core_simd::SimdI16<$lanes>: core_simd::LanesAtMost64,
-                    core_simd::SimdI32<$lanes>: core_simd::LanesAtMost64,
-                    core_simd::SimdI64<$lanes>: core_simd::LanesAtMost64,
-                    core_simd::SimdI128<$lanes>: core_simd::LanesAtMost64,
-                    core_simd::SimdIsize<$lanes>: core_simd::LanesAtMost64,
-                    core_simd::SimdF32<$lanes>: core_simd::LanesAtMost64,
-                    core_simd::SimdF64<$lanes>: core_simd::LanesAtMost64,
-                    core_simd::BitMask<$lanes>: core_simd::LanesAtMost64,
+                    core_simd::SimdU8<$lanes>: core_simd::LanesAtMost32,
+                    core_simd::SimdU16<$lanes>: core_simd::LanesAtMost32,
+                    core_simd::SimdU32<$lanes>: core_simd::LanesAtMost32,
+                    core_simd::SimdU64<$lanes>: core_simd::LanesAtMost32,
+                    core_simd::SimdU128<$lanes>: core_simd::LanesAtMost32,
+                    core_simd::SimdUsize<$lanes>: core_simd::LanesAtMost32,
+                    core_simd::SimdI8<$lanes>: core_simd::LanesAtMost32,
+                    core_simd::SimdI16<$lanes>: core_simd::LanesAtMost32,
+                    core_simd::SimdI32<$lanes>: core_simd::LanesAtMost32,
+                    core_simd::SimdI64<$lanes>: core_simd::LanesAtMost32,
+                    core_simd::SimdI128<$lanes>: core_simd::LanesAtMost32,
+                    core_simd::SimdIsize<$lanes>: core_simd::LanesAtMost32,
+                    core_simd::SimdF32<$lanes>: core_simd::LanesAtMost32,
+                    core_simd::SimdF64<$lanes>: core_simd::LanesAtMost32,
+                    core_simd::BitMask<$lanes>: core_simd::LanesAtMost32,
                 $body
 
                 #[cfg(target_arch = "wasm32")]
@@ -324,12 +324,6 @@ macro_rules! test_lanes {
                 fn lanes_32() {
                     implementation::<32>();
                 }
-
-                #[test]
-                #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test::wasm_bindgen_test)]
-                fn lanes_64() {
-                    implementation::<64>();
-                }
             }
         )*
     }
@@ -347,21 +341,21 @@ macro_rules! test_lanes_panic {
 
                 fn implementation<const $lanes: usize>()
                 where
-                    core_simd::SimdU8<$lanes>: core_simd::LanesAtMost64,
-                    core_simd::SimdU16<$lanes>: core_simd::LanesAtMost64,
-                    core_simd::SimdU32<$lanes>: core_simd::LanesAtMost64,
-                    core_simd::SimdU64<$lanes>: core_simd::LanesAtMost64,
-                    core_simd::SimdU128<$lanes>: core_simd::LanesAtMost64,
-                    core_simd::SimdUsize<$lanes>: core_simd::LanesAtMost64,
-                    core_simd::SimdI8<$lanes>: core_simd::LanesAtMost64,
-                    core_simd::SimdI16<$lanes>: core_simd::LanesAtMost64,
-                    core_simd::SimdI32<$lanes>: core_simd::LanesAtMost64,
-                    core_simd::SimdI64<$lanes>: core_simd::LanesAtMost64,
-                    core_simd::SimdI128<$lanes>: core_simd::LanesAtMost64,
-                    core_simd::SimdIsize<$lanes>: core_simd::LanesAtMost64,
-                    core_simd::SimdF32<$lanes>: core_simd::LanesAtMost64,
-                    core_simd::SimdF64<$lanes>: core_simd::LanesAtMost64,
-                    core_simd::BitMask<$lanes>: core_simd::LanesAtMost64,
+                    core_simd::SimdU8<$lanes>: core_simd::LanesAtMost32,
+                    core_simd::SimdU16<$lanes>: core_simd::LanesAtMost32,
+                    core_simd::SimdU32<$lanes>: core_simd::LanesAtMost32,
+                    core_simd::SimdU64<$lanes>: core_simd::LanesAtMost32,
+                    core_simd::SimdU128<$lanes>: core_simd::LanesAtMost32,
+                    core_simd::SimdUsize<$lanes>: core_simd::LanesAtMost32,
+                    core_simd::SimdI8<$lanes>: core_simd::LanesAtMost32,
+                    core_simd::SimdI16<$lanes>: core_simd::LanesAtMost32,
+                    core_simd::SimdI32<$lanes>: core_simd::LanesAtMost32,
+                    core_simd::SimdI64<$lanes>: core_simd::LanesAtMost32,
+                    core_simd::SimdI128<$lanes>: core_simd::LanesAtMost32,
+                    core_simd::SimdIsize<$lanes>: core_simd::LanesAtMost32,
+                    core_simd::SimdF32<$lanes>: core_simd::LanesAtMost32,
+                    core_simd::SimdF64<$lanes>: core_simd::LanesAtMost32,
+                    core_simd::BitMask<$lanes>: core_simd::LanesAtMost32,
                 $body
 
                 #[test]
@@ -399,12 +393,6 @@ macro_rules! test_lanes_panic {
                 fn lanes_32() {
                     implementation::<32>();
                 }
-
-                #[test]
-                #[should_panic]
-                fn lanes_64() {
-                    implementation::<64>();
-                }
             }
         )*
     }

From b0a005dcfbbf4d395e4506963d5ab81877a226d2 Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Sun, 14 Feb 2021 23:35:24 -0500
Subject: [PATCH 112/249] Add floating-point classification functions

---
 crates/core_simd/src/lib.rs | 1 +
 1 file changed, 1 insertion(+)

diff --git a/crates/core_simd/src/lib.rs b/crates/core_simd/src/lib.rs
index 906ee3f06ae5..8d9fccd238a6 100644
--- a/crates/core_simd/src/lib.rs
+++ b/crates/core_simd/src/lib.rs
@@ -17,6 +17,7 @@ mod fmt;
 mod intrinsics;
 mod ops;
 mod round;
+mod comparisons;
 
 mod math;
 

From d7649f46f3f562960f1a87b93e61a35dcd0cc857 Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Sat, 6 Mar 2021 02:14:58 -0500
Subject: [PATCH 113/249] Various bug fixes

---
 crates/core_simd/src/lib.rs | 1 -
 1 file changed, 1 deletion(-)

diff --git a/crates/core_simd/src/lib.rs b/crates/core_simd/src/lib.rs
index 8d9fccd238a6..906ee3f06ae5 100644
--- a/crates/core_simd/src/lib.rs
+++ b/crates/core_simd/src/lib.rs
@@ -17,7 +17,6 @@ mod fmt;
 mod intrinsics;
 mod ops;
 mod round;
-mod comparisons;
 
 mod math;
 

From 926cf3aba3fe453e36bc7e56b2b8b8894fca5377 Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Sun, 7 Mar 2021 19:45:31 -0500
Subject: [PATCH 114/249] Add intrinsics

---
 crates/core_simd/src/intrinsics.rs | 11 +++++++++++
 1 file changed, 11 insertions(+)

diff --git a/crates/core_simd/src/intrinsics.rs b/crates/core_simd/src/intrinsics.rs
index fafeed6a62a8..13cda880a6c1 100644
--- a/crates/core_simd/src/intrinsics.rs
+++ b/crates/core_simd/src/intrinsics.rs
@@ -67,4 +67,15 @@ extern "platform-intrinsic" {
 
     // {s,u}sub.sat
     pub(crate) fn simd_saturating_sub<T>(x: T, y: T) -> T;
+
+    // reductions
+    pub(crate) fn simd_reduce_add_ordered<T, U>(x: T, y: U) -> U;
+    pub(crate) fn simd_reduce_mul_ordered<T, U>(x: T, y: U) -> U;
+    pub(crate) fn simd_reduce_all<T>(x: T) -> bool;
+    pub(crate) fn simd_reduce_any<T>(x: T) -> bool;
+    pub(crate) fn simd_reduce_max<T, U>(x: T) -> U;
+    pub(crate) fn simd_reduce_min<T, U>(x: T) -> U;
+    pub(crate) fn simd_reduce_and<T, U>(x: T) -> U;
+    pub(crate) fn simd_reduce_or<T, U>(x: T) -> U;
+    pub(crate) fn simd_reduce_xor<T, U>(x: T) -> U;
 }

From 875b31c33f6b0ccbb8590c2b3c9cbf1b11ed6165 Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Sun, 7 Mar 2021 21:15:56 -0500
Subject: [PATCH 115/249] Implement reductions

---
 crates/core_simd/src/lib.rs              |   2 +
 crates/core_simd/src/masks/bitmask.rs    |   2 +-
 crates/core_simd/src/masks/full_masks.rs | 130 ++++++++++++---------
 crates/core_simd/src/masks/mod.rs        | 128 ++++++++++----------
 crates/core_simd/src/reduction.rs        | 142 +++++++++++++++++++++++
 crates/core_simd/src/vector/float.rs     |   1 +
 crates/core_simd/src/vector/int.rs       |   1 +
 crates/core_simd/src/vector/uint.rs      |   1 +
 8 files changed, 289 insertions(+), 118 deletions(-)
 create mode 100644 crates/core_simd/src/reduction.rs

diff --git a/crates/core_simd/src/lib.rs b/crates/core_simd/src/lib.rs
index 906ee3f06ae5..0fc2641516dd 100644
--- a/crates/core_simd/src/lib.rs
+++ b/crates/core_simd/src/lib.rs
@@ -11,6 +11,8 @@ mod first;
 mod permute;
 #[macro_use]
 mod transmute;
+#[macro_use]
+mod reduction;
 
 mod comparisons;
 mod fmt;
diff --git a/crates/core_simd/src/masks/bitmask.rs b/crates/core_simd/src/masks/bitmask.rs
index d7400699fde8..1d25db46742f 100644
--- a/crates/core_simd/src/masks/bitmask.rs
+++ b/crates/core_simd/src/masks/bitmask.rs
@@ -3,7 +3,7 @@ use crate::LanesAtMost32;
 /// A mask where each lane is represented by a single bit.
 #[derive(Copy, Clone, Debug)]
 #[repr(transparent)]
-pub struct BitMask<const LANES: usize>(u64)
+pub struct BitMask<const LANES: usize>(pub(crate) u64)
 where
     BitMask<LANES>: LanesAtMost32;
 
diff --git a/crates/core_simd/src/masks/full_masks.rs b/crates/core_simd/src/masks/full_masks.rs
index cca077b14d04..a6689ce48c66 100644
--- a/crates/core_simd/src/masks/full_masks.rs
+++ b/crates/core_simd/src/masks/full_masks.rs
@@ -14,22 +14,27 @@ impl core::fmt::Display for TryFromMaskError {
 }
 
 macro_rules! define_mask {
-    { $(#[$attr:meta])* struct $name:ident<const $lanes:ident: usize>($type:ty); } => {
+    {
+        $(#[$attr:meta])*
+        struct $name:ident<const $lanes:ident: usize>(
+            crate::$type:ident<$lanes2:ident>
+        );
+    } => {
         $(#[$attr])*
         #[derive(Default, PartialEq, PartialOrd, Eq, Ord, Hash)]
         #[repr(transparent)]
-        pub struct $name<const $lanes: usize>($type)
+        pub struct $name<const $lanes: usize>(crate::$type<$lanes2>)
         where
-            $type: crate::LanesAtMost32;
+            crate::$type<LANES>: crate::LanesAtMost32;
 
         impl<const LANES: usize> Copy for $name<LANES>
         where
-            $type: crate::LanesAtMost32,
+            crate::$type<LANES>: crate::LanesAtMost32,
         {}
 
         impl<const LANES: usize> Clone for $name<LANES>
         where
-            $type: crate::LanesAtMost32,
+            crate::$type<LANES>: crate::LanesAtMost32,
         {
             #[inline]
             fn clone(&self) -> Self {
@@ -37,13 +42,13 @@ macro_rules! define_mask {
             }
         }
 
-        impl<const $lanes: usize> $name<$lanes>
+        impl<const LANES: usize> $name<LANES>
         where
-            $type: crate::LanesAtMost32,
+            crate::$type<LANES>: crate::LanesAtMost32,
         {
             /// Construct a mask by setting all lanes to the given value.
             pub fn splat(value: bool) -> Self {
-                Self(<$type>::splat(
+                Self(<crate::$type<LANES>>::splat(
                     if value {
                         -1
                     } else {
@@ -76,42 +81,51 @@ macro_rules! define_mask {
                 }
             }
 
-            /// Creates a mask from an integer vector.
-            ///
-            /// # Safety
-            /// All lanes must be either 0 or -1.
+            /// Converts the mask to the equivalent integer representation, where -1 represents
+            /// "set" and 0 represents "unset".
             #[inline]
-            pub unsafe fn from_int_unchecked(value: $type) -> Self {
+            pub fn to_int(self) -> crate::$type<LANES> {
+                self.0
+            }
+
+            /// Creates a  mask from the equivalent integer representation, where -1 represents
+            /// "set" and 0 represents "unset".
+            ///
+            /// Each provided lane must be either 0 or -1.
+            #[inline]
+            pub unsafe fn from_int_unchecked(value: crate::$type<LANES>) -> Self {
                 Self(value)
             }
 
-            /// Creates a mask from an integer vector.
+            /// Creates a mask from the equivalent integer representation, where -1 represents
+            /// "set" and 0 represents "unset".
             ///
             /// # Panics
             /// Panics if any lane is not 0 or -1.
             #[inline]
-            pub fn from_int(value: $type) -> Self {
+            pub fn from_int(value: crate::$type<LANES>) -> Self {
                 use core::convert::TryInto;
                 value.try_into().unwrap()
             }
         }
 
-        impl<const $lanes: usize> core::convert::From<bool> for $name<$lanes>
+        impl<const LANES: usize> core::convert::From<bool> for $name<LANES>
         where
-            $type: crate::LanesAtMost32,
+            crate::$type<LANES>: crate::LanesAtMost32,
         {
             fn from(value: bool) -> Self {
                 Self::splat(value)
             }
         }
 
-        impl<const $lanes: usize> core::convert::TryFrom<$type> for $name<$lanes>
+        impl<const LANES: usize> core::convert::TryFrom<crate::$type<LANES>> for $name<LANES>
         where
-            $type: crate::LanesAtMost32,
+            crate::$type<LANES>: crate::LanesAtMost32,
         {
             type Error = TryFromMaskError;
-            fn try_from(value: $type) -> Result<Self, Self::Error> {
-                if value.as_slice().iter().all(|x| *x == 0 || *x == -1) {
+            fn try_from(value: crate::$type<LANES>) -> Result<Self, Self::Error> {
+                let valid = (value.lanes_eq(crate::$type::<LANES>::splat(0)) | value.lanes_eq(crate::$type::<LANES>::splat(-1))).all();
+                if valid {
                     Ok(Self(value))
                 } else {
                     Err(TryFromMaskError(()))
@@ -119,21 +133,21 @@ macro_rules! define_mask {
             }
         }
 
-        impl<const $lanes: usize> core::convert::From<$name<$lanes>> for $type
+        impl<const LANES: usize> core::convert::From<$name<LANES>> for crate::$type<LANES>
         where
-            $type: crate::LanesAtMost32,
+            crate::$type<LANES>: crate::LanesAtMost32,
         {
-            fn from(value: $name<$lanes>) -> Self {
+            fn from(value: $name<LANES>) -> Self {
                 value.0
             }
         }
 
-        impl<const $lanes: usize> core::convert::From<crate::BitMask<$lanes>> for $name<$lanes>
+        impl<const LANES: usize> core::convert::From<crate::BitMask<LANES>> for $name<LANES>
         where
-            $type: crate::LanesAtMost32,
-            crate::BitMask<$lanes>: crate::LanesAtMost32,
+            crate::$type<LANES>: crate::LanesAtMost32,
+            crate::BitMask<LANES>: crate::LanesAtMost32,
         {
-            fn from(value: crate::BitMask<$lanes>) -> Self {
+            fn from(value: crate::BitMask<LANES>) -> Self {
                 // TODO use an intrinsic to do this efficiently (with LLVM's sext instruction)
                 let mut mask = Self::splat(false);
                 for lane in 0..LANES {
@@ -143,10 +157,10 @@ macro_rules! define_mask {
             }
         }
 
-        impl<const $lanes: usize> core::convert::From<$name<$lanes>> for crate::BitMask<$lanes>
+        impl<const LANES: usize> core::convert::From<$name<LANES>> for crate::BitMask<LANES>
         where
-            $type: crate::LanesAtMost32,
-            crate::BitMask<$lanes>: crate::LanesAtMost32,
+            crate::$type<LANES>: crate::LanesAtMost32,
+            crate::BitMask<LANES>: crate::LanesAtMost32,
         {
             fn from(value: $name<$lanes>) -> Self {
                 // TODO use an intrinsic to do this efficiently (with LLVM's trunc instruction)
@@ -158,9 +172,9 @@ macro_rules! define_mask {
             }
         }
 
-        impl<const $lanes: usize> core::fmt::Debug for $name<$lanes>
+        impl<const LANES: usize> core::fmt::Debug for $name<LANES>
         where
-            $type: crate::LanesAtMost32,
+            crate::$type<LANES>: crate::LanesAtMost32,
         {
             fn fmt(&self, f: &mut core::fmt::Formatter) -> core::fmt::Result {
                 f.debug_list()
@@ -169,36 +183,36 @@ macro_rules! define_mask {
             }
         }
 
-        impl<const $lanes: usize> core::fmt::Binary for $name<$lanes>
+        impl<const LANES: usize> core::fmt::Binary for $name<LANES>
         where
-            $type: crate::LanesAtMost32,
+            crate::$type<LANES>: crate::LanesAtMost32,
         {
             fn fmt(&self, f: &mut core::fmt::Formatter) -> core::fmt::Result {
                 core::fmt::Binary::fmt(&self.0, f)
             }
         }
 
-        impl<const $lanes: usize> core::fmt::Octal for $name<$lanes>
+        impl<const LANES: usize> core::fmt::Octal for $name<LANES>
         where
-            $type: crate::LanesAtMost32,
+            crate::$type<LANES>: crate::LanesAtMost32,
         {
             fn fmt(&self, f: &mut core::fmt::Formatter) -> core::fmt::Result {
                 core::fmt::Octal::fmt(&self.0, f)
             }
         }
 
-        impl<const $lanes: usize> core::fmt::LowerHex for $name<$lanes>
+        impl<const LANES: usize> core::fmt::LowerHex for $name<LANES>
         where
-            $type: crate::LanesAtMost32,
+            crate::$type<LANES>: crate::LanesAtMost32,
         {
             fn fmt(&self, f: &mut core::fmt::Formatter) -> core::fmt::Result {
                 core::fmt::LowerHex::fmt(&self.0, f)
             }
         }
 
-        impl<const $lanes: usize> core::fmt::UpperHex for $name<$lanes>
+        impl<const LANES: usize> core::fmt::UpperHex for $name<LANES>
         where
-            $type: crate::LanesAtMost32,
+            crate::$type<LANES>: crate::LanesAtMost32,
         {
             fn fmt(&self, f: &mut core::fmt::Formatter) -> core::fmt::Result {
                 core::fmt::UpperHex::fmt(&self.0, f)
@@ -207,7 +221,7 @@ macro_rules! define_mask {
 
         impl<const LANES: usize> core::ops::BitAnd for $name<LANES>
         where
-            $type: crate::LanesAtMost32,
+            crate::$type<LANES>: crate::LanesAtMost32,
         {
             type Output = Self;
             #[inline]
@@ -218,7 +232,7 @@ macro_rules! define_mask {
 
         impl<const LANES: usize> core::ops::BitAnd<bool> for $name<LANES>
         where
-            $type: crate::LanesAtMost32,
+            crate::$type<LANES>: crate::LanesAtMost32,
         {
             type Output = Self;
             #[inline]
@@ -229,7 +243,7 @@ macro_rules! define_mask {
 
         impl<const LANES: usize> core::ops::BitAnd<$name<LANES>> for bool
         where
-            $type: crate::LanesAtMost32,
+            crate::$type<LANES>: crate::LanesAtMost32,
         {
             type Output = $name<LANES>;
             #[inline]
@@ -240,7 +254,7 @@ macro_rules! define_mask {
 
         impl<const LANES: usize> core::ops::BitOr for $name<LANES>
         where
-            $type: crate::LanesAtMost32,
+            crate::$type<LANES>: crate::LanesAtMost32,
         {
             type Output = Self;
             #[inline]
@@ -251,7 +265,7 @@ macro_rules! define_mask {
 
         impl<const LANES: usize> core::ops::BitOr<bool> for $name<LANES>
         where
-            $type: crate::LanesAtMost32,
+            crate::$type<LANES>: crate::LanesAtMost32,
         {
             type Output = Self;
             #[inline]
@@ -262,7 +276,7 @@ macro_rules! define_mask {
 
         impl<const LANES: usize> core::ops::BitOr<$name<LANES>> for bool
         where
-            $type: crate::LanesAtMost32,
+            crate::$type<LANES>: crate::LanesAtMost32,
         {
             type Output = $name<LANES>;
             #[inline]
@@ -273,7 +287,7 @@ macro_rules! define_mask {
 
         impl<const LANES: usize> core::ops::BitXor for $name<LANES>
         where
-            $type: crate::LanesAtMost32,
+            crate::$type<LANES>: crate::LanesAtMost32,
         {
             type Output = Self;
             #[inline]
@@ -284,7 +298,7 @@ macro_rules! define_mask {
 
         impl<const LANES: usize> core::ops::BitXor<bool> for $name<LANES>
         where
-            $type: crate::LanesAtMost32,
+            crate::$type<LANES>: crate::LanesAtMost32,
         {
             type Output = Self;
             #[inline]
@@ -295,7 +309,7 @@ macro_rules! define_mask {
 
         impl<const LANES: usize> core::ops::BitXor<$name<LANES>> for bool
         where
-            $type: crate::LanesAtMost32,
+            crate::$type<LANES>: crate::LanesAtMost32,
         {
             type Output = $name<LANES>;
             #[inline]
@@ -306,7 +320,7 @@ macro_rules! define_mask {
 
         impl<const LANES: usize> core::ops::Not for $name<LANES>
         where
-            $type: crate::LanesAtMost32,
+            crate::$type<LANES>: crate::LanesAtMost32,
         {
             type Output = $name<LANES>;
             #[inline]
@@ -317,7 +331,7 @@ macro_rules! define_mask {
 
         impl<const LANES: usize> core::ops::BitAndAssign for $name<LANES>
         where
-            $type: crate::LanesAtMost32,
+            crate::$type<LANES>: crate::LanesAtMost32,
         {
             #[inline]
             fn bitand_assign(&mut self, rhs: Self) {
@@ -327,7 +341,7 @@ macro_rules! define_mask {
 
         impl<const LANES: usize> core::ops::BitAndAssign<bool> for $name<LANES>
         where
-            $type: crate::LanesAtMost32,
+            crate::$type<LANES>: crate::LanesAtMost32,
         {
             #[inline]
             fn bitand_assign(&mut self, rhs: bool) {
@@ -337,7 +351,7 @@ macro_rules! define_mask {
 
         impl<const LANES: usize> core::ops::BitOrAssign for $name<LANES>
         where
-            $type: crate::LanesAtMost32,
+            crate::$type<LANES>: crate::LanesAtMost32,
         {
             #[inline]
             fn bitor_assign(&mut self, rhs: Self) {
@@ -347,7 +361,7 @@ macro_rules! define_mask {
 
         impl<const LANES: usize> core::ops::BitOrAssign<bool> for $name<LANES>
         where
-            $type: crate::LanesAtMost32,
+            crate::$type<LANES>: crate::LanesAtMost32,
         {
             #[inline]
             fn bitor_assign(&mut self, rhs: bool) {
@@ -357,7 +371,7 @@ macro_rules! define_mask {
 
         impl<const LANES: usize> core::ops::BitXorAssign for $name<LANES>
         where
-            $type: crate::LanesAtMost32,
+            crate::$type<LANES>: crate::LanesAtMost32,
         {
             #[inline]
             fn bitxor_assign(&mut self, rhs: Self) {
@@ -367,13 +381,15 @@ macro_rules! define_mask {
 
         impl<const LANES: usize> core::ops::BitXorAssign<bool> for $name<LANES>
         where
-            $type: crate::LanesAtMost32,
+            crate::$type<LANES>: crate::LanesAtMost32,
         {
             #[inline]
             fn bitxor_assign(&mut self, rhs: bool) {
                 *self ^= Self::splat(rhs);
             }
         }
+
+        impl_full_mask_reductions! { $name, $type }
     }
 }
 
diff --git a/crates/core_simd/src/masks/mod.rs b/crates/core_simd/src/masks/mod.rs
index 0b986aaf7e17..4503187e4b8a 100644
--- a/crates/core_simd/src/masks/mod.rs
+++ b/crates/core_simd/src/masks/mod.rs
@@ -7,25 +7,27 @@ pub use full_masks::*;
 mod bitmask;
 pub use bitmask::*;
 
-use crate::LanesAtMost32;
+use crate::{LanesAtMost32, SimdI128, SimdI16, SimdI32, SimdI64, SimdI8, SimdIsize};
 
 macro_rules! define_opaque_mask {
     {
         $(#[$attr:meta])*
-        struct $name:ident<const $lanes:ident: usize>($inner_ty:ty);
-        @bits $bits_ty:ty
+        struct $name:ident<const $lanes:ident: usize>($inner_ty:ident<$lanes2:ident>);
+        @bits $bits_ty:ident
     } => {
         $(#[$attr])*
         #[allow(non_camel_case_types)]
-        pub struct $name<const $lanes: usize>($inner_ty) where $bits_ty: LanesAtMost32;
+        pub struct $name<const LANES: usize>($inner_ty<LANES>) where $bits_ty<LANES>: LanesAtMost32;
 
-        impl<const $lanes: usize> $name<$lanes>
+        impl_opaque_mask_reductions! { $name, $inner_ty, $bits_ty }
+
+        impl<const LANES: usize> $name<LANES>
         where
-            $bits_ty: LanesAtMost32
+            $bits_ty<LANES>: LanesAtMost32
         {
             /// Construct a mask by setting all lanes to the given value.
             pub fn splat(value: bool) -> Self {
-                Self(<$inner_ty>::splat(value))
+                Self(<$inner_ty<LANES>>::splat(value))
             }
 
             /// Converts an array to a SIMD vector.
@@ -69,66 +71,72 @@ macro_rules! define_opaque_mask {
             }
         }
 
-        impl<const $lanes: usize> From<BitMask<$lanes>> for $name<$lanes>
+        impl<const LANES: usize> From<BitMask<LANES>> for $name<LANES>
         where
-            $bits_ty: LanesAtMost32,
-            BitMask<$lanes>: LanesAtMost32,
+            $bits_ty<LANES>: LanesAtMost32,
+            BitMask<LANES>: LanesAtMost32,
         {
-            fn from(value: BitMask<$lanes>) -> Self {
+            fn from(value: BitMask<LANES>) -> Self {
                 Self(value.into())
             }
         }
 
-        impl<const $lanes: usize> From<$name<$lanes>> for crate::BitMask<$lanes>
+        impl<const LANES: usize> From<$name<LANES>> for crate::BitMask<LANES>
         where
-            $bits_ty: LanesAtMost32,
-            BitMask<$lanes>: LanesAtMost32,
+            $bits_ty<LANES>: LanesAtMost32,
+            BitMask<LANES>: LanesAtMost32,
         {
-            fn from(value: $name<$lanes>) -> Self {
+            fn from(value: $name<LANES>) -> Self {
                 value.0.into()
             }
         }
 
-        impl<const $lanes: usize> From<$inner_ty> for $name<$lanes>
+        impl<const LANES: usize> From<$inner_ty<LANES>> for $name<LANES>
         where
-            $bits_ty: LanesAtMost32,
+            $bits_ty<LANES>: LanesAtMost32,
         {
-            fn from(value: $inner_ty) -> Self {
+            fn from(value: $inner_ty<LANES>) -> Self {
                 Self(value)
             }
         }
 
-        impl<const $lanes: usize> From<$name<$lanes>> for $inner_ty
+        impl<const LANES: usize> From<$name<LANES>> for $inner_ty<LANES>
         where
-            $bits_ty: LanesAtMost32,
+            $bits_ty<LANES>: LanesAtMost32,
         {
-            fn from(value: $name<$lanes>) -> Self {
+            fn from(value: $name<LANES>) -> Self {
                 value.0
             }
         }
 
         // vector/array conversion
-        impl<const $lanes: usize> From<[bool; $lanes]> for $name<$lanes> where $bits_ty: crate::LanesAtMost32 {
-            fn from(array: [bool; $lanes]) -> Self {
+        impl<const LANES: usize> From<[bool; LANES]> for $name<LANES>
+        where
+            $bits_ty<LANES>: crate::LanesAtMost32
+        {
+            fn from(array: [bool; LANES]) -> Self {
                 Self::from_array(array)
             }
         }
 
-        impl <const $lanes: usize> From<$name<$lanes>> for [bool; $lanes] where $bits_ty: crate::LanesAtMost32 {
-            fn from(vector: $name<$lanes>) -> Self {
+        impl <const LANES: usize> From<$name<LANES>> for [bool; LANES]
+        where
+            $bits_ty<LANES>: crate::LanesAtMost32
+        {
+            fn from(vector: $name<LANES>) -> Self {
                 vector.to_array()
             }
         }
 
-        impl<const $lanes: usize> Copy for $name<$lanes>
+        impl<const LANES: usize> Copy for $name<LANES>
         where
-            $inner_ty: Copy,
-            $bits_ty: LanesAtMost32,
+            $inner_ty<LANES>: Copy,
+            $bits_ty<LANES>: LanesAtMost32,
         {}
 
-        impl<const $lanes: usize> Clone for $name<$lanes>
+        impl<const LANES: usize> Clone for $name<LANES>
         where
-            $bits_ty: LanesAtMost32,
+            $bits_ty<LANES>: LanesAtMost32,
         {
             #[inline]
             fn clone(&self) -> Self {
@@ -136,9 +144,9 @@ macro_rules! define_opaque_mask {
             }
         }
 
-        impl<const $lanes: usize> Default for $name<$lanes>
+        impl<const LANES: usize> Default for $name<LANES>
         where
-            $bits_ty: LanesAtMost32,
+            $bits_ty<LANES>: LanesAtMost32,
         {
             #[inline]
             fn default() -> Self {
@@ -146,9 +154,9 @@ macro_rules! define_opaque_mask {
             }
         }
 
-        impl<const $lanes: usize> PartialEq for $name<$lanes>
+        impl<const LANES: usize> PartialEq for $name<LANES>
         where
-            $bits_ty: LanesAtMost32,
+            $bits_ty<LANES>: LanesAtMost32,
         {
             #[inline]
             fn eq(&self, other: &Self) -> bool {
@@ -156,9 +164,9 @@ macro_rules! define_opaque_mask {
             }
         }
 
-        impl<const $lanes: usize> PartialOrd for $name<$lanes>
+        impl<const LANES: usize> PartialOrd for $name<LANES>
         where
-            $bits_ty: LanesAtMost32,
+            $bits_ty<LANES>: LanesAtMost32,
         {
             #[inline]
             fn partial_cmp(&self, other: &Self) -> Option<core::cmp::Ordering> {
@@ -166,9 +174,9 @@ macro_rules! define_opaque_mask {
             }
         }
 
-        impl<const $lanes: usize> core::fmt::Debug for $name<$lanes>
+        impl<const LANES: usize> core::fmt::Debug for $name<LANES>
         where
-            $bits_ty: LanesAtMost32,
+            $bits_ty<LANES>: LanesAtMost32,
         {
             fn fmt(&self, f: &mut core::fmt::Formatter) -> core::fmt::Result {
                 core::fmt::Debug::fmt(&self.0, f)
@@ -177,7 +185,7 @@ macro_rules! define_opaque_mask {
 
         impl<const LANES: usize> core::ops::BitAnd for $name<LANES>
         where
-            $bits_ty: LanesAtMost32,
+            $bits_ty<LANES>: LanesAtMost32,
         {
             type Output = Self;
             #[inline]
@@ -188,7 +196,7 @@ macro_rules! define_opaque_mask {
 
         impl<const LANES: usize> core::ops::BitAnd<bool> for $name<LANES>
         where
-            $bits_ty: LanesAtMost32,
+            $bits_ty<LANES>: LanesAtMost32,
         {
             type Output = Self;
             #[inline]
@@ -199,7 +207,7 @@ macro_rules! define_opaque_mask {
 
         impl<const LANES: usize> core::ops::BitAnd<$name<LANES>> for bool
         where
-            $bits_ty: LanesAtMost32,
+            $bits_ty<LANES>: LanesAtMost32,
         {
             type Output = $name<LANES>;
             #[inline]
@@ -210,7 +218,7 @@ macro_rules! define_opaque_mask {
 
         impl<const LANES: usize> core::ops::BitOr for $name<LANES>
         where
-            $bits_ty: LanesAtMost32,
+            $bits_ty<LANES>: LanesAtMost32,
         {
             type Output = Self;
             #[inline]
@@ -221,7 +229,7 @@ macro_rules! define_opaque_mask {
 
         impl<const LANES: usize> core::ops::BitOr<bool> for $name<LANES>
         where
-            $bits_ty: LanesAtMost32,
+            $bits_ty<LANES>: LanesAtMost32,
         {
             type Output = Self;
             #[inline]
@@ -232,7 +240,7 @@ macro_rules! define_opaque_mask {
 
         impl<const LANES: usize> core::ops::BitOr<$name<LANES>> for bool
         where
-            $bits_ty: LanesAtMost32,
+            $bits_ty<LANES>: LanesAtMost32,
         {
             type Output = $name<LANES>;
             #[inline]
@@ -243,7 +251,7 @@ macro_rules! define_opaque_mask {
 
         impl<const LANES: usize> core::ops::BitXor for $name<LANES>
         where
-            $bits_ty: LanesAtMost32,
+            $bits_ty<LANES>: LanesAtMost32,
         {
             type Output = Self;
             #[inline]
@@ -254,7 +262,7 @@ macro_rules! define_opaque_mask {
 
         impl<const LANES: usize> core::ops::BitXor<bool> for $name<LANES>
         where
-            $bits_ty: LanesAtMost32,
+            $bits_ty<LANES>: LanesAtMost32,
         {
             type Output = Self;
             #[inline]
@@ -265,7 +273,7 @@ macro_rules! define_opaque_mask {
 
         impl<const LANES: usize> core::ops::BitXor<$name<LANES>> for bool
         where
-            $bits_ty: LanesAtMost32,
+            $bits_ty<LANES>: LanesAtMost32,
         {
             type Output = $name<LANES>;
             #[inline]
@@ -276,7 +284,7 @@ macro_rules! define_opaque_mask {
 
         impl<const LANES: usize> core::ops::Not for $name<LANES>
         where
-            $bits_ty: LanesAtMost32,
+            $bits_ty<LANES>: LanesAtMost32,
         {
             type Output = $name<LANES>;
             #[inline]
@@ -287,7 +295,7 @@ macro_rules! define_opaque_mask {
 
         impl<const LANES: usize> core::ops::BitAndAssign for $name<LANES>
         where
-            $bits_ty: LanesAtMost32,
+            $bits_ty<LANES>: LanesAtMost32,
         {
             #[inline]
             fn bitand_assign(&mut self, rhs: Self) {
@@ -297,7 +305,7 @@ macro_rules! define_opaque_mask {
 
         impl<const LANES: usize> core::ops::BitAndAssign<bool> for $name<LANES>
         where
-            $bits_ty: LanesAtMost32,
+            $bits_ty<LANES>: LanesAtMost32,
         {
             #[inline]
             fn bitand_assign(&mut self, rhs: bool) {
@@ -307,7 +315,7 @@ macro_rules! define_opaque_mask {
 
         impl<const LANES: usize> core::ops::BitOrAssign for $name<LANES>
         where
-            $bits_ty: LanesAtMost32,
+            $bits_ty<LANES>: LanesAtMost32,
         {
             #[inline]
             fn bitor_assign(&mut self, rhs: Self) {
@@ -317,7 +325,7 @@ macro_rules! define_opaque_mask {
 
         impl<const LANES: usize> core::ops::BitOrAssign<bool> for $name<LANES>
         where
-            $bits_ty: LanesAtMost32,
+            $bits_ty<LANES>: LanesAtMost32,
         {
             #[inline]
             fn bitor_assign(&mut self, rhs: bool) {
@@ -327,7 +335,7 @@ macro_rules! define_opaque_mask {
 
         impl<const LANES: usize> core::ops::BitXorAssign for $name<LANES>
         where
-            $bits_ty: LanesAtMost32,
+            $bits_ty<LANES>: LanesAtMost32,
         {
             #[inline]
             fn bitxor_assign(&mut self, rhs: Self) {
@@ -337,7 +345,7 @@ macro_rules! define_opaque_mask {
 
         impl<const LANES: usize> core::ops::BitXorAssign<bool> for $name<LANES>
         where
-            $bits_ty: LanesAtMost32,
+            $bits_ty<LANES>: LanesAtMost32,
         {
             #[inline]
             fn bitxor_assign(&mut self, rhs: bool) {
@@ -352,7 +360,7 @@ define_opaque_mask! {
     ///
     /// The layout of this type is unspecified.
     struct Mask8<const LANES: usize>(SimdMask8<LANES>);
-    @bits crate::SimdI8<LANES>
+    @bits SimdI8
 }
 
 define_opaque_mask! {
@@ -360,7 +368,7 @@ define_opaque_mask! {
     ///
     /// The layout of this type is unspecified.
     struct Mask16<const LANES: usize>(SimdMask16<LANES>);
-    @bits crate::SimdI16<LANES>
+    @bits SimdI16
 }
 
 define_opaque_mask! {
@@ -368,7 +376,7 @@ define_opaque_mask! {
     ///
     /// The layout of this type is unspecified.
     struct Mask32<const LANES: usize>(SimdMask32<LANES>);
-    @bits crate::SimdI32<LANES>
+    @bits SimdI32
 }
 
 define_opaque_mask! {
@@ -376,7 +384,7 @@ define_opaque_mask! {
     ///
     /// The layout of this type is unspecified.
     struct Mask64<const LANES: usize>(SimdMask64<LANES>);
-    @bits crate::SimdI64<LANES>
+    @bits SimdI64
 }
 
 define_opaque_mask! {
@@ -384,7 +392,7 @@ define_opaque_mask! {
     ///
     /// The layout of this type is unspecified.
     struct Mask128<const LANES: usize>(SimdMask128<LANES>);
-    @bits crate::SimdI128<LANES>
+    @bits SimdI128
 }
 
 define_opaque_mask! {
@@ -392,7 +400,7 @@ define_opaque_mask! {
     ///
     /// The layout of this type is unspecified.
     struct MaskSize<const LANES: usize>(SimdMaskSize<LANES>);
-    @bits crate::SimdIsize<LANES>
+    @bits SimdIsize
 }
 
 /// Vector of eight 8-bit masks
diff --git a/crates/core_simd/src/reduction.rs b/crates/core_simd/src/reduction.rs
new file mode 100644
index 000000000000..0c6d91a2befe
--- /dev/null
+++ b/crates/core_simd/src/reduction.rs
@@ -0,0 +1,142 @@
+macro_rules! impl_integer_reductions {
+    { $name:ident, $scalar:ty } => {
+        impl<const LANES: usize> crate::$name<LANES>
+        where
+            Self: crate::LanesAtMost32
+        {
+            /// Produces the sum of the lanes of the vector, with wrapping addition.
+            #[inline]
+            pub fn wrapping_sum(self) -> $scalar {
+                unsafe { crate::intrinsics::simd_reduce_add_ordered(self, 0) }
+            }
+
+            /// Produces the sum of the lanes of the vector, with wrapping multiplication.
+            #[inline]
+            pub fn wrapping_product(self) -> $scalar {
+                unsafe { crate::intrinsics::simd_reduce_mul_ordered(self, 1) }
+            }
+
+            /// Sequentially performs bitwise "and" between the lanes of the vector.
+            #[inline]
+            pub fn and_lanes(self) -> $scalar {
+                unsafe { crate::intrinsics::simd_reduce_and(self) }
+            }
+
+            /// Sequentially performs bitwise "or" between the lanes of the vector.
+            #[inline]
+            pub fn or_lanes(self) -> $scalar {
+                unsafe { crate::intrinsics::simd_reduce_or(self) }
+            }
+
+            /// Sequentially performs bitwise "xor" between the lanes of the vector.
+            #[inline]
+            pub fn xor_lanes(self) -> $scalar {
+                unsafe { crate::intrinsics::simd_reduce_xor(self) }
+            }
+
+            /// Returns the maximum lane in the vector.
+            #[inline]
+            pub fn max_lane(self) -> $scalar {
+                unsafe { crate::intrinsics::simd_reduce_max(self) }
+            }
+
+            /// Returns the minimum lane in the vector.
+            #[inline]
+            pub fn min_lane(self) -> $scalar {
+                unsafe { crate::intrinsics::simd_reduce_min(self) }
+            }
+        }
+    }
+}
+
+macro_rules! impl_float_reductions {
+    { $name:ident, $scalar:ty } => {
+        impl<const LANES: usize> crate::$name<LANES>
+        where
+            Self: crate::LanesAtMost32
+        {
+            /// Produces the sum of the lanes of the vector.
+            #[inline]
+            pub fn sum(self) -> $scalar {
+                unsafe { crate::intrinsics::simd_reduce_add_ordered(self, 0.) }
+            }
+
+            /// Produces the sum of the lanes of the vector.
+            #[inline]
+            pub fn product(self) -> $scalar {
+                unsafe { crate::intrinsics::simd_reduce_mul_ordered(self, 1.) }
+            }
+
+            /// Returns the maximum lane in the vector.
+            #[inline]
+            pub fn max_lane(self) -> $scalar {
+                unsafe { crate::intrinsics::simd_reduce_max(self) }
+            }
+
+            /// Returns the minimum lane in the vector.
+            #[inline]
+            pub fn min_lane(self) -> $scalar {
+                unsafe { crate::intrinsics::simd_reduce_min(self) }
+            }
+        }
+    }
+}
+
+macro_rules! impl_full_mask_reductions {
+    { $name:ident, $inner:ident } => {
+        impl<const LANES: usize> crate::$name<LANES>
+        where
+            crate::$inner<LANES>: crate::LanesAtMost32
+        {
+            /// Returns true if any lane is set, or false otherwise.
+            #[inline]
+            pub fn any(self) -> bool {
+                unsafe { crate::intrinsics::simd_reduce_any(self.to_int()) }
+            }
+
+            /// Returns true if all lanes are set, or false otherwise.
+            #[inline]
+            pub fn all(self) -> bool {
+                unsafe { crate::intrinsics::simd_reduce_all(self.to_int()) }
+            }
+        }
+    }
+}
+
+macro_rules! impl_opaque_mask_reductions {
+    { $name:ident, $inner:ident, $bits_ty:ident } => {
+        impl<const LANES: usize> $name<LANES>
+        where
+            $bits_ty<LANES>: crate::LanesAtMost32
+        {
+            /// Returns true if any lane is set, or false otherwise.
+            #[inline]
+            pub fn any(self) -> bool {
+                self.0.any()
+            }
+
+            /// Returns true if all lanes are set, or false otherwise.
+            #[inline]
+            pub fn all(self) -> bool {
+                self.0.all()
+            }
+        }
+    }
+}
+
+impl<const LANES: usize> crate::BitMask<LANES>
+where
+    crate::BitMask<LANES>: crate::LanesAtMost32,
+{
+    /// Returns true if any lane is set, or false otherwise.
+    #[inline]
+    pub fn any(self) -> bool {
+        self.0 != 0
+    }
+
+    /// Returns true if all lanes are set, or false otherwise.
+    #[inline]
+    pub fn all(self) -> bool {
+        self.0 == (!0) >> (64 - LANES)
+    }
+}
diff --git a/crates/core_simd/src/vector/float.rs b/crates/core_simd/src/vector/float.rs
index 393e39023d9c..91585b469461 100644
--- a/crates/core_simd/src/vector/float.rs
+++ b/crates/core_simd/src/vector/float.rs
@@ -6,6 +6,7 @@
 macro_rules! impl_float_vector {
     { $name:ident, $type:ty, $bits_ty:ident, $mask_ty:ident, $mask_impl_ty:ident } => {
         impl_vector! { $name, $type }
+        impl_float_reductions! { $name, $type }
 
         impl<const LANES: usize> $name<LANES>
         where
diff --git a/crates/core_simd/src/vector/int.rs b/crates/core_simd/src/vector/int.rs
index 5304d11cd6ea..24f77cb3e106 100644
--- a/crates/core_simd/src/vector/int.rs
+++ b/crates/core_simd/src/vector/int.rs
@@ -4,6 +4,7 @@
 macro_rules! impl_integer_vector {
     { $name:ident, $type:ty, $mask_ty:ident, $mask_impl_ty:ident } => {
         impl_vector! { $name, $type }
+        impl_integer_reductions! { $name, $type }
 
         impl<const LANES: usize> Eq for $name<LANES> where Self: crate::LanesAtMost32 {}
 
diff --git a/crates/core_simd/src/vector/uint.rs b/crates/core_simd/src/vector/uint.rs
index 71b5b295112a..3866b9ca5c6e 100644
--- a/crates/core_simd/src/vector/uint.rs
+++ b/crates/core_simd/src/vector/uint.rs
@@ -5,6 +5,7 @@
 macro_rules! impl_unsigned_vector {
     { $name:ident, $type:ty } => {
         impl_vector! { $name, $type }
+        impl_integer_reductions! { $name, $type }
 
         impl<const LANES: usize> Eq for $name<LANES> where Self: crate::LanesAtMost32 {}
 

From a7b82adb12a9bfbaaf4e446b4b17dcb35a546223 Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Mon, 8 Mar 2021 00:48:18 -0500
Subject: [PATCH 116/249] Add tests

---
 crates/core_simd/tests/masks.rs      |  18 ++++
 crates/core_simd/tests/ops_macros.rs | 121 +++++++++++++++++++++++++++
 crates/test_helpers/src/biteq.rs     |   2 +-
 3 files changed, 140 insertions(+), 1 deletion(-)

diff --git a/crates/core_simd/tests/masks.rs b/crates/core_simd/tests/masks.rs
index 03a835b9c66f..59da77de622b 100644
--- a/crates/core_simd/tests/masks.rs
+++ b/crates/core_simd/tests/masks.rs
@@ -59,6 +59,24 @@ macro_rules! test_mask_api {
                 let mask = core_simd::$name::<8>::splat(false);
                 let _ = mask.test(8);
             }
+
+            #[test]
+            fn any() {
+                assert!(!core_simd::$name::<8>::splat(false).any());
+                assert!(core_simd::$name::<8>::splat(true).any());
+                let mut v = core_simd::$name::<8>::splat(false);
+                v.set(2, true);
+                assert!(v.any());
+            }
+
+            #[test]
+            fn all() {
+                assert!(!core_simd::$name::<8>::splat(false).all());
+                assert!(core_simd::$name::<8>::splat(true).all());
+                let mut v = core_simd::$name::<8>::splat(false);
+                v.set(2, true);
+                assert!(!v.all());
+            }
         }
     }
 }
diff --git a/crates/core_simd/tests/ops_macros.rs b/crates/core_simd/tests/ops_macros.rs
index a70a8a9c48b9..d9f705cf3901 100644
--- a/crates/core_simd/tests/ops_macros.rs
+++ b/crates/core_simd/tests/ops_macros.rs
@@ -136,6 +136,83 @@ macro_rules! impl_binary_checked_op_test {
     };
 }
 
+#[macro_export]
+macro_rules! impl_common_integer_tests {
+    { $vector:ident, $scalar:ident } => {
+        test_helpers::test_lanes! {
+            fn wrapping_sum<const LANES: usize>() {
+                test_helpers::test_1(&|x| {
+                    test_helpers::prop_assert_biteq! (
+                        $vector::<LANES>::from_array(x).wrapping_sum(),
+                        x.iter().copied().fold(0 as $scalar, $scalar::wrapping_add),
+                    );
+                    Ok(())
+                });
+            }
+
+            fn wrapping_product<const LANES: usize>() {
+                test_helpers::test_1(&|x| {
+                    test_helpers::prop_assert_biteq! (
+                        $vector::<LANES>::from_array(x).wrapping_product(),
+                        x.iter().copied().fold(1 as $scalar, $scalar::wrapping_mul),
+                    );
+                    Ok(())
+                });
+            }
+
+            fn and_lanes<const LANES: usize>() {
+                test_helpers::test_1(&|x| {
+                    test_helpers::prop_assert_biteq! (
+                        $vector::<LANES>::from_array(x).and_lanes(),
+                        x.iter().copied().fold(-1i8 as $scalar, <$scalar as core::ops::BitAnd>::bitand),
+                    );
+                    Ok(())
+                });
+            }
+
+            fn or_lanes<const LANES: usize>() {
+                test_helpers::test_1(&|x| {
+                    test_helpers::prop_assert_biteq! (
+                        $vector::<LANES>::from_array(x).or_lanes(),
+                        x.iter().copied().fold(0 as $scalar, <$scalar as core::ops::BitOr>::bitor),
+                    );
+                    Ok(())
+                });
+            }
+
+            fn xor_lanes<const LANES: usize>() {
+                test_helpers::test_1(&|x| {
+                    test_helpers::prop_assert_biteq! (
+                        $vector::<LANES>::from_array(x).xor_lanes(),
+                        x.iter().copied().fold(0 as $scalar, <$scalar as core::ops::BitXor>::bitxor),
+                    );
+                    Ok(())
+                });
+            }
+
+            fn max_lane<const LANES: usize>() {
+                test_helpers::test_1(&|x| {
+                    test_helpers::prop_assert_biteq! (
+                        $vector::<LANES>::from_array(x).max_lane(),
+                        x.iter().copied().max().unwrap(),
+                    );
+                    Ok(())
+                });
+            }
+
+            fn min_lane<const LANES: usize>() {
+                test_helpers::test_1(&|x| {
+                    test_helpers::prop_assert_biteq! (
+                        $vector::<LANES>::from_array(x).min_lane(),
+                        x.iter().copied().min().unwrap(),
+                    );
+                    Ok(())
+                });
+            }
+        }
+    }
+}
+
 /// Implement tests for signed integers.
 #[macro_export]
 macro_rules! impl_signed_tests {
@@ -144,6 +221,8 @@ macro_rules! impl_signed_tests {
             type Vector<const LANES: usize> = core_simd::$vector<LANES>;
             type Scalar = $scalar;
 
+            impl_common_integer_tests! { Vector, Scalar }
+
             test_helpers::test_lanes! {
                 fn neg<const LANES: usize>() {
                     test_helpers::test_unary_elementwise(
@@ -241,6 +320,8 @@ macro_rules! impl_unsigned_tests {
             type Vector<const LANES: usize> = core_simd::$vector<LANES>;
             type Scalar = $scalar;
 
+            impl_common_integer_tests! { Vector, Scalar }
+
             test_helpers::test_lanes_panic! {
                 fn rem_zero_panic<const LANES: usize>() {
                     let a = Vector::<LANES>::splat(42);
@@ -397,6 +478,46 @@ macro_rules! impl_float_tests {
                         },
                     ).unwrap();
                 }
+
+                fn sum<const LANES: usize>() {
+                    test_helpers::test_1(&|x| {
+                        test_helpers::prop_assert_biteq! (
+                            Vector::<LANES>::from_array(x).sum(),
+                            x.iter().copied().fold(0 as Scalar, <Scalar as core::ops::Add>::add),
+                        );
+                        Ok(())
+                    });
+                }
+
+                fn product<const LANES: usize>() {
+                    test_helpers::test_1(&|x| {
+                        test_helpers::prop_assert_biteq! (
+                            Vector::<LANES>::from_array(x).product(),
+                            x.iter().copied().fold(1. as Scalar, <Scalar as core::ops::Mul>::mul),
+                        );
+                        Ok(())
+                    });
+                }
+
+                fn max_lane<const LANES: usize>() {
+                    test_helpers::test_1(&|x| {
+                        test_helpers::prop_assert_biteq! (
+                            Vector::<LANES>::from_array(x).max_lane(),
+                            x.iter().copied().fold(Scalar::NAN, Scalar::max),
+                        );
+                        Ok(())
+                    });
+                }
+
+                fn min_lane<const LANES: usize>() {
+                    test_helpers::test_1(&|x| {
+                        test_helpers::prop_assert_biteq! (
+                            Vector::<LANES>::from_array(x).min_lane(),
+                            x.iter().copied().fold(Scalar::NAN, Scalar::min),
+                        );
+                        Ok(())
+                    });
+                }
             }
         }
     }
diff --git a/crates/test_helpers/src/biteq.rs b/crates/test_helpers/src/biteq.rs
index 4a41fe3a16ea..00350e22418d 100644
--- a/crates/test_helpers/src/biteq.rs
+++ b/crates/test_helpers/src/biteq.rs
@@ -95,7 +95,7 @@ impl<T: BitEq> core::fmt::Debug for BitEqWrapper<'_, T> {
 
 #[macro_export]
 macro_rules! prop_assert_biteq {
-    { $a:expr, $b:expr } => {
+    { $a:expr, $b:expr $(,)? } => {
         {
             use $crate::biteq::BitEqWrapper;
             let a = $a;

From 193cd14b4a7a04d0713593d21f3630de012fd811 Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Wed, 10 Mar 2021 23:47:43 -0500
Subject: [PATCH 117/249] Enable special handling of zero

---
 crates/core_simd/tests/ops_macros.rs | 20 ++++++++++++--------
 1 file changed, 12 insertions(+), 8 deletions(-)

diff --git a/crates/core_simd/tests/ops_macros.rs b/crates/core_simd/tests/ops_macros.rs
index d9f705cf3901..2b65d5146238 100644
--- a/crates/core_simd/tests/ops_macros.rs
+++ b/crates/core_simd/tests/ops_macros.rs
@@ -501,20 +501,24 @@ macro_rules! impl_float_tests {
 
                 fn max_lane<const LANES: usize>() {
                     test_helpers::test_1(&|x| {
-                        test_helpers::prop_assert_biteq! (
-                            Vector::<LANES>::from_array(x).max_lane(),
-                            x.iter().copied().fold(Scalar::NAN, Scalar::max),
-                        );
+                        let vmax = Vector::<LANES>::from_array(x).max_lane();
+                        let smax = x.iter().copied().fold(Scalar::NAN, Scalar::max);
+                        // 0 and -0 are treated the same
+                        if !(x.contains(&0.) && x.contains(&-0.) && vmax.abs() == 0. && smax.abs() == 0.) {
+                            test_helpers::prop_assert_biteq!(vmax, smax);
+                        }
                         Ok(())
                     });
                 }
 
                 fn min_lane<const LANES: usize>() {
                     test_helpers::test_1(&|x| {
-                        test_helpers::prop_assert_biteq! (
-                            Vector::<LANES>::from_array(x).min_lane(),
-                            x.iter().copied().fold(Scalar::NAN, Scalar::min),
-                        );
+                        let vmax = Vector::<LANES>::from_array(x).min_lane();
+                        let smax = x.iter().copied().fold(Scalar::NAN, Scalar::min);
+                        // 0 and -0 are treated the same
+                        if !(x.contains(&0.) && x.contains(&-0.) && vmax.abs() == 0. && smax.abs() == 0.) {
+                            test_helpers::prop_assert_biteq!(vmax, smax);
+                        }
                         Ok(())
                     });
                 }

From 02608d44f7542981202792234540915484e0560d Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Thu, 11 Mar 2021 00:05:20 -0500
Subject: [PATCH 118/249] Fix mask ops

---
 crates/core_simd/src/masks/bitmask.rs | 6 +++---
 crates/core_simd/src/reduction.rs     | 4 ++--
 2 files changed, 5 insertions(+), 5 deletions(-)

diff --git a/crates/core_simd/src/masks/bitmask.rs b/crates/core_simd/src/masks/bitmask.rs
index 1d25db46742f..b4d1b6d9557d 100644
--- a/crates/core_simd/src/masks/bitmask.rs
+++ b/crates/core_simd/src/masks/bitmask.rs
@@ -1,9 +1,9 @@
 use crate::LanesAtMost32;
 
 /// A mask where each lane is represented by a single bit.
-#[derive(Copy, Clone, Debug)]
+#[derive(Copy, Clone, Debug, PartialOrd, PartialEq, Ord, Eq, Hash)]
 #[repr(transparent)]
-pub struct BitMask<const LANES: usize>(pub(crate) u64)
+pub struct BitMask<const LANES: usize>(u64)
 where
     BitMask<LANES>: LanesAtMost32;
 
@@ -14,7 +14,7 @@ where
     /// Construct a mask by setting all lanes to the given value.
     pub fn splat(value: bool) -> Self {
         if value {
-            Self(u64::MAX)
+            Self(u64::MAX >> (64 - LANES))
         } else {
             Self(u64::MIN)
         }
diff --git a/crates/core_simd/src/reduction.rs b/crates/core_simd/src/reduction.rs
index 0c6d91a2befe..d314cc737edf 100644
--- a/crates/core_simd/src/reduction.rs
+++ b/crates/core_simd/src/reduction.rs
@@ -131,12 +131,12 @@ where
     /// Returns true if any lane is set, or false otherwise.
     #[inline]
     pub fn any(self) -> bool {
-        self.0 != 0
+        self != Self::splat(false)
     }
 
     /// Returns true if all lanes are set, or false otherwise.
     #[inline]
     pub fn all(self) -> bool {
-        self.0 == (!0) >> (64 - LANES)
+        self == Self::splat(true)
     }
 }

From 64f564866bf09f98ae7a044fa8ca98a53bbbff1f Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Thu, 11 Mar 2021 00:27:47 -0500
Subject: [PATCH 119/249] Update documentation and fix i586 inaccuracy

---
 crates/core_simd/src/reduction.rs | 20 ++++++++++++++++++--
 1 file changed, 18 insertions(+), 2 deletions(-)

diff --git a/crates/core_simd/src/reduction.rs b/crates/core_simd/src/reduction.rs
index d314cc737edf..684879021b48 100644
--- a/crates/core_simd/src/reduction.rs
+++ b/crates/core_simd/src/reduction.rs
@@ -58,22 +58,38 @@ macro_rules! impl_float_reductions {
             /// Produces the sum of the lanes of the vector.
             #[inline]
             pub fn sum(self) -> $scalar {
-                unsafe { crate::intrinsics::simd_reduce_add_ordered(self, 0.) }
+                // f32 SIMD sum is inaccurate on i586
+                if cfg!(target_arch = "i586") && core::mem::size_of::<$scalar>() == 4 {
+                    self.as_slice().iter().sum()
+                } else {
+                    unsafe { crate::intrinsics::simd_reduce_add_ordered(self, 0.) }
+                }
             }
 
             /// Produces the sum of the lanes of the vector.
             #[inline]
             pub fn product(self) -> $scalar {
-                unsafe { crate::intrinsics::simd_reduce_mul_ordered(self, 1.) }
+                // f32 SIMD product is inaccurate on i586
+                if cfg!(target_arch = "i586") && core::mem::size_of::<$scalar>() == 4 {
+                    self.as_slice().iter().product()
+                } else {
+                    unsafe { crate::intrinsics::simd_reduce_mul_ordered(self, 1.) }
+                }
             }
 
             /// Returns the maximum lane in the vector.
+            ///
+            /// Returns values based on equality, so a vector containing both `0.` and `-0.` may
+            /// return either.  This function will not return `NaN` unless all lanes are `NaN`.
             #[inline]
             pub fn max_lane(self) -> $scalar {
                 unsafe { crate::intrinsics::simd_reduce_max(self) }
             }
 
             /// Returns the minimum lane in the vector.
+            ///
+            /// Returns values based on equality, so a vector containing both `0.` and `-0.` may
+            /// return either.  This function will not return `NaN` unless all lanes are `NaN`.
             #[inline]
             pub fn min_lane(self) -> $scalar {
                 unsafe { crate::intrinsics::simd_reduce_min(self) }

From 4b8cbd5385e8d6e851edb2d1e37ddbf843dda02a Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Thu, 11 Mar 2021 01:02:47 -0500
Subject: [PATCH 120/249] Fix i586 detection

---
 crates/core_simd/src/reduction.rs | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/crates/core_simd/src/reduction.rs b/crates/core_simd/src/reduction.rs
index 684879021b48..e59bf93baa31 100644
--- a/crates/core_simd/src/reduction.rs
+++ b/crates/core_simd/src/reduction.rs
@@ -55,11 +55,12 @@ macro_rules! impl_float_reductions {
         where
             Self: crate::LanesAtMost32
         {
+
             /// Produces the sum of the lanes of the vector.
             #[inline]
             pub fn sum(self) -> $scalar {
                 // f32 SIMD sum is inaccurate on i586
-                if cfg!(target_arch = "i586") && core::mem::size_of::<$scalar>() == 4 {
+                if cfg!(all(target_arch = "x86", not(target_feature = "sse2"))) && core::mem::size_of::<$scalar>() == 4 {
                     self.as_slice().iter().sum()
                 } else {
                     unsafe { crate::intrinsics::simd_reduce_add_ordered(self, 0.) }
@@ -70,7 +71,7 @@ macro_rules! impl_float_reductions {
             #[inline]
             pub fn product(self) -> $scalar {
                 // f32 SIMD product is inaccurate on i586
-                if cfg!(target_arch = "i586") && core::mem::size_of::<$scalar>() == 4 {
+                if cfg!(all(target_arch = "x86", not(target_feature = "sse2"))) && core::mem::size_of::<$scalar>() == 4 {
                     self.as_slice().iter().product()
                 } else {
                     unsafe { crate::intrinsics::simd_reduce_mul_ordered(self, 1.) }

From b51febbd348924a4cee970ef302dcaf5ff0fac18 Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Fri, 12 Mar 2021 00:29:18 -0500
Subject: [PATCH 121/249] Revert i586 fix, fix test instead

---
 crates/core_simd/src/reduction.rs    | 14 ++------------
 crates/core_simd/tests/ops_macros.rs |  4 ++--
 2 files changed, 4 insertions(+), 14 deletions(-)

diff --git a/crates/core_simd/src/reduction.rs b/crates/core_simd/src/reduction.rs
index e59bf93baa31..177669ff444e 100644
--- a/crates/core_simd/src/reduction.rs
+++ b/crates/core_simd/src/reduction.rs
@@ -59,23 +59,13 @@ macro_rules! impl_float_reductions {
             /// Produces the sum of the lanes of the vector.
             #[inline]
             pub fn sum(self) -> $scalar {
-                // f32 SIMD sum is inaccurate on i586
-                if cfg!(all(target_arch = "x86", not(target_feature = "sse2"))) && core::mem::size_of::<$scalar>() == 4 {
-                    self.as_slice().iter().sum()
-                } else {
-                    unsafe { crate::intrinsics::simd_reduce_add_ordered(self, 0.) }
-                }
+                unsafe { crate::intrinsics::simd_reduce_add_ordered(self, 0.) }
             }
 
             /// Produces the sum of the lanes of the vector.
             #[inline]
             pub fn product(self) -> $scalar {
-                // f32 SIMD product is inaccurate on i586
-                if cfg!(all(target_arch = "x86", not(target_feature = "sse2"))) && core::mem::size_of::<$scalar>() == 4 {
-                    self.as_slice().iter().product()
-                } else {
-                    unsafe { crate::intrinsics::simd_reduce_mul_ordered(self, 1.) }
-                }
+                unsafe { crate::intrinsics::simd_reduce_mul_ordered(self, 1.) }
             }
 
             /// Returns the maximum lane in the vector.
diff --git a/crates/core_simd/tests/ops_macros.rs b/crates/core_simd/tests/ops_macros.rs
index 2b65d5146238..59e923ac5c14 100644
--- a/crates/core_simd/tests/ops_macros.rs
+++ b/crates/core_simd/tests/ops_macros.rs
@@ -483,7 +483,7 @@ macro_rules! impl_float_tests {
                     test_helpers::test_1(&|x| {
                         test_helpers::prop_assert_biteq! (
                             Vector::<LANES>::from_array(x).sum(),
-                            x.iter().copied().fold(0 as Scalar, <Scalar as core::ops::Add>::add),
+                            x.iter().sum(),
                         );
                         Ok(())
                     });
@@ -493,7 +493,7 @@ macro_rules! impl_float_tests {
                     test_helpers::test_1(&|x| {
                         test_helpers::prop_assert_biteq! (
                             Vector::<LANES>::from_array(x).product(),
-                            x.iter().copied().fold(1. as Scalar, <Scalar as core::ops::Mul>::mul),
+                            x.iter().product(),
                         );
                         Ok(())
                     });

From 3fae09bd08b4ffacd3f81cc6ec13772e99d29796 Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Fri, 12 Mar 2021 20:09:31 -0500
Subject: [PATCH 122/249] Revert "Revert i586 fix, fix test instead"

This reverts commit 1ea2f128821339d8050ca936f24b71677352437e.
---
 crates/core_simd/src/reduction.rs    | 14 ++++++++++++--
 crates/core_simd/tests/ops_macros.rs |  4 ++--
 2 files changed, 14 insertions(+), 4 deletions(-)

diff --git a/crates/core_simd/src/reduction.rs b/crates/core_simd/src/reduction.rs
index 177669ff444e..e59bf93baa31 100644
--- a/crates/core_simd/src/reduction.rs
+++ b/crates/core_simd/src/reduction.rs
@@ -59,13 +59,23 @@ macro_rules! impl_float_reductions {
             /// Produces the sum of the lanes of the vector.
             #[inline]
             pub fn sum(self) -> $scalar {
-                unsafe { crate::intrinsics::simd_reduce_add_ordered(self, 0.) }
+                // f32 SIMD sum is inaccurate on i586
+                if cfg!(all(target_arch = "x86", not(target_feature = "sse2"))) && core::mem::size_of::<$scalar>() == 4 {
+                    self.as_slice().iter().sum()
+                } else {
+                    unsafe { crate::intrinsics::simd_reduce_add_ordered(self, 0.) }
+                }
             }
 
             /// Produces the sum of the lanes of the vector.
             #[inline]
             pub fn product(self) -> $scalar {
-                unsafe { crate::intrinsics::simd_reduce_mul_ordered(self, 1.) }
+                // f32 SIMD product is inaccurate on i586
+                if cfg!(all(target_arch = "x86", not(target_feature = "sse2"))) && core::mem::size_of::<$scalar>() == 4 {
+                    self.as_slice().iter().product()
+                } else {
+                    unsafe { crate::intrinsics::simd_reduce_mul_ordered(self, 1.) }
+                }
             }
 
             /// Returns the maximum lane in the vector.
diff --git a/crates/core_simd/tests/ops_macros.rs b/crates/core_simd/tests/ops_macros.rs
index 59e923ac5c14..2b65d5146238 100644
--- a/crates/core_simd/tests/ops_macros.rs
+++ b/crates/core_simd/tests/ops_macros.rs
@@ -483,7 +483,7 @@ macro_rules! impl_float_tests {
                     test_helpers::test_1(&|x| {
                         test_helpers::prop_assert_biteq! (
                             Vector::<LANES>::from_array(x).sum(),
-                            x.iter().sum(),
+                            x.iter().copied().fold(0 as Scalar, <Scalar as core::ops::Add>::add),
                         );
                         Ok(())
                     });
@@ -493,7 +493,7 @@ macro_rules! impl_float_tests {
                     test_helpers::test_1(&|x| {
                         test_helpers::prop_assert_biteq! (
                             Vector::<LANES>::from_array(x).product(),
-                            x.iter().product(),
+                            x.iter().copied().fold(1. as Scalar, <Scalar as core::ops::Mul>::mul),
                         );
                         Ok(())
                     });

From 3cf970fc0997591cb1a0388874506e58a8c44baf Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Fri, 12 Mar 2021 20:10:51 -0500
Subject: [PATCH 123/249] Fix test sum/product implementation

---
 crates/core_simd/tests/ops_macros.rs | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/crates/core_simd/tests/ops_macros.rs b/crates/core_simd/tests/ops_macros.rs
index 2b65d5146238..59e923ac5c14 100644
--- a/crates/core_simd/tests/ops_macros.rs
+++ b/crates/core_simd/tests/ops_macros.rs
@@ -483,7 +483,7 @@ macro_rules! impl_float_tests {
                     test_helpers::test_1(&|x| {
                         test_helpers::prop_assert_biteq! (
                             Vector::<LANES>::from_array(x).sum(),
-                            x.iter().copied().fold(0 as Scalar, <Scalar as core::ops::Add>::add),
+                            x.iter().sum(),
                         );
                         Ok(())
                     });
@@ -493,7 +493,7 @@ macro_rules! impl_float_tests {
                     test_helpers::test_1(&|x| {
                         test_helpers::prop_assert_biteq! (
                             Vector::<LANES>::from_array(x).product(),
-                            x.iter().copied().fold(1. as Scalar, <Scalar as core::ops::Mul>::mul),
+                            x.iter().product(),
                         );
                         Ok(())
                     });

From e2fa502617175e90f47e0e50873774e512a3ce62 Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Fri, 12 Mar 2021 20:31:30 -0500
Subject: [PATCH 124/249] Enable i586 workaround for both f32 and f64

---
 crates/core_simd/src/reduction.rs | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/crates/core_simd/src/reduction.rs b/crates/core_simd/src/reduction.rs
index e59bf93baa31..a2b652189c84 100644
--- a/crates/core_simd/src/reduction.rs
+++ b/crates/core_simd/src/reduction.rs
@@ -59,8 +59,8 @@ macro_rules! impl_float_reductions {
             /// Produces the sum of the lanes of the vector.
             #[inline]
             pub fn sum(self) -> $scalar {
-                // f32 SIMD sum is inaccurate on i586
-                if cfg!(all(target_arch = "x86", not(target_feature = "sse2"))) && core::mem::size_of::<$scalar>() == 4 {
+                // LLVM sum is inaccurate on i586
+                if cfg!(all(target_arch = "x86", not(target_feature = "sse2"))) {
                     self.as_slice().iter().sum()
                 } else {
                     unsafe { crate::intrinsics::simd_reduce_add_ordered(self, 0.) }
@@ -70,8 +70,8 @@ macro_rules! impl_float_reductions {
             /// Produces the sum of the lanes of the vector.
             #[inline]
             pub fn product(self) -> $scalar {
-                // f32 SIMD product is inaccurate on i586
-                if cfg!(all(target_arch = "x86", not(target_feature = "sse2"))) && core::mem::size_of::<$scalar>() == 4 {
+                // LLVM product is inaccurate on i586
+                if cfg!(all(target_arch = "x86", not(target_feature = "sse2"))) {
                     self.as_slice().iter().product()
                 } else {
                     unsafe { crate::intrinsics::simd_reduce_mul_ordered(self, 1.) }

From e12758670900aa079b4151bb262d7e4dc0f375c6 Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Sun, 11 Apr 2021 10:59:05 -0400
Subject: [PATCH 125/249] Improve function names and docs

---
 crates/core_simd/src/reduction.rs    | 39 +++++++++++++++-------------
 crates/core_simd/tests/ops_macros.rs | 28 ++++++++++----------
 2 files changed, 35 insertions(+), 32 deletions(-)

diff --git a/crates/core_simd/src/reduction.rs b/crates/core_simd/src/reduction.rs
index a2b652189c84..e728f8ad82a9 100644
--- a/crates/core_simd/src/reduction.rs
+++ b/crates/core_simd/src/reduction.rs
@@ -4,45 +4,48 @@ macro_rules! impl_integer_reductions {
         where
             Self: crate::LanesAtMost32
         {
-            /// Produces the sum of the lanes of the vector, with wrapping addition.
+            /// Horizontal wrapping add.  Computes the sum of the lanes of the vector, with wrapping addition.
             #[inline]
             pub fn wrapping_sum(self) -> $scalar {
                 unsafe { crate::intrinsics::simd_reduce_add_ordered(self, 0) }
             }
 
-            /// Produces the sum of the lanes of the vector, with wrapping multiplication.
+            /// Horizontal wrapping multiply.  Computes the product of the lanes of the vector, with wrapping multiplication.
             #[inline]
             pub fn wrapping_product(self) -> $scalar {
                 unsafe { crate::intrinsics::simd_reduce_mul_ordered(self, 1) }
             }
 
-            /// Sequentially performs bitwise "and" between the lanes of the vector.
+            /// Horizontal bitwise "and".  Computes the cumulative bitwise "and" across the lanes of
+            /// the vector.
             #[inline]
-            pub fn and_lanes(self) -> $scalar {
+            pub fn horizontal_and(self) -> $scalar {
                 unsafe { crate::intrinsics::simd_reduce_and(self) }
             }
 
-            /// Sequentially performs bitwise "or" between the lanes of the vector.
+            /// Horizontal bitwise "or".  Computes the cumulative bitwise "or" across the lanes of
+            /// the vector.
             #[inline]
-            pub fn or_lanes(self) -> $scalar {
+            pub fn horizontal_or(self) -> $scalar {
                 unsafe { crate::intrinsics::simd_reduce_or(self) }
             }
 
-            /// Sequentially performs bitwise "xor" between the lanes of the vector.
+            /// Horizontal bitwise "xor".  Computes the cumulative bitwise "xor" across the lanes of
+            /// the vector.
             #[inline]
-            pub fn xor_lanes(self) -> $scalar {
+            pub fn horizontal_xor(self) -> $scalar {
                 unsafe { crate::intrinsics::simd_reduce_xor(self) }
             }
 
-            /// Returns the maximum lane in the vector.
+            /// Horizontal maximum.  Computes the maximum lane in the vector.
             #[inline]
-            pub fn max_lane(self) -> $scalar {
+            pub fn horizontal_max(self) -> $scalar {
                 unsafe { crate::intrinsics::simd_reduce_max(self) }
             }
 
-            /// Returns the minimum lane in the vector.
+            /// Horizontal minimum.  Computes the minimum lane in the vector.
             #[inline]
-            pub fn min_lane(self) -> $scalar {
+            pub fn horizontal_min(self) -> $scalar {
                 unsafe { crate::intrinsics::simd_reduce_min(self) }
             }
         }
@@ -56,7 +59,7 @@ macro_rules! impl_float_reductions {
             Self: crate::LanesAtMost32
         {
 
-            /// Produces the sum of the lanes of the vector.
+            /// Horizontal add.  Computes the sum of the lanes of the vector.
             #[inline]
             pub fn sum(self) -> $scalar {
                 // LLVM sum is inaccurate on i586
@@ -67,7 +70,7 @@ macro_rules! impl_float_reductions {
                 }
             }
 
-            /// Produces the sum of the lanes of the vector.
+            /// Horizontal multiply.  Computes the sum of the lanes of the vector.
             #[inline]
             pub fn product(self) -> $scalar {
                 // LLVM product is inaccurate on i586
@@ -78,21 +81,21 @@ macro_rules! impl_float_reductions {
                 }
             }
 
-            /// Returns the maximum lane in the vector.
+            /// Horizontal maximum.  Computes the maximum lane in the vector.
             ///
             /// Returns values based on equality, so a vector containing both `0.` and `-0.` may
             /// return either.  This function will not return `NaN` unless all lanes are `NaN`.
             #[inline]
-            pub fn max_lane(self) -> $scalar {
+            pub fn horizontal_max(self) -> $scalar {
                 unsafe { crate::intrinsics::simd_reduce_max(self) }
             }
 
-            /// Returns the minimum lane in the vector.
+            /// Horizontal minimum.  Computes the minimum lane in the vector.
             ///
             /// Returns values based on equality, so a vector containing both `0.` and `-0.` may
             /// return either.  This function will not return `NaN` unless all lanes are `NaN`.
             #[inline]
-            pub fn min_lane(self) -> $scalar {
+            pub fn horizontal_min(self) -> $scalar {
                 unsafe { crate::intrinsics::simd_reduce_min(self) }
             }
         }
diff --git a/crates/core_simd/tests/ops_macros.rs b/crates/core_simd/tests/ops_macros.rs
index 59e923ac5c14..7ce85b77254a 100644
--- a/crates/core_simd/tests/ops_macros.rs
+++ b/crates/core_simd/tests/ops_macros.rs
@@ -160,50 +160,50 @@ macro_rules! impl_common_integer_tests {
                 });
             }
 
-            fn and_lanes<const LANES: usize>() {
+            fn horizontal_and<const LANES: usize>() {
                 test_helpers::test_1(&|x| {
                     test_helpers::prop_assert_biteq! (
-                        $vector::<LANES>::from_array(x).and_lanes(),
+                        $vector::<LANES>::from_array(x).horizontal_and(),
                         x.iter().copied().fold(-1i8 as $scalar, <$scalar as core::ops::BitAnd>::bitand),
                     );
                     Ok(())
                 });
             }
 
-            fn or_lanes<const LANES: usize>() {
+            fn horizontal_or<const LANES: usize>() {
                 test_helpers::test_1(&|x| {
                     test_helpers::prop_assert_biteq! (
-                        $vector::<LANES>::from_array(x).or_lanes(),
+                        $vector::<LANES>::from_array(x).horizontal_or(),
                         x.iter().copied().fold(0 as $scalar, <$scalar as core::ops::BitOr>::bitor),
                     );
                     Ok(())
                 });
             }
 
-            fn xor_lanes<const LANES: usize>() {
+            fn horizontal_xor<const LANES: usize>() {
                 test_helpers::test_1(&|x| {
                     test_helpers::prop_assert_biteq! (
-                        $vector::<LANES>::from_array(x).xor_lanes(),
+                        $vector::<LANES>::from_array(x).horizontal_xor(),
                         x.iter().copied().fold(0 as $scalar, <$scalar as core::ops::BitXor>::bitxor),
                     );
                     Ok(())
                 });
             }
 
-            fn max_lane<const LANES: usize>() {
+            fn horizontal_max<const LANES: usize>() {
                 test_helpers::test_1(&|x| {
                     test_helpers::prop_assert_biteq! (
-                        $vector::<LANES>::from_array(x).max_lane(),
+                        $vector::<LANES>::from_array(x).horizontal_max(),
                         x.iter().copied().max().unwrap(),
                     );
                     Ok(())
                 });
             }
 
-            fn min_lane<const LANES: usize>() {
+            fn horizontal_min<const LANES: usize>() {
                 test_helpers::test_1(&|x| {
                     test_helpers::prop_assert_biteq! (
-                        $vector::<LANES>::from_array(x).min_lane(),
+                        $vector::<LANES>::from_array(x).horizontal_min(),
                         x.iter().copied().min().unwrap(),
                     );
                     Ok(())
@@ -499,9 +499,9 @@ macro_rules! impl_float_tests {
                     });
                 }
 
-                fn max_lane<const LANES: usize>() {
+                fn horizontal_max<const LANES: usize>() {
                     test_helpers::test_1(&|x| {
-                        let vmax = Vector::<LANES>::from_array(x).max_lane();
+                        let vmax = Vector::<LANES>::from_array(x).horizontal_max();
                         let smax = x.iter().copied().fold(Scalar::NAN, Scalar::max);
                         // 0 and -0 are treated the same
                         if !(x.contains(&0.) && x.contains(&-0.) && vmax.abs() == 0. && smax.abs() == 0.) {
@@ -511,9 +511,9 @@ macro_rules! impl_float_tests {
                     });
                 }
 
-                fn min_lane<const LANES: usize>() {
+                fn horizontal_min<const LANES: usize>() {
                     test_helpers::test_1(&|x| {
-                        let vmax = Vector::<LANES>::from_array(x).min_lane();
+                        let vmax = Vector::<LANES>::from_array(x).horizontal_min();
                         let smax = x.iter().copied().fold(Scalar::NAN, Scalar::min);
                         // 0 and -0 are treated the same
                         if !(x.contains(&0.) && x.contains(&-0.) && vmax.abs() == 0. && smax.abs() == 0.) {

From e3f0124baf711f9a749d924346b9979ca665603e Mon Sep 17 00:00:00 2001
From: Jubilee Young <workingjubilee@gmail.com>
Date: Thu, 15 Apr 2021 15:06:30 -0700
Subject: [PATCH 126/249] Silence warnings

---
 crates/core_simd/src/transmute.rs | 1 +
 crates/core_simd/tests/f32_ops.rs | 2 --
 crates/core_simd/tests/f64_ops.rs | 2 --
 3 files changed, 1 insertion(+), 4 deletions(-)

diff --git a/crates/core_simd/src/transmute.rs b/crates/core_simd/src/transmute.rs
index 835d863029c5..b7072b334fdd 100644
--- a/crates/core_simd/src/transmute.rs
+++ b/crates/core_simd/src/transmute.rs
@@ -1,4 +1,5 @@
 /// Provides implementations of `From<$a> for $b` and `From<$b> for $a` that transmutes the value.
+#[cfg(any(target_arch = "x86", target_arch = "x86_64"))]
 macro_rules! from_transmute {
     { unsafe $a:ty => $b:ty } => {
         from_transmute!{ @impl $a => $b }
diff --git a/crates/core_simd/tests/f32_ops.rs b/crates/core_simd/tests/f32_ops.rs
index 048c070a391d..ac5499b7ffeb 100644
--- a/crates/core_simd/tests/f32_ops.rs
+++ b/crates/core_simd/tests/f32_ops.rs
@@ -1,5 +1,3 @@
-#![feature(is_subnormal)]
-
 #[macro_use]
 mod ops_macros;
 impl_float_tests! { SimdF32, f32, i32 }
diff --git a/crates/core_simd/tests/f64_ops.rs b/crates/core_simd/tests/f64_ops.rs
index 8f0dd6b73655..dcdb2aa31522 100644
--- a/crates/core_simd/tests/f64_ops.rs
+++ b/crates/core_simd/tests/f64_ops.rs
@@ -1,5 +1,3 @@
-#![feature(is_subnormal)]
-
 #[macro_use]
 mod ops_macros;
 impl_float_tests! { SimdF64, f64, i64 }

From 894062f8945b8f1d1499690da0162f13649b1ed5 Mon Sep 17 00:00:00 2001
From: Jubilee Young <workingjubilee@gmail.com>
Date: Thu, 15 Apr 2021 20:24:32 -0700
Subject: [PATCH 127/249] Burn Chrome again

---
 .travis.yml | 17 ++++++++---------
 1 file changed, 8 insertions(+), 9 deletions(-)

diff --git a/.travis.yml b/.travis.yml
index a06fa62d50fd..dc99ecf8fe63 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -42,29 +42,28 @@ matrix:
       env: RUSTFLAGS=-Ctarget-feature=+avx512vl
 
     # WebAssembly (wasm-bindgen)
-    - name: "wasm32-unknown-unknown (node, firefox, chrome)"
+    - name: "wasm32-unknown-unknown (firefox)"
       os: linux
       arch: amd64
       addons:
-        firefox: latest
-        chrome: stable
+        firefox: latest-nightly
       install:
         - curl https://rustwasm.github.io/wasm-pack/installer/init.sh -sSf | sh
       script:
-        - wasm-pack test --node --firefox --chrome --headless crates/core_simd
-        - wasm-pack test --node --firefox --chrome --headless crates/core_simd --release
+        - wasm-pack test --firefox --headless crates/core_simd
+        - wasm-pack test --firefox --headless crates/core_simd --release
 
-    - name: "wasm32-unknown-unknown+simd128 (chrome)"
+    - name: "wasm32-unknown-unknown+simd128 (firefox)"
       os: linux
       arch: amd64
       addons:
-        chrome: stable
+        firefox: latest-nightly
       install:
         - curl https://rustwasm.github.io/wasm-pack/installer/init.sh -sSf | sh
       script:
         - export RUSTFLAGS="-C target-feature=+simd128"
-        - wasm-pack test --chrome --headless crates/core_simd
-        - wasm-pack test --chrome --headless crates/core_simd --release
+        - wasm-pack test --firefox --headless crates/core_simd
+        - wasm-pack test --firefox --headless crates/core_simd --release
 
 script:
   - echo "## Requested target configuration (RUSTFLAGS=$RUSTFLAGS)"

From 81c96338b7ec3209b5c786da58fd6b4a11d0b629 Mon Sep 17 00:00:00 2001
From: Jubilee Young <workingjubilee@gmail.com>
Date: Fri, 16 Apr 2021 11:33:02 -0700
Subject: [PATCH 128/249] Drop wasm SIMD tests

---
 .travis.yml | 23 ++++++++++++-----------
 1 file changed, 12 insertions(+), 11 deletions(-)

diff --git a/.travis.yml b/.travis.yml
index dc99ecf8fe63..6b284c87ecbe 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -53,17 +53,18 @@ matrix:
         - wasm-pack test --firefox --headless crates/core_simd
         - wasm-pack test --firefox --headless crates/core_simd --release
 
-    - name: "wasm32-unknown-unknown+simd128 (firefox)"
-      os: linux
-      arch: amd64
-      addons:
-        firefox: latest-nightly
-      install:
-        - curl https://rustwasm.github.io/wasm-pack/installer/init.sh -sSf | sh
-      script:
-        - export RUSTFLAGS="-C target-feature=+simd128"
-        - wasm-pack test --firefox --headless crates/core_simd
-        - wasm-pack test --firefox --headless crates/core_simd --release
+    # FIXME: See https://github.com/rust-lang/stdsimd/issues/92
+    # - name: "wasm32-unknown-unknown+simd128 (firefox)"
+    #   os: linux
+    #   arch: amd64
+    #   addons:
+    #     firefox: latest-nightly
+    #   install:
+    #     - curl https://rustwasm.github.io/wasm-pack/installer/init.sh -sSf | sh
+    #   script:
+    #     - export RUSTFLAGS="-C target-feature=+simd128"
+    #     - wasm-pack test --firefox --headless crates/core_simd
+    #     - wasm-pack test --firefox --headless crates/core_simd --release
 
 script:
   - echo "## Requested target configuration (RUSTFLAGS=$RUSTFLAGS)"

From 87b7207acd6bfb07f0f82b72511b22c866a74553 Mon Sep 17 00:00:00 2001
From: Jubilee Young <workingjubilee@gmail.com>
Date: Thu, 8 Apr 2021 15:11:20 -0700
Subject: [PATCH 129/249] Use neg intrinsics

---
 crates/core_simd/src/intrinsics.rs |  3 +++
 crates/core_simd/src/ops.rs        | 22 ++--------------------
 2 files changed, 5 insertions(+), 20 deletions(-)

diff --git a/crates/core_simd/src/intrinsics.rs b/crates/core_simd/src/intrinsics.rs
index fafeed6a62a8..4f89d00deb23 100644
--- a/crates/core_simd/src/intrinsics.rs
+++ b/crates/core_simd/src/intrinsics.rs
@@ -40,6 +40,9 @@ extern "platform-intrinsic" {
     /// fptoui/fptosi/uitofp/sitofp
     pub(crate) fn simd_cast<T, U>(x: T) -> U;
 
+    /// neg/fneg
+    pub(crate) fn simd_neg<T>(x: T) -> T;
+
     // floor
     #[cfg(feature = "std")]
     pub(crate) fn simd_floor<T>(x: T) -> T;
diff --git a/crates/core_simd/src/ops.rs b/crates/core_simd/src/ops.rs
index 12d675a06409..513eeb423d92 100644
--- a/crates/core_simd/src/ops.rs
+++ b/crates/core_simd/src/ops.rs
@@ -185,25 +185,7 @@ macro_rules! impl_op {
             {
                 type Output = Self;
                 fn neg(self) -> Self::Output {
-                    Self::splat(0) - self
-                }
-            }
-        }
-    };
-
-    { impl Neg for $type:ident, $scalar:ty, @float } => {
-        impl_ref_ops! {
-            impl<const LANES: usize> core::ops::Neg for crate::$type<LANES>
-            where
-                crate::$type<LANES>: LanesAtMost32,
-                crate::SimdU32<LANES>: LanesAtMost32,
-                crate::SimdU64<LANES>: LanesAtMost32,
-            {
-                type Output = Self;
-                fn neg(self) -> Self::Output {
-                    // FIXME: Replace this with fneg intrinsic once available.
-                    // https://github.com/rust-lang/stdsimd/issues/32
-                    Self::from_bits(Self::splat(-0.0).to_bits() ^ self.to_bits())
+                    unsafe { crate::intrinsics::simd_neg(self) }
                 }
             }
         }
@@ -318,7 +300,7 @@ macro_rules! impl_float_ops {
                 impl_op! { impl Mul for $vector, $scalar }
                 impl_op! { impl Div for $vector, $scalar }
                 impl_op! { impl Rem for $vector, $scalar }
-                impl_op! { impl Neg for $vector, $scalar, @float }
+                impl_op! { impl Neg for $vector, $scalar }
                 impl_op! { impl Index for $vector, $scalar }
             )*
         )*

From 01d78aa21aee98ccf5b71a2ee9a136aa9e5f290c Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Sat, 17 Apr 2021 01:32:45 +0000
Subject: [PATCH 130/249] Update docs

---
 crates/core_simd/src/reduction.rs | 22 +++++++++++-----------
 1 file changed, 11 insertions(+), 11 deletions(-)

diff --git a/crates/core_simd/src/reduction.rs b/crates/core_simd/src/reduction.rs
index e728f8ad82a9..e1fc82e328ad 100644
--- a/crates/core_simd/src/reduction.rs
+++ b/crates/core_simd/src/reduction.rs
@@ -4,46 +4,46 @@ macro_rules! impl_integer_reductions {
         where
             Self: crate::LanesAtMost32
         {
-            /// Horizontal wrapping add.  Computes the sum of the lanes of the vector, with wrapping addition.
+            /// Horizontal wrapping add.  Returns the sum of the lanes of the vector, with wrapping addition.
             #[inline]
             pub fn wrapping_sum(self) -> $scalar {
                 unsafe { crate::intrinsics::simd_reduce_add_ordered(self, 0) }
             }
 
-            /// Horizontal wrapping multiply.  Computes the product of the lanes of the vector, with wrapping multiplication.
+            /// Horizontal wrapping multiply.  Returns the product of the lanes of the vector, with wrapping multiplication.
             #[inline]
             pub fn wrapping_product(self) -> $scalar {
                 unsafe { crate::intrinsics::simd_reduce_mul_ordered(self, 1) }
             }
 
-            /// Horizontal bitwise "and".  Computes the cumulative bitwise "and" across the lanes of
+            /// Horizontal bitwise "and".  Returns the cumulative bitwise "and" across the lanes of
             /// the vector.
             #[inline]
             pub fn horizontal_and(self) -> $scalar {
                 unsafe { crate::intrinsics::simd_reduce_and(self) }
             }
 
-            /// Horizontal bitwise "or".  Computes the cumulative bitwise "or" across the lanes of
+            /// Horizontal bitwise "or".  Returns the cumulative bitwise "or" across the lanes of
             /// the vector.
             #[inline]
             pub fn horizontal_or(self) -> $scalar {
                 unsafe { crate::intrinsics::simd_reduce_or(self) }
             }
 
-            /// Horizontal bitwise "xor".  Computes the cumulative bitwise "xor" across the lanes of
+            /// Horizontal bitwise "xor".  Returns the cumulative bitwise "xor" across the lanes of
             /// the vector.
             #[inline]
             pub fn horizontal_xor(self) -> $scalar {
                 unsafe { crate::intrinsics::simd_reduce_xor(self) }
             }
 
-            /// Horizontal maximum.  Computes the maximum lane in the vector.
+            /// Horizontal maximum.  Returns the maximum lane in the vector.
             #[inline]
             pub fn horizontal_max(self) -> $scalar {
                 unsafe { crate::intrinsics::simd_reduce_max(self) }
             }
 
-            /// Horizontal minimum.  Computes the minimum lane in the vector.
+            /// Horizontal minimum.  Returns the minimum lane in the vector.
             #[inline]
             pub fn horizontal_min(self) -> $scalar {
                 unsafe { crate::intrinsics::simd_reduce_min(self) }
@@ -59,7 +59,7 @@ macro_rules! impl_float_reductions {
             Self: crate::LanesAtMost32
         {
 
-            /// Horizontal add.  Computes the sum of the lanes of the vector.
+            /// Horizontal add.  Returns the sum of the lanes of the vector.
             #[inline]
             pub fn sum(self) -> $scalar {
                 // LLVM sum is inaccurate on i586
@@ -70,7 +70,7 @@ macro_rules! impl_float_reductions {
                 }
             }
 
-            /// Horizontal multiply.  Computes the sum of the lanes of the vector.
+            /// Horizontal multiply.  Returns the product of the lanes of the vector.
             #[inline]
             pub fn product(self) -> $scalar {
                 // LLVM product is inaccurate on i586
@@ -81,7 +81,7 @@ macro_rules! impl_float_reductions {
                 }
             }
 
-            /// Horizontal maximum.  Computes the maximum lane in the vector.
+            /// Horizontal maximum.  Returns the maximum lane in the vector.
             ///
             /// Returns values based on equality, so a vector containing both `0.` and `-0.` may
             /// return either.  This function will not return `NaN` unless all lanes are `NaN`.
@@ -90,7 +90,7 @@ macro_rules! impl_float_reductions {
                 unsafe { crate::intrinsics::simd_reduce_max(self) }
             }
 
-            /// Horizontal minimum.  Computes the minimum lane in the vector.
+            /// Horizontal minimum.  Returns the minimum lane in the vector.
             ///
             /// Returns values based on equality, so a vector containing both `0.` and `-0.` may
             /// return either.  This function will not return `NaN` unless all lanes are `NaN`.

From 977f26f692790fa2d024e9f3d726d34c0fd3616d Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Sat, 17 Apr 2021 17:00:14 +0000
Subject: [PATCH 131/249] Add some common shuffles

---
 crates/core_simd/src/permute.rs   | 83 +++++++++++++++++++++++++++++++
 crates/core_simd/tests/permute.rs | 20 ++++++++
 2 files changed, 103 insertions(+)

diff --git a/crates/core_simd/src/permute.rs b/crates/core_simd/src/permute.rs
index b27b0a9e1412..4f7ffd3a9554 100644
--- a/crates/core_simd/src/permute.rs
+++ b/crates/core_simd/src/permute.rs
@@ -13,6 +13,89 @@ macro_rules! impl_shuffle_lane {
             pub fn shuffle<const IDX: [u32; $n]>(self, second: Self) -> Self {
                 unsafe { crate::intrinsics::$fn(self, second, IDX) }
             }
+
+            /// Reverse the order of the lanes in the vector.
+            #[inline]
+            pub fn reverse(self) -> Self {
+                const fn idx() -> [u32; $n] {
+                    let mut idx = [0u32; $n];
+                    let mut i = 0;
+                    while i < $n {
+                        idx[i] = ($n - i - 1) as u32;
+                        i += 1;
+                    }
+                    idx
+                }
+                self.shuffle::<{ idx() }>(self)
+            }
+
+            /// Interleave two vectors.
+            ///
+            /// The even lanes of the first result contain the lower half of `self`, and the odd
+            /// lanes contain the lower half of `other`.
+            ///
+            /// The even lanes of the second result contain the upper half of `self`, and the odd
+            /// lanes contain the upper half of `other`.
+            #[inline]
+            pub fn interleave(self, other: Self) -> (Self, Self) {
+                const fn lo() -> [u32; $n] {
+                    let mut idx = [0u32; $n];
+                    let mut i = 0;
+                    while i < $n {
+                        let offset = i / 2;
+                        idx[i] = if i % 2 == 0 {
+                            offset
+                        } else {
+                            $n + offset
+                        } as u32;
+                        i += 1;
+                    }
+                    idx
+                }
+                const fn hi() -> [u32; $n] {
+                    let mut idx = [0u32; $n];
+                    let mut i = 0;
+                    while i < $n {
+                        let offset = ($n + i) / 2;
+                        idx[i] = if i % 2 == 0 {
+                            offset
+                        } else {
+                            $n + offset
+                        } as u32;
+                        i += 1;
+                    }
+                    idx
+                }
+                (self.shuffle::<{ lo() }>(other), self.shuffle::<{ hi() }>(other))
+            }
+
+            /// Deinterleave two vectors.
+            ///
+            /// The first result contains the even lanes of `self` and `other` concatenated.
+            ///
+            /// The second result contains the odd lanes of `self` and `other` concatenated.
+            #[inline]
+            pub fn deinterleave(self, other: Self) -> (Self, Self) {
+                const fn even() -> [u32; $n] {
+                    let mut idx = [0u32; $n];
+                    let mut i = 0;
+                    while i < $n {
+                        idx[i] = 2 * i as u32;
+                        i += 1;
+                    }
+                    idx
+                }
+                const fn odd() -> [u32; $n] {
+                    let mut idx = [0u32; $n];
+                    let mut i = 0;
+                    while i < $n {
+                        idx[i] = 1 + 2 * i as u32;
+                        i += 1;
+                    }
+                    idx
+                }
+                (self.shuffle::<{ even() }>(other), self.shuffle::<{ odd() }>(other))
+            }
         }
     }
 }
diff --git a/crates/core_simd/tests/permute.rs b/crates/core_simd/tests/permute.rs
index 1c6c391d8d06..2be43c9cf3ce 100644
--- a/crates/core_simd/tests/permute.rs
+++ b/crates/core_simd/tests/permute.rs
@@ -13,3 +13,23 @@ fn simple_shuffle() {
     let b = a;
     assert_eq!(a.shuffle::<{ [3, 1, 4, 6] }>(b).to_array(), [9, 4, 2, 1]);
 }
+
+#[test]
+#[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
+fn reverse() {
+    let a = SimdU32::from_array([0, 1, 2, 3, 4, 5, 6, 7]);
+    assert_eq!(a.reverse().to_array(), [7, 6, 5, 4, 3, 2, 1, 0]);
+}
+
+#[test]
+#[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
+fn interleave() {
+    let a = SimdU32::from_array([0, 1, 2, 3, 4, 5, 6, 7]);
+    let b = SimdU32::from_array([8, 9, 10, 11, 12, 13, 14, 15]);
+    let (lo, hi) = a.interleave(b);
+    assert_eq!(lo.to_array(), [0, 8, 1, 9, 2, 10, 3, 11]);
+    assert_eq!(hi.to_array(), [4, 12, 5, 13, 6, 14, 7, 15]);
+    let (even, odd) = lo.deinterleave(hi);
+    assert_eq!(even, a);
+    assert_eq!(odd, b);
+}

From 1999c54890d57aec3432335715ce29ec87abfeda Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Sat, 17 Apr 2021 15:21:25 -0400
Subject: [PATCH 132/249] Clarify concatenation order

Co-authored-by: Jubilee <46493976+workingjubilee@users.noreply.github.com>
---
 crates/core_simd/src/permute.rs | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/crates/core_simd/src/permute.rs b/crates/core_simd/src/permute.rs
index 4f7ffd3a9554..ba14a019a395 100644
--- a/crates/core_simd/src/permute.rs
+++ b/crates/core_simd/src/permute.rs
@@ -71,9 +71,9 @@ macro_rules! impl_shuffle_lane {
 
             /// Deinterleave two vectors.
             ///
-            /// The first result contains the even lanes of `self` and `other` concatenated.
+            /// The first result contains the even lanes of `self` and then `other`, concatenated.
             ///
-            /// The second result contains the odd lanes of `self` and `other` concatenated.
+            /// The second result contains the odd lanes of `self` and then `other`, concatenated.
             #[inline]
             pub fn deinterleave(self, other: Self) -> (Self, Self) {
                 const fn even() -> [u32; $n] {

From 9acc11209019b7d3f31bd1945066522ea9c2f88c Mon Sep 17 00:00:00 2001
From: Jubilee Young <workingjubilee@gmail.com>
Date: Tue, 13 Apr 2021 19:24:20 -0700
Subject: [PATCH 133/249] Use fabs intrinsic

---
 crates/core_simd/src/intrinsics.rs   | 3 +++
 crates/core_simd/src/vector/float.rs | 3 +--
 2 files changed, 4 insertions(+), 2 deletions(-)

diff --git a/crates/core_simd/src/intrinsics.rs b/crates/core_simd/src/intrinsics.rs
index 4f89d00deb23..b85a3ad99224 100644
--- a/crates/core_simd/src/intrinsics.rs
+++ b/crates/core_simd/src/intrinsics.rs
@@ -51,6 +51,9 @@ extern "platform-intrinsic" {
     #[cfg(feature = "std")]
     pub(crate) fn simd_ceil<T>(x: T) -> T;
 
+    /// fabs
+    pub(crate) fn simd_fabs<T>(x: T) -> T;
+
     pub(crate) fn simd_eq<T, U>(x: T, y: T) -> U;
     pub(crate) fn simd_ne<T, U>(x: T, y: T) -> U;
     pub(crate) fn simd_lt<T, U>(x: T, y: T) -> U;
diff --git a/crates/core_simd/src/vector/float.rs b/crates/core_simd/src/vector/float.rs
index 393e39023d9c..46e4229ddb59 100644
--- a/crates/core_simd/src/vector/float.rs
+++ b/crates/core_simd/src/vector/float.rs
@@ -32,8 +32,7 @@ macro_rules! impl_float_vector {
             /// equivalently-indexed lane in `self`.
             #[inline]
             pub fn abs(self) -> Self {
-                let no_sign = crate::$bits_ty::splat(!0 >> 1);
-                Self::from_bits(self.to_bits() & no_sign)
+                unsafe { crate::intrinsics::simd_fabs(self) }
             }
         }
 

From 828b274ae75efb984ec6a848ea85868f30c587f9 Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Mon, 19 Apr 2021 23:41:11 +0000
Subject: [PATCH 134/249] Rename sum, product to horizontal_{sum,product}

---
 crates/core_simd/src/reduction.rs    |  8 ++++----
 crates/core_simd/tests/ops_macros.rs | 16 ++++++++--------
 2 files changed, 12 insertions(+), 12 deletions(-)

diff --git a/crates/core_simd/src/reduction.rs b/crates/core_simd/src/reduction.rs
index e1fc82e328ad..86a34e4455d6 100644
--- a/crates/core_simd/src/reduction.rs
+++ b/crates/core_simd/src/reduction.rs
@@ -6,13 +6,13 @@ macro_rules! impl_integer_reductions {
         {
             /// Horizontal wrapping add.  Returns the sum of the lanes of the vector, with wrapping addition.
             #[inline]
-            pub fn wrapping_sum(self) -> $scalar {
+            pub fn horizontal_wrapping_sum(self) -> $scalar {
                 unsafe { crate::intrinsics::simd_reduce_add_ordered(self, 0) }
             }
 
             /// Horizontal wrapping multiply.  Returns the product of the lanes of the vector, with wrapping multiplication.
             #[inline]
-            pub fn wrapping_product(self) -> $scalar {
+            pub fn horizontal_wrapping_product(self) -> $scalar {
                 unsafe { crate::intrinsics::simd_reduce_mul_ordered(self, 1) }
             }
 
@@ -61,7 +61,7 @@ macro_rules! impl_float_reductions {
 
             /// Horizontal add.  Returns the sum of the lanes of the vector.
             #[inline]
-            pub fn sum(self) -> $scalar {
+            pub fn horizontal_sum(self) -> $scalar {
                 // LLVM sum is inaccurate on i586
                 if cfg!(all(target_arch = "x86", not(target_feature = "sse2"))) {
                     self.as_slice().iter().sum()
@@ -72,7 +72,7 @@ macro_rules! impl_float_reductions {
 
             /// Horizontal multiply.  Returns the product of the lanes of the vector.
             #[inline]
-            pub fn product(self) -> $scalar {
+            pub fn horizontal_product(self) -> $scalar {
                 // LLVM product is inaccurate on i586
                 if cfg!(all(target_arch = "x86", not(target_feature = "sse2"))) {
                     self.as_slice().iter().product()
diff --git a/crates/core_simd/tests/ops_macros.rs b/crates/core_simd/tests/ops_macros.rs
index 7ce85b77254a..a1213e39e34f 100644
--- a/crates/core_simd/tests/ops_macros.rs
+++ b/crates/core_simd/tests/ops_macros.rs
@@ -140,20 +140,20 @@ macro_rules! impl_binary_checked_op_test {
 macro_rules! impl_common_integer_tests {
     { $vector:ident, $scalar:ident } => {
         test_helpers::test_lanes! {
-            fn wrapping_sum<const LANES: usize>() {
+            fn horizontal_wrapping_sum<const LANES: usize>() {
                 test_helpers::test_1(&|x| {
                     test_helpers::prop_assert_biteq! (
-                        $vector::<LANES>::from_array(x).wrapping_sum(),
+                        $vector::<LANES>::from_array(x).horizontal_wrapping_sum(),
                         x.iter().copied().fold(0 as $scalar, $scalar::wrapping_add),
                     );
                     Ok(())
                 });
             }
 
-            fn wrapping_product<const LANES: usize>() {
+            fn horizontal_wrapping_product<const LANES: usize>() {
                 test_helpers::test_1(&|x| {
                     test_helpers::prop_assert_biteq! (
-                        $vector::<LANES>::from_array(x).wrapping_product(),
+                        $vector::<LANES>::from_array(x).horizontal_wrapping_product(),
                         x.iter().copied().fold(1 as $scalar, $scalar::wrapping_mul),
                     );
                     Ok(())
@@ -479,20 +479,20 @@ macro_rules! impl_float_tests {
                     ).unwrap();
                 }
 
-                fn sum<const LANES: usize>() {
+                fn horizontal_sum<const LANES: usize>() {
                     test_helpers::test_1(&|x| {
                         test_helpers::prop_assert_biteq! (
-                            Vector::<LANES>::from_array(x).sum(),
+                            Vector::<LANES>::from_array(x).horizontal_sum(),
                             x.iter().sum(),
                         );
                         Ok(())
                     });
                 }
 
-                fn product<const LANES: usize>() {
+                fn horizontal_product<const LANES: usize>() {
                     test_helpers::test_1(&|x| {
                         test_helpers::prop_assert_biteq! (
-                            Vector::<LANES>::from_array(x).product(),
+                            Vector::<LANES>::from_array(x).horizontal_product(),
                             x.iter().product(),
                         );
                         Ok(())

From 7028a5829464be5b0087afc7aedfff064b2e545f Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Tue, 20 Apr 2021 01:51:06 +0000
Subject: [PATCH 135/249] Attempt to clarify interleave/deinterleave

---
 crates/core_simd/src/permute.rs | 38 +++++++++++++++++++++++++++------
 1 file changed, 32 insertions(+), 6 deletions(-)

diff --git a/crates/core_simd/src/permute.rs b/crates/core_simd/src/permute.rs
index ba14a019a395..dd63c69c63db 100644
--- a/crates/core_simd/src/permute.rs
+++ b/crates/core_simd/src/permute.rs
@@ -31,11 +31,24 @@ macro_rules! impl_shuffle_lane {
 
             /// Interleave two vectors.
             ///
-            /// The even lanes of the first result contain the lower half of `self`, and the odd
-            /// lanes contain the lower half of `other`.
+            /// Produces two vectors with lanes taken alternately from `self` and `other`.
             ///
-            /// The even lanes of the second result contain the upper half of `self`, and the odd
-            /// lanes contain the upper half of `other`.
+            /// The first result contains the first `LANES / 2` lanes from `self` and `other`,
+            /// alternating, starting with the first lane of `self`.
+            ///
+            /// The second result contains the last `LANES / 2` lanes from `self` and `other`,
+            /// alternating, starting with the lane `LANES / 2` from the start of `self`.
+            ///
+            /// This particular permutation is efficient on many architectures.
+            ///
+            /// ```
+            /// # use core_simd::SimdU32;
+            /// let a = SimdU32::from_array([0, 1, 2, 3]);
+            /// let b = SimdU32::from_array([4, 5, 6, 7]);
+            /// let (x, y) = a.interleave(b);
+            /// assert_eq!(x.to_array(), [0, 4, 1, 5]);
+            /// assert_eq!(y.to_array(), [2, 6, 3, 7]);
+            /// ```
             #[inline]
             pub fn interleave(self, other: Self) -> (Self, Self) {
                 const fn lo() -> [u32; $n] {
@@ -71,9 +84,22 @@ macro_rules! impl_shuffle_lane {
 
             /// Deinterleave two vectors.
             ///
-            /// The first result contains the even lanes of `self` and then `other`, concatenated.
+            /// The first result takes every other lane of `self` and then `other`, starting with
+            /// the first lane.
             ///
-            /// The second result contains the odd lanes of `self` and then `other`, concatenated.
+            /// The second result takes every other lane of `self` and then `other`, starting with
+            /// the second lane.
+            ///
+            /// This particular permutation is efficient on many architectures.
+            ///
+            /// ```
+            /// # use core_simd::SimdU32;
+            /// let a = SimdU32::from_array([0, 4, 1, 5]);
+            /// let b = SimdU32::from_array([2, 6, 3, 7]);
+            /// let (x, y) = a.deinterleave(b);
+            /// assert_eq!(x.to_array(), [0, 1, 2, 3]);
+            /// assert_eq!(y.to_array(), [4, 5, 6, 7]);
+            /// ```
             #[inline]
             pub fn deinterleave(self, other: Self) -> (Self, Self) {
                 const fn even() -> [u32; $n] {

From 04ee1073237dc77b3742e7a1c0d3740c1df499c4 Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Thu, 22 Apr 2021 22:41:12 +0000
Subject: [PATCH 136/249] Remove wrapping from sum/product fns

---
 crates/core_simd/src/reduction.rs    | 4 ++--
 crates/core_simd/tests/ops_macros.rs | 8 ++++----
 2 files changed, 6 insertions(+), 6 deletions(-)

diff --git a/crates/core_simd/src/reduction.rs b/crates/core_simd/src/reduction.rs
index 86a34e4455d6..382d366dd3d1 100644
--- a/crates/core_simd/src/reduction.rs
+++ b/crates/core_simd/src/reduction.rs
@@ -6,13 +6,13 @@ macro_rules! impl_integer_reductions {
         {
             /// Horizontal wrapping add.  Returns the sum of the lanes of the vector, with wrapping addition.
             #[inline]
-            pub fn horizontal_wrapping_sum(self) -> $scalar {
+            pub fn horizontal_sum(self) -> $scalar {
                 unsafe { crate::intrinsics::simd_reduce_add_ordered(self, 0) }
             }
 
             /// Horizontal wrapping multiply.  Returns the product of the lanes of the vector, with wrapping multiplication.
             #[inline]
-            pub fn horizontal_wrapping_product(self) -> $scalar {
+            pub fn horizontal_product(self) -> $scalar {
                 unsafe { crate::intrinsics::simd_reduce_mul_ordered(self, 1) }
             }
 
diff --git a/crates/core_simd/tests/ops_macros.rs b/crates/core_simd/tests/ops_macros.rs
index a1213e39e34f..37f3b49a3306 100644
--- a/crates/core_simd/tests/ops_macros.rs
+++ b/crates/core_simd/tests/ops_macros.rs
@@ -140,20 +140,20 @@ macro_rules! impl_binary_checked_op_test {
 macro_rules! impl_common_integer_tests {
     { $vector:ident, $scalar:ident } => {
         test_helpers::test_lanes! {
-            fn horizontal_wrapping_sum<const LANES: usize>() {
+            fn horizontal_sum<const LANES: usize>() {
                 test_helpers::test_1(&|x| {
                     test_helpers::prop_assert_biteq! (
-                        $vector::<LANES>::from_array(x).horizontal_wrapping_sum(),
+                        $vector::<LANES>::from_array(x).horizontal_sum(),
                         x.iter().copied().fold(0 as $scalar, $scalar::wrapping_add),
                     );
                     Ok(())
                 });
             }
 
-            fn horizontal_wrapping_product<const LANES: usize>() {
+            fn horizontal_product<const LANES: usize>() {
                 test_helpers::test_1(&|x| {
                     test_helpers::prop_assert_biteq! (
-                        $vector::<LANES>::from_array(x).horizontal_wrapping_product(),
+                        $vector::<LANES>::from_array(x).horizontal_product(),
                         x.iter().copied().fold(1 as $scalar, $scalar::wrapping_mul),
                     );
                     Ok(())

From 1f4e902ee70e178f1009e4242b2dac083bb942d4 Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Sun, 11 Apr 2021 11:42:29 -0400
Subject: [PATCH 137/249] Fix saturating math docs

---
 crates/core_simd/src/math.rs | 22 +++++++++++-----------
 1 file changed, 11 insertions(+), 11 deletions(-)

diff --git a/crates/core_simd/src/math.rs b/crates/core_simd/src/math.rs
index 23ff83f11a1c..e987ec4e9c9f 100644
--- a/crates/core_simd/src/math.rs
+++ b/crates/core_simd/src/math.rs
@@ -1,5 +1,5 @@
 macro_rules! impl_uint_arith {
-    ($(($name:ident, $n:ty)),+) => {
+    ($(($name:ident, $n:ident)),+) => {
         $( impl<const LANES: usize> $name<LANES> where Self: crate::LanesAtMost32 {
 
             /// Lanewise saturating add.
@@ -41,7 +41,7 @@ macro_rules! impl_uint_arith {
 }
 
 macro_rules! impl_int_arith {
-    ($(($name:ident, $n:ty)),+) => {
+    ($(($name:ident, $n:ident)),+) => {
         $( impl<const LANES: usize> $name<LANES> where Self: crate::LanesAtMost32 {
 
             /// Lanewise saturating add.
@@ -83,13 +83,12 @@ macro_rules! impl_int_arith {
             /// As abs(), except the MIN value becomes MAX instead of itself.
             ///
             /// # Examples
+            /// ```
             /// # use core_simd::*;
             #[doc = concat!("# use core::", stringify!($n), "::{MIN, MAX};")]
-            #[doc = concat!("let x = ", stringify!($name), "::splat([MIN, -2, 0, 3]);")]
-            /// let unsat = x.abs();
-            /// let sat = x.saturating_abs();
-            #[doc = concat!("assert_eq!(unsat, ", stringify!($name), "::from_array([MIN, 2, 0, 3]);")]
-            #[doc = concat!("assert_eq!(sat, ", stringify!($name), "::from_array([MAX, 2, 0, 3]));")]
+            #[doc = concat!("let x = ", stringify!($name), "::from_array([MIN, -2, 0, 3]);")]
+            /// let abs = x.saturating_abs();
+            #[doc = concat!("assert_eq!(abs, ", stringify!($name), "::from_array([MAX, 2, 0, 3]));")]
             /// ```
             #[inline]
             pub fn saturating_abs(self) -> Self {
@@ -103,12 +102,13 @@ macro_rules! impl_int_arith {
             /// As neg(), except the MIN value becomes MAX instead of itself.
             ///
             /// # Examples
+            /// ```
             /// # use core_simd::*;
             #[doc = concat!("# use core::", stringify!($n), "::{MIN, MAX};")]
-            #[doc = concat!("let x = ", stringify!($name), "::splat([MIN, -2, 3, MAX]);")]
+            #[doc = concat!("let x = ", stringify!($name), "::from_array([MIN, -2, 3, MAX]);")]
             /// let unsat = -x;
             /// let sat = x.saturating_neg();
-            #[doc = concat!("assert_eq!(unsat, ", stringify!($name), "::from_array([MIN, 2, -3, MIN + 1]);")]
+            #[doc = concat!("assert_eq!(unsat, ", stringify!($name), "::from_array([MIN, 2, -3, MIN + 1]));")]
             #[doc = concat!("assert_eq!(sat, ", stringify!($name), "::from_array([MAX, 2, -3, MIN + 1]));")]
             /// ```
             #[inline]
@@ -121,5 +121,5 @@ macro_rules! impl_int_arith {
 
 use crate::vector::*;
 
-impl_uint_arith! { (SimdU8, u8), (SimdU16, u16), (SimdU32, u32), (SimdU64, u64), (SimdUsize, usize) }
-impl_int_arith! { (SimdI8, i8), (SimdI16, i16), (SimdI32, i32), (SimdI64, i64), (SimdIsize, isize) }
+impl_uint_arith! { (SimdU8, u8), (SimdU16, u16), (SimdU32, u32), (SimdU64, u64), (SimdU128, u128), (SimdUsize, usize) }
+impl_int_arith! { (SimdI8, i8), (SimdI16, i16), (SimdI32, i32), (SimdI64, i64), (SimdI128, i128), (SimdIsize, isize) }

From e8b6bca694098e4865d602ef438458ea52335e6a Mon Sep 17 00:00:00 2001
From: Jubilee Young <workingjubilee@gmail.com>
Date: Tue, 13 Apr 2021 20:19:53 -0700
Subject: [PATCH 138/249] Finish fixing up abs docs

---
 crates/core_simd/src/math.rs | 27 ++++++++++++++++++++++++---
 1 file changed, 24 insertions(+), 3 deletions(-)

diff --git a/crates/core_simd/src/math.rs b/crates/core_simd/src/math.rs
index e987ec4e9c9f..baf92ee097bd 100644
--- a/crates/core_simd/src/math.rs
+++ b/crates/core_simd/src/math.rs
@@ -79,6 +79,25 @@ macro_rules! impl_int_arith {
                 unsafe { crate::intrinsics::simd_saturating_sub(self, second) }
             }
 
+            /// Lanewise absolute value, implemented in Rust.
+            /// Every lane becomes its absolute value.
+            ///
+            /// # Examples
+            /// ```
+            /// # use core_simd::*;
+            #[doc = concat!("# use core::", stringify!($n), "::{MIN, MAX};")]
+            #[doc = concat!("let xs = ", stringify!($name), "::from_array([MIN, MIN +1, -5, 0]);")]
+            #[doc = concat!("assert_eq!(xs.abs(), ", stringify!($name), "::from_array([MIN, MAX, 5, 0]));")]
+            /// ```
+            #[inline]
+            pub fn abs(self) -> Self {
+                let mut xs = self.to_array();
+                for (i, x) in xs.clone().iter().enumerate() {
+                    xs[i] = x.wrapping_abs()
+                }
+                $name::from_array(xs)
+            }
+
             /// Lanewise saturating absolute value, implemented in Rust.
             /// As abs(), except the MIN value becomes MAX instead of itself.
             ///
@@ -86,9 +105,11 @@ macro_rules! impl_int_arith {
             /// ```
             /// # use core_simd::*;
             #[doc = concat!("# use core::", stringify!($n), "::{MIN, MAX};")]
-            #[doc = concat!("let x = ", stringify!($name), "::from_array([MIN, -2, 0, 3]);")]
-            /// let abs = x.saturating_abs();
-            #[doc = concat!("assert_eq!(abs, ", stringify!($name), "::from_array([MAX, 2, 0, 3]));")]
+            #[doc = concat!("let xs = ", stringify!($name), "::from_array([MIN, -2, 0, 3]);")]
+            /// let unsat = xs.abs();
+            /// let sat = xs.saturating_abs();
+            #[doc = concat!("assert_eq!(unsat, ", stringify!($name), "::from_array([MIN, 2, 0, 3]));")]
+            #[doc = concat!("assert_eq!(sat, ", stringify!($name), "::from_array([MAX, 2, 0, 3]));")]
             /// ```
             #[inline]
             pub fn saturating_abs(self) -> Self {

From 91134e614ef7ae4e758cc894eceaae6054ec5631 Mon Sep 17 00:00:00 2001
From: Jubilee Young <workingjubilee@gmail.com>
Date: Tue, 13 Apr 2021 21:15:20 -0700
Subject: [PATCH 139/249] Branchless abs

---
 crates/core_simd/src/math.rs | 8 +++-----
 1 file changed, 3 insertions(+), 5 deletions(-)

diff --git a/crates/core_simd/src/math.rs b/crates/core_simd/src/math.rs
index baf92ee097bd..50e8f2c10c9b 100644
--- a/crates/core_simd/src/math.rs
+++ b/crates/core_simd/src/math.rs
@@ -91,11 +91,9 @@ macro_rules! impl_int_arith {
             /// ```
             #[inline]
             pub fn abs(self) -> Self {
-                let mut xs = self.to_array();
-                for (i, x) in xs.clone().iter().enumerate() {
-                    xs[i] = x.wrapping_abs()
-                }
-                $name::from_array(xs)
+                const SHR: $n = <$n>::BITS as $n - 1;
+                let m = self >> SHR;
+                (self^m) - m
             }
 
             /// Lanewise saturating absolute value, implemented in Rust.

From f06427f4d691897cf54aee64fb618e472d60411f Mon Sep 17 00:00:00 2001
From: Jubilee Young <workingjubilee@gmail.com>
Date: Sun, 25 Apr 2021 16:40:18 -0700
Subject: [PATCH 140/249] Move lanes_at_most_64 to _32

---
 crates/core_simd/src/{lanes_at_most_64.rs => lanes_at_most_32.rs} | 0
 1 file changed, 0 insertions(+), 0 deletions(-)
 rename crates/core_simd/src/{lanes_at_most_64.rs => lanes_at_most_32.rs} (100%)

diff --git a/crates/core_simd/src/lanes_at_most_64.rs b/crates/core_simd/src/lanes_at_most_32.rs
similarity index 100%
rename from crates/core_simd/src/lanes_at_most_64.rs
rename to crates/core_simd/src/lanes_at_most_32.rs

From 92d643b6283b0171873de4c46f708005af025f33 Mon Sep 17 00:00:00 2001
From: Jubilee Young <workingjubilee@gmail.com>
Date: Sun, 25 Apr 2021 16:40:35 -0700
Subject: [PATCH 141/249] Remove Simd{U,I}128

---
 crates/core_simd/src/comparisons.rs             |  2 --
 crates/core_simd/src/fmt.rs                     |  4 ++--
 crates/core_simd/src/lanes_at_most_32.rs        |  2 --
 crates/core_simd/src/lib.rs                     |  4 ++--
 crates/core_simd/src/masks/full_masks.rs        |  6 ------
 crates/core_simd/src/masks/mod.rs               | 16 +---------------
 crates/core_simd/src/math.rs                    |  4 ++--
 crates/core_simd/src/ops.rs                     |  2 --
 crates/core_simd/src/vector/int.rs              | 17 -----------------
 crates/core_simd/src/vector/uint.rs             | 17 -----------------
 crates/core_simd/tests/i128_ops.rs              |  3 ---
 crates/core_simd/tests/mask_ops_impl/mask128.rs |  2 --
 crates/core_simd/tests/mask_ops_impl/mod.rs     |  1 -
 crates/core_simd/tests/u128_ops.rs              |  3 ---
 crates/test_helpers/src/lib.rs                  |  4 ----
 15 files changed, 7 insertions(+), 80 deletions(-)
 delete mode 100644 crates/core_simd/tests/i128_ops.rs
 delete mode 100644 crates/core_simd/tests/mask_ops_impl/mask128.rs
 delete mode 100644 crates/core_simd/tests/u128_ops.rs

diff --git a/crates/core_simd/src/comparisons.rs b/crates/core_simd/src/comparisons.rs
index 455f30dc97ee..988ff857eab5 100644
--- a/crates/core_simd/src/comparisons.rs
+++ b/crates/core_simd/src/comparisons.rs
@@ -71,14 +71,12 @@ implement_mask_ops! {
     SimdI16 => Mask16 (SimdMask16, SimdI16),
     SimdI32 => Mask32 (SimdMask32, SimdI32),
     SimdI64 => Mask64 (SimdMask64, SimdI64),
-    SimdI128 => Mask128 (SimdMask128, SimdI128),
     SimdIsize => MaskSize (SimdMaskSize, SimdIsize),
 
     SimdU8 => Mask8 (SimdMask8, SimdI8),
     SimdU16 => Mask16 (SimdMask16, SimdI16),
     SimdU32 => Mask32 (SimdMask32, SimdI32),
     SimdU64 => Mask64 (SimdMask64, SimdI64),
-    SimdU128 => Mask128 (SimdMask128, SimdI128),
     SimdUsize => MaskSize (SimdMaskSize, SimdIsize),
 
     SimdF32 => Mask32 (SimdMask32, SimdI32),
diff --git a/crates/core_simd/src/fmt.rs b/crates/core_simd/src/fmt.rs
index faf0c20e922e..1d5010843eb7 100644
--- a/crates/core_simd/src/fmt.rs
+++ b/crates/core_simd/src/fmt.rs
@@ -77,8 +77,8 @@ macro_rules! impl_fmt_trait {
 
 impl_fmt_trait! {
     integers:
-        SimdU8, SimdU16, SimdU32, SimdU64, SimdU128,
-        SimdI8, SimdI16, SimdI32, SimdI64, SimdI128,
+        SimdU8, SimdU16, SimdU32, SimdU64,
+        SimdI8, SimdI16, SimdI32, SimdI64,
         SimdUsize, SimdIsize,
 }
 
diff --git a/crates/core_simd/src/lanes_at_most_32.rs b/crates/core_simd/src/lanes_at_most_32.rs
index dc0e02c22a26..1e2f7e952c62 100644
--- a/crates/core_simd/src/lanes_at_most_32.rs
+++ b/crates/core_simd/src/lanes_at_most_32.rs
@@ -18,14 +18,12 @@ impl_for! { SimdU8 }
 impl_for! { SimdU16 }
 impl_for! { SimdU32 }
 impl_for! { SimdU64 }
-impl_for! { SimdU128 }
 impl_for! { SimdUsize }
 
 impl_for! { SimdI8 }
 impl_for! { SimdI16 }
 impl_for! { SimdI32 }
 impl_for! { SimdI64 }
-impl_for! { SimdI128 }
 impl_for! { SimdIsize }
 
 impl_for! { SimdF32 }
diff --git a/crates/core_simd/src/lib.rs b/crates/core_simd/src/lib.rs
index 0fc2641516dd..2d4176ce342c 100644
--- a/crates/core_simd/src/lib.rs
+++ b/crates/core_simd/src/lib.rs
@@ -22,8 +22,8 @@ mod round;
 
 mod math;
 
-mod lanes_at_most_64;
-pub use lanes_at_most_64::LanesAtMost32;
+mod lanes_at_most_32;
+pub use lanes_at_most_32::LanesAtMost32;
 
 mod masks;
 pub use masks::*;
diff --git a/crates/core_simd/src/masks/full_masks.rs b/crates/core_simd/src/masks/full_masks.rs
index a6689ce48c66..60a6cb5fdbe8 100644
--- a/crates/core_simd/src/masks/full_masks.rs
+++ b/crates/core_simd/src/masks/full_masks.rs
@@ -417,12 +417,6 @@ define_mask! {
     struct SimdMask64<const LANES: usize>(crate::SimdI64<LANES>);
 }
 
-define_mask! {
-    /// A mask equivalent to [SimdI128](crate::SimdI128), where all bits in the lane must be either set
-    /// or unset.
-    struct SimdMask128<const LANES: usize>(crate::SimdI128<LANES>);
-}
-
 define_mask! {
     /// A mask equivalent to [SimdIsize](crate::SimdIsize), where all bits in the lane must be either set
     /// or unset.
diff --git a/crates/core_simd/src/masks/mod.rs b/crates/core_simd/src/masks/mod.rs
index 4503187e4b8a..c394c7003a35 100644
--- a/crates/core_simd/src/masks/mod.rs
+++ b/crates/core_simd/src/masks/mod.rs
@@ -7,7 +7,7 @@ pub use full_masks::*;
 mod bitmask;
 pub use bitmask::*;
 
-use crate::{LanesAtMost32, SimdI128, SimdI16, SimdI32, SimdI64, SimdI8, SimdIsize};
+use crate::{LanesAtMost32, SimdI16, SimdI32, SimdI64, SimdI8, SimdIsize};
 
 macro_rules! define_opaque_mask {
     {
@@ -387,14 +387,6 @@ define_opaque_mask! {
     @bits SimdI64
 }
 
-define_opaque_mask! {
-    /// Mask for vectors with `LANES` 128-bit elements.
-    ///
-    /// The layout of this type is unspecified.
-    struct Mask128<const LANES: usize>(SimdMask128<LANES>);
-    @bits SimdI128
-}
-
 define_opaque_mask! {
     /// Mask for vectors with `LANES` pointer-width elements.
     ///
@@ -448,12 +440,6 @@ pub type mask64x4 = Mask64<4>;
 /// Vector of eight 64-bit masks
 pub type mask64x8 = Mask64<8>;
 
-/// Vector of two 128-bit masks
-pub type mask128x2 = Mask128<2>;
-
-/// Vector of four 128-bit masks
-pub type mask128x4 = Mask128<4>;
-
 /// Vector of two pointer-width masks
 pub type masksizex2 = MaskSize<2>;
 
diff --git a/crates/core_simd/src/math.rs b/crates/core_simd/src/math.rs
index 50e8f2c10c9b..6a243dbd1962 100644
--- a/crates/core_simd/src/math.rs
+++ b/crates/core_simd/src/math.rs
@@ -140,5 +140,5 @@ macro_rules! impl_int_arith {
 
 use crate::vector::*;
 
-impl_uint_arith! { (SimdU8, u8), (SimdU16, u16), (SimdU32, u32), (SimdU64, u64), (SimdU128, u128), (SimdUsize, usize) }
-impl_int_arith! { (SimdI8, i8), (SimdI16, i16), (SimdI32, i32), (SimdI64, i64), (SimdI128, i128), (SimdIsize, isize) }
+impl_uint_arith! { (SimdU8, u8), (SimdU16, u16), (SimdU32, u32), (SimdU64, u64), (SimdUsize, usize) }
+impl_int_arith! { (SimdI8, i8), (SimdI16, i16), (SimdI32, i32), (SimdI64, i64), (SimdIsize, isize) }
diff --git a/crates/core_simd/src/ops.rs b/crates/core_simd/src/ops.rs
index 513eeb423d92..c7037d2acbc6 100644
--- a/crates/core_simd/src/ops.rs
+++ b/crates/core_simd/src/ops.rs
@@ -653,7 +653,6 @@ impl_unsigned_int_ops! {
     u16 => SimdU16;
     u32 => SimdU32;
     u64 => SimdU64;
-    u128 => SimdU128;
     usize => SimdUsize;
 }
 
@@ -662,7 +661,6 @@ impl_signed_int_ops! {
     i16 => SimdI16;
     i32 => SimdI32;
     i64 => SimdI64;
-    i128 => SimdI128;
     isize => SimdIsize;
 }
 
diff --git a/crates/core_simd/src/vector/int.rs b/crates/core_simd/src/vector/int.rs
index 24f77cb3e106..30b09a229e9b 100644
--- a/crates/core_simd/src/vector/int.rs
+++ b/crates/core_simd/src/vector/int.rs
@@ -64,17 +64,6 @@ from_transmute_x86! { unsafe isizex4 => __m256i }
 //#[cfg(target_pointer_width = "64")]
 //from_transmute_x86! { unsafe isizex8 => __m512i }
 
-/// A SIMD vector of containing `LANES` `i128` values.
-#[repr(simd)]
-pub struct SimdI128<const LANES: usize>([i128; LANES])
-where
-    Self: crate::LanesAtMost32;
-
-impl_integer_vector! { SimdI128, i128, Mask128, SimdI128 }
-
-from_transmute_x86! { unsafe i128x2 => __m256i }
-//from_transmute_x86! { unsafe i128x4 => __m512i }
-
 /// A SIMD vector of containing `LANES` `i16` values.
 #[repr(simd)]
 pub struct SimdI16<const LANES: usize>([i16; LANES])
@@ -132,12 +121,6 @@ pub type isizex4 = SimdIsize<4>;
 /// Vector of eight `isize` values
 pub type isizex8 = SimdIsize<8>;
 
-/// Vector of two `i128` values
-pub type i128x2 = SimdI128<2>;
-
-/// Vector of four `i128` values
-pub type i128x4 = SimdI128<4>;
-
 /// Vector of four `i16` values
 pub type i16x4 = SimdI16<4>;
 
diff --git a/crates/core_simd/src/vector/uint.rs b/crates/core_simd/src/vector/uint.rs
index 3866b9ca5c6e..53e780520a79 100644
--- a/crates/core_simd/src/vector/uint.rs
+++ b/crates/core_simd/src/vector/uint.rs
@@ -49,17 +49,6 @@ from_transmute_x86! { unsafe usizex4 => __m256i }
 //#[cfg(target_pointer_width = "64")]
 //from_transmute_x86! { unsafe usizex8 => __m512i }
 
-/// A SIMD vector of containing `LANES` `u128` values.
-#[repr(simd)]
-pub struct SimdU128<const LANES: usize>([u128; LANES])
-where
-    Self: crate::LanesAtMost32;
-
-impl_unsigned_vector! { SimdU128, u128 }
-
-from_transmute_x86! { unsafe u128x2 => __m256i }
-//from_transmute_x86! { unsafe u128x4 => __m512i }
-
 /// A SIMD vector of containing `LANES` `u16` values.
 #[repr(simd)]
 pub struct SimdU16<const LANES: usize>([u16; LANES])
@@ -117,12 +106,6 @@ pub type usizex4 = SimdUsize<4>;
 /// Vector of eight `usize` values
 pub type usizex8 = SimdUsize<8>;
 
-/// Vector of two `u128` values
-pub type u128x2 = SimdU128<2>;
-
-/// Vector of four `u128` values
-pub type u128x4 = SimdU128<4>;
-
 /// Vector of four `u16` values
 pub type u16x4 = SimdU16<4>;
 
diff --git a/crates/core_simd/tests/i128_ops.rs b/crates/core_simd/tests/i128_ops.rs
deleted file mode 100644
index 3e3fa1d20682..000000000000
--- a/crates/core_simd/tests/i128_ops.rs
+++ /dev/null
@@ -1,3 +0,0 @@
-#[macro_use]
-mod ops_macros;
-impl_signed_tests! { SimdI128, i128 }
diff --git a/crates/core_simd/tests/mask_ops_impl/mask128.rs b/crates/core_simd/tests/mask_ops_impl/mask128.rs
deleted file mode 100644
index 27ba4e2d29fe..000000000000
--- a/crates/core_simd/tests/mask_ops_impl/mask128.rs
+++ /dev/null
@@ -1,2 +0,0 @@
-mask_tests! { mask128x2, 2 }
-mask_tests! { mask128x4, 4 }
diff --git a/crates/core_simd/tests/mask_ops_impl/mod.rs b/crates/core_simd/tests/mask_ops_impl/mod.rs
index 99d735be2935..ff36af956515 100644
--- a/crates/core_simd/tests/mask_ops_impl/mod.rs
+++ b/crates/core_simd/tests/mask_ops_impl/mod.rs
@@ -5,5 +5,4 @@ mod mask8;
 mod mask16;
 mod mask32;
 mod mask64;
-mod mask128;
 mod masksize;
diff --git a/crates/core_simd/tests/u128_ops.rs b/crates/core_simd/tests/u128_ops.rs
deleted file mode 100644
index 4be7d751ffd8..000000000000
--- a/crates/core_simd/tests/u128_ops.rs
+++ /dev/null
@@ -1,3 +0,0 @@
-#[macro_use]
-mod ops_macros;
-impl_unsigned_tests! { SimdU128, u128 }
diff --git a/crates/test_helpers/src/lib.rs b/crates/test_helpers/src/lib.rs
index b5bfd96dde85..9e8790842b49 100644
--- a/crates/test_helpers/src/lib.rs
+++ b/crates/test_helpers/src/lib.rs
@@ -273,13 +273,11 @@ macro_rules! test_lanes {
                     core_simd::SimdU16<$lanes>: core_simd::LanesAtMost32,
                     core_simd::SimdU32<$lanes>: core_simd::LanesAtMost32,
                     core_simd::SimdU64<$lanes>: core_simd::LanesAtMost32,
-                    core_simd::SimdU128<$lanes>: core_simd::LanesAtMost32,
                     core_simd::SimdUsize<$lanes>: core_simd::LanesAtMost32,
                     core_simd::SimdI8<$lanes>: core_simd::LanesAtMost32,
                     core_simd::SimdI16<$lanes>: core_simd::LanesAtMost32,
                     core_simd::SimdI32<$lanes>: core_simd::LanesAtMost32,
                     core_simd::SimdI64<$lanes>: core_simd::LanesAtMost32,
-                    core_simd::SimdI128<$lanes>: core_simd::LanesAtMost32,
                     core_simd::SimdIsize<$lanes>: core_simd::LanesAtMost32,
                     core_simd::SimdF32<$lanes>: core_simd::LanesAtMost32,
                     core_simd::SimdF64<$lanes>: core_simd::LanesAtMost32,
@@ -345,13 +343,11 @@ macro_rules! test_lanes_panic {
                     core_simd::SimdU16<$lanes>: core_simd::LanesAtMost32,
                     core_simd::SimdU32<$lanes>: core_simd::LanesAtMost32,
                     core_simd::SimdU64<$lanes>: core_simd::LanesAtMost32,
-                    core_simd::SimdU128<$lanes>: core_simd::LanesAtMost32,
                     core_simd::SimdUsize<$lanes>: core_simd::LanesAtMost32,
                     core_simd::SimdI8<$lanes>: core_simd::LanesAtMost32,
                     core_simd::SimdI16<$lanes>: core_simd::LanesAtMost32,
                     core_simd::SimdI32<$lanes>: core_simd::LanesAtMost32,
                     core_simd::SimdI64<$lanes>: core_simd::LanesAtMost32,
-                    core_simd::SimdI128<$lanes>: core_simd::LanesAtMost32,
                     core_simd::SimdIsize<$lanes>: core_simd::LanesAtMost32,
                     core_simd::SimdF32<$lanes>: core_simd::LanesAtMost32,
                     core_simd::SimdF64<$lanes>: core_simd::LanesAtMost32,

From b4fda6ef06be190aa655bb23a9c66e46589daff4 Mon Sep 17 00:00:00 2001
From: Jubilee Young <workingjubilee@gmail.com>
Date: Sun, 25 Apr 2021 15:01:05 -0700
Subject: [PATCH 142/249] Give rounding intrinsics their own modules

---
 crates/core_simd/src/intrinsics.rs   | 22 ++++++----
 crates/core_simd/src/round.rs        |  3 +-
 crates/core_simd/tests/ops_macros.rs | 53 ----------------------
 crates/core_simd/tests/round.rs      | 66 ++++++++++++++++++++++++++++
 4 files changed, 81 insertions(+), 63 deletions(-)
 create mode 100644 crates/core_simd/tests/round.rs

diff --git a/crates/core_simd/src/intrinsics.rs b/crates/core_simd/src/intrinsics.rs
index 57e666873c1b..e8bcca22f684 100644
--- a/crates/core_simd/src/intrinsics.rs
+++ b/crates/core_simd/src/intrinsics.rs
@@ -43,14 +43,6 @@ extern "platform-intrinsic" {
     /// neg/fneg
     pub(crate) fn simd_neg<T>(x: T) -> T;
 
-    // floor
-    #[cfg(feature = "std")]
-    pub(crate) fn simd_floor<T>(x: T) -> T;
-
-    // ceil
-    #[cfg(feature = "std")]
-    pub(crate) fn simd_ceil<T>(x: T) -> T;
-
     /// fabs
     pub(crate) fn simd_fabs<T>(x: T) -> T;
 
@@ -85,3 +77,17 @@ extern "platform-intrinsic" {
     pub(crate) fn simd_reduce_or<T, U>(x: T) -> U;
     pub(crate) fn simd_reduce_xor<T, U>(x: T) -> U;
 }
+
+#[cfg(feature = "std")]
+mod std {
+    extern "platform-intrinsic" {
+        // ceil
+        pub(crate) fn simd_ceil<T>(x: T) -> T;
+
+        // floor
+        pub(crate) fn simd_floor<T>(x: T) -> T;
+    }
+}
+
+#[cfg(feature = "std")]
+pub(crate) use crate::intrinsics::std::*;
diff --git a/crates/core_simd/src/round.rs b/crates/core_simd/src/round.rs
index ccad1aad9c44..1855c1480d26 100644
--- a/crates/core_simd/src/round.rs
+++ b/crates/core_simd/src/round.rs
@@ -2,12 +2,12 @@ macro_rules! implement {
     {
         $type:ident, $int_type:ident
     } => {
+        #[cfg(feature = "std")]
         impl<const LANES: usize> crate::$type<LANES>
         where
             Self: crate::LanesAtMost32,
         {
             /// Returns the largest integer less than or equal to each lane.
-            #[cfg(feature = "std")]
             #[must_use = "method returns a new vector and does not mutate the original value"]
             #[inline]
             pub fn floor(self) -> Self {
@@ -15,7 +15,6 @@ macro_rules! implement {
             }
 
             /// Returns the smallest integer greater than or equal to each lane.
-            #[cfg(feature = "std")]
             #[must_use = "method returns a new vector and does not mutate the original value"]
             #[inline]
             pub fn ceil(self) -> Self {
diff --git a/crates/core_simd/tests/ops_macros.rs b/crates/core_simd/tests/ops_macros.rs
index 37f3b49a3306..9f9992258770 100644
--- a/crates/core_simd/tests/ops_macros.rs
+++ b/crates/core_simd/tests/ops_macros.rs
@@ -353,7 +353,6 @@ macro_rules! impl_float_tests {
         mod $scalar {
             type Vector<const LANES: usize> = core_simd::$vector<LANES>;
             type Scalar = $scalar;
-            type IntScalar = $int_scalar;
 
             impl_unary_op_test!(Vector<LANES>, Scalar, Neg::neg);
             impl_binary_op_test!(Vector<LANES>, Scalar, Add::add, AddAssign::add_assign);
@@ -362,25 +361,6 @@ macro_rules! impl_float_tests {
             impl_binary_op_test!(Vector<LANES>, Scalar, Div::div, DivAssign::div_assign);
             impl_binary_op_test!(Vector<LANES>, Scalar, Rem::rem, RemAssign::rem_assign);
 
-            #[cfg(feature = "std")]
-            test_helpers::test_lanes! {
-                fn ceil<const LANES: usize>() {
-                    test_helpers::test_unary_elementwise(
-                        &Vector::<LANES>::ceil,
-                        &Scalar::ceil,
-                        &|_| true,
-                    )
-                }
-
-                fn floor<const LANES: usize>() {
-                    test_helpers::test_unary_elementwise(
-                        &Vector::<LANES>::floor,
-                        &Scalar::floor,
-                        &|_| true,
-                    )
-                }
-            }
-
             test_helpers::test_lanes! {
                 fn is_sign_positive<const LANES: usize>() {
                     test_helpers::test_unary_mask_elementwise(
@@ -446,39 +426,6 @@ macro_rules! impl_float_tests {
                     )
                 }
 
-                fn round_from_int<const LANES: usize>() {
-                    test_helpers::test_unary_elementwise(
-                        &Vector::<LANES>::round_from_int,
-                        &|x| x as Scalar,
-                        &|_| true,
-                    )
-                }
-
-                fn to_int_unchecked<const LANES: usize>() {
-                    // The maximum integer that can be represented by the equivalently sized float has
-                    // all of the mantissa digits set to 1, pushed up to the MSB.
-                    const ALL_MANTISSA_BITS: IntScalar = ((1 << <Scalar>::MANTISSA_DIGITS) - 1);
-                    const MAX_REPRESENTABLE_VALUE: Scalar =
-                        (ALL_MANTISSA_BITS << (core::mem::size_of::<Scalar>() * 8 - <Scalar>::MANTISSA_DIGITS as usize - 1)) as Scalar;
-
-                    let mut runner = proptest::test_runner::TestRunner::default();
-                    runner.run(
-                        &test_helpers::array::UniformArrayStrategy::new(-MAX_REPRESENTABLE_VALUE..MAX_REPRESENTABLE_VALUE),
-                        |x| {
-                            let result_1 = unsafe { Vector::from_array(x).to_int_unchecked().to_array() };
-                            let result_2 = {
-                                let mut result = [0; LANES];
-                                for (i, o) in x.iter().zip(result.iter_mut()) {
-                                    *o = unsafe { i.to_int_unchecked() };
-                                }
-                                result
-                            };
-                            test_helpers::prop_assert_biteq!(result_1, result_2);
-                            Ok(())
-                        },
-                    ).unwrap();
-                }
-
                 fn horizontal_sum<const LANES: usize>() {
                     test_helpers::test_1(&|x| {
                         test_helpers::prop_assert_biteq! (
diff --git a/crates/core_simd/tests/round.rs b/crates/core_simd/tests/round.rs
new file mode 100644
index 000000000000..dc9c8ad4ad25
--- /dev/null
+++ b/crates/core_simd/tests/round.rs
@@ -0,0 +1,66 @@
+macro_rules! float_rounding_test {
+    { $vector:ident, $scalar:tt, $int_scalar:tt } => {
+        mod $scalar {
+            type Vector<const LANES: usize> = core_simd::$vector<LANES>;
+            type Scalar = $scalar;
+            type IntScalar = $int_scalar;
+
+            #[cfg(feature = "std")]
+            test_helpers::test_lanes! {
+                fn ceil<const LANES: usize>() {
+                    test_helpers::test_unary_elementwise(
+                        &Vector::<LANES>::ceil,
+                        &Scalar::ceil,
+                        &|_| true,
+                    )
+                }
+
+                fn floor<const LANES: usize>() {
+                    test_helpers::test_unary_elementwise(
+                        &Vector::<LANES>::floor,
+                        &Scalar::floor,
+                        &|_| true,
+                    )
+                }
+            }
+
+            test_helpers::test_lanes! {
+                fn from_int<const LANES: usize>() {
+                    test_helpers::test_unary_elementwise(
+                        &Vector::<LANES>::round_from_int,
+                        &|x| x as Scalar,
+                        &|_| true,
+                    )
+                }
+
+                fn to_int_unchecked<const LANES: usize>() {
+                    // The maximum integer that can be represented by the equivalently sized float has
+                    // all of the mantissa digits set to 1, pushed up to the MSB.
+                    const ALL_MANTISSA_BITS: IntScalar = ((1 << <Scalar>::MANTISSA_DIGITS) - 1);
+                    const MAX_REPRESENTABLE_VALUE: Scalar =
+                        (ALL_MANTISSA_BITS << (core::mem::size_of::<Scalar>() * 8 - <Scalar>::MANTISSA_DIGITS as usize - 1)) as Scalar;
+
+                    let mut runner = proptest::test_runner::TestRunner::default();
+                    runner.run(
+                        &test_helpers::array::UniformArrayStrategy::new(-MAX_REPRESENTABLE_VALUE..MAX_REPRESENTABLE_VALUE),
+                        |x| {
+                            let result_1 = unsafe { Vector::from_array(x).to_int_unchecked().to_array() };
+                            let result_2 = {
+                                let mut result = [0; LANES];
+                                for (i, o) in x.iter().zip(result.iter_mut()) {
+                                    *o = unsafe { i.to_int_unchecked() };
+                                }
+                                result
+                            };
+                            test_helpers::prop_assert_biteq!(result_1, result_2);
+                            Ok(())
+                        },
+                    ).unwrap();
+                }
+            }
+        }
+    }
+}
+
+float_rounding_test! { SimdF32, f32, i32 }
+float_rounding_test! { SimdF64, f64, i64 }

From 6ea08d8d5fd207bdd6ebd8b26b1a552a708f36b5 Mon Sep 17 00:00:00 2001
From: Jubilee Young <workingjubilee@gmail.com>
Date: Sun, 25 Apr 2021 15:03:17 -0700
Subject: [PATCH 143/249] Add SIMD round, trunc, fract

---
 crates/core_simd/src/intrinsics.rs |  6 ++++++
 crates/core_simd/src/round.rs      | 29 +++++++++++++++++++++++++----
 crates/core_simd/tests/round.rs    | 24 ++++++++++++++++++++++++
 3 files changed, 55 insertions(+), 4 deletions(-)

diff --git a/crates/core_simd/src/intrinsics.rs b/crates/core_simd/src/intrinsics.rs
index e8bcca22f684..665dc1a51d74 100644
--- a/crates/core_simd/src/intrinsics.rs
+++ b/crates/core_simd/src/intrinsics.rs
@@ -86,6 +86,12 @@ mod std {
 
         // floor
         pub(crate) fn simd_floor<T>(x: T) -> T;
+
+        // round
+        pub(crate) fn simd_round<T>(x: T) -> T;
+
+        // trunc
+        pub(crate) fn simd_trunc<T>(x: T) -> T;
     }
 }
 
diff --git a/crates/core_simd/src/round.rs b/crates/core_simd/src/round.rs
index 1855c1480d26..281851c68ace 100644
--- a/crates/core_simd/src/round.rs
+++ b/crates/core_simd/src/round.rs
@@ -7,18 +7,39 @@ macro_rules! implement {
         where
             Self: crate::LanesAtMost32,
         {
-            /// Returns the largest integer less than or equal to each lane.
+            /// Returns the smallest integer greater than or equal to each lane.
+            #[must_use = "method returns a new vector and does not mutate the original value"]
+            #[inline]
+            pub fn ceil(self) -> Self {
+                unsafe { crate::intrinsics::simd_ceil(self) }
+            }
+
+            /// Returns the largest integer value less than or equal to each lane.
             #[must_use = "method returns a new vector and does not mutate the original value"]
             #[inline]
             pub fn floor(self) -> Self {
                 unsafe { crate::intrinsics::simd_floor(self) }
             }
 
-            /// Returns the smallest integer greater than or equal to each lane.
+            /// Rounds to the nearest integer value. Ties round toward zero.
             #[must_use = "method returns a new vector and does not mutate the original value"]
             #[inline]
-            pub fn ceil(self) -> Self {
-                unsafe { crate::intrinsics::simd_ceil(self) }
+            pub fn round(self) -> Self {
+                unsafe { crate::intrinsics::simd_round(self) }
+            }
+
+            /// Returns the floating point's integer value, with its fractional part removed.
+            #[must_use = "method returns a new vector and does not mutate the original value"]
+            #[inline]
+            pub fn trunc(self) -> Self {
+                unsafe { crate::intrinsics::simd_trunc(self) }
+            }
+
+            /// Returns the floating point's fractional value, with its integer part removed.
+            #[must_use = "method returns a new vector and does not mutate the original value"]
+            #[inline]
+            pub fn fract(self) -> Self {
+                self - self.trunc()
             }
         }
 
diff --git a/crates/core_simd/tests/round.rs b/crates/core_simd/tests/round.rs
index dc9c8ad4ad25..85853c0e8778 100644
--- a/crates/core_simd/tests/round.rs
+++ b/crates/core_simd/tests/round.rs
@@ -22,6 +22,30 @@ macro_rules! float_rounding_test {
                         &|_| true,
                     )
                 }
+
+                fn round<const LANES: usize>() {
+                    test_helpers::test_unary_elementwise(
+                        &Vector::<LANES>::round,
+                        &Scalar::round,
+                        &|_| true,
+                    )
+                }
+
+                fn trunc<const LANES: usize>() {
+                    test_helpers::test_unary_elementwise(
+                        &Vector::<LANES>::trunc,
+                        &Scalar::trunc,
+                        &|_| true,
+                    )
+                }
+
+                fn fract<const LANES: usize>() {
+                    test_helpers::test_unary_elementwise(
+                        &Vector::<LANES>::fract,
+                        &Scalar::fract,
+                        &|_| true,
+                    )
+                }
             }
 
             test_helpers::test_lanes! {

From da42aa5403659a6f1f6f4bc4b65d177f13fb6536 Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Sun, 18 Apr 2021 19:26:07 +0000
Subject: [PATCH 144/249] Begin reducing mask API

---
 crates/core_simd/src/comparisons.rs      |  44 +++--
 crates/core_simd/src/lanes_at_most_32.rs |   4 +-
 crates/core_simd/src/masks/bitmask.rs    |  18 +-
 crates/core_simd/src/masks/full_masks.rs | 203 +----------------------
 crates/core_simd/src/masks/mod.rs        |  77 +++------
 crates/core_simd/src/reduction.rs        |  29 +---
 crates/core_simd/tests/masks.rs          |  23 ---
 crates/test_helpers/src/lib.rs           |   2 -
 8 files changed, 59 insertions(+), 341 deletions(-)

diff --git a/crates/core_simd/src/comparisons.rs b/crates/core_simd/src/comparisons.rs
index 988ff857eab5..f3a1954fda22 100644
--- a/crates/core_simd/src/comparisons.rs
+++ b/crates/core_simd/src/comparisons.rs
@@ -1,7 +1,7 @@
 use crate::LanesAtMost32;
 
 macro_rules! implement_mask_ops {
-    { $($vector:ident => $mask:ident ($inner_mask_ty:ident, $inner_ty:ident),)* } => {
+    { $($vector:ident => $mask:ident ($inner_ty:ident),)* } => {
         $(
             impl<const LANES: usize> crate::$vector<LANES>
             where
@@ -12,8 +12,7 @@ macro_rules! implement_mask_ops {
                 #[inline]
                 pub fn lanes_eq(self, other: Self) -> crate::$mask<LANES> {
                     unsafe {
-                        crate::$inner_mask_ty::from_int_unchecked(crate::intrinsics::simd_eq(self, other))
-                            .into()
+                        crate::$mask::from_int_unchecked(crate::intrinsics::simd_eq(self, other))
                     }
                 }
 
@@ -21,8 +20,7 @@ macro_rules! implement_mask_ops {
                 #[inline]
                 pub fn lanes_ne(self, other: Self) -> crate::$mask<LANES> {
                     unsafe {
-                        crate::$inner_mask_ty::from_int_unchecked(crate::intrinsics::simd_ne(self, other))
-                            .into()
+                        crate::$mask::from_int_unchecked(crate::intrinsics::simd_ne(self, other))
                     }
                 }
 
@@ -30,8 +28,7 @@ macro_rules! implement_mask_ops {
                 #[inline]
                 pub fn lanes_lt(self, other: Self) -> crate::$mask<LANES> {
                     unsafe {
-                        crate::$inner_mask_ty::from_int_unchecked(crate::intrinsics::simd_lt(self, other))
-                            .into()
+                        crate::$mask::from_int_unchecked(crate::intrinsics::simd_lt(self, other))
                     }
                 }
 
@@ -39,8 +36,7 @@ macro_rules! implement_mask_ops {
                 #[inline]
                 pub fn lanes_gt(self, other: Self) -> crate::$mask<LANES> {
                     unsafe {
-                        crate::$inner_mask_ty::from_int_unchecked(crate::intrinsics::simd_gt(self, other))
-                            .into()
+                        crate::$mask::from_int_unchecked(crate::intrinsics::simd_gt(self, other))
                     }
                 }
 
@@ -48,8 +44,7 @@ macro_rules! implement_mask_ops {
                 #[inline]
                 pub fn lanes_le(self, other: Self) -> crate::$mask<LANES> {
                     unsafe {
-                        crate::$inner_mask_ty::from_int_unchecked(crate::intrinsics::simd_le(self, other))
-                            .into()
+                        crate::$mask::from_int_unchecked(crate::intrinsics::simd_le(self, other))
                     }
                 }
 
@@ -57,8 +52,7 @@ macro_rules! implement_mask_ops {
                 #[inline]
                 pub fn lanes_ge(self, other: Self) -> crate::$mask<LANES> {
                     unsafe {
-                        crate::$inner_mask_ty::from_int_unchecked(crate::intrinsics::simd_ge(self, other))
-                            .into()
+                        crate::$mask::from_int_unchecked(crate::intrinsics::simd_ge(self, other))
                     }
                 }
             }
@@ -67,18 +61,18 @@ macro_rules! implement_mask_ops {
 }
 
 implement_mask_ops! {
-    SimdI8 => Mask8 (SimdMask8, SimdI8),
-    SimdI16 => Mask16 (SimdMask16, SimdI16),
-    SimdI32 => Mask32 (SimdMask32, SimdI32),
-    SimdI64 => Mask64 (SimdMask64, SimdI64),
-    SimdIsize => MaskSize (SimdMaskSize, SimdIsize),
+    SimdI8 => Mask8 (SimdI8),
+    SimdI16 => Mask16 (SimdI16),
+    SimdI32 => Mask32 (SimdI32),
+    SimdI64 => Mask64 (SimdI64),
+    SimdIsize => MaskSize (SimdIsize),
 
-    SimdU8 => Mask8 (SimdMask8, SimdI8),
-    SimdU16 => Mask16 (SimdMask16, SimdI16),
-    SimdU32 => Mask32 (SimdMask32, SimdI32),
-    SimdU64 => Mask64 (SimdMask64, SimdI64),
-    SimdUsize => MaskSize (SimdMaskSize, SimdIsize),
+    SimdU8 => Mask8 (SimdI8),
+    SimdU16 => Mask16 (SimdI16),
+    SimdU32 => Mask32 (SimdI32),
+    SimdU64 => Mask64 (SimdI64),
+    SimdUsize => MaskSize (SimdIsize),
 
-    SimdF32 => Mask32 (SimdMask32, SimdI32),
-    SimdF64 => Mask64 (SimdMask64, SimdI64),
+    SimdF32 => Mask32 (SimdI32),
+    SimdF64 => Mask64 (SimdI64),
 }
diff --git a/crates/core_simd/src/lanes_at_most_32.rs b/crates/core_simd/src/lanes_at_most_32.rs
index 1e2f7e952c62..2fee9ca91891 100644
--- a/crates/core_simd/src/lanes_at_most_32.rs
+++ b/crates/core_simd/src/lanes_at_most_32.rs
@@ -1,4 +1,4 @@
-/// Implemented for bitmask sizes that are supported by the implementation.
+/// Implemented for vectors that are supported by the implementation.
 pub trait LanesAtMost32 {}
 
 macro_rules! impl_for {
@@ -28,5 +28,3 @@ impl_for! { SimdIsize }
 
 impl_for! { SimdF32 }
 impl_for! { SimdF64 }
-
-impl_for! { BitMask }
diff --git a/crates/core_simd/src/masks/bitmask.rs b/crates/core_simd/src/masks/bitmask.rs
index b4d1b6d9557d..32e2ffb86153 100644
--- a/crates/core_simd/src/masks/bitmask.rs
+++ b/crates/core_simd/src/masks/bitmask.rs
@@ -4,14 +4,12 @@ use crate::LanesAtMost32;
 #[derive(Copy, Clone, Debug, PartialOrd, PartialEq, Ord, Eq, Hash)]
 #[repr(transparent)]
 pub struct BitMask<const LANES: usize>(u64)
-where
-    BitMask<LANES>: LanesAtMost32;
 
 impl<const LANES: usize> BitMask<LANES>
 where
     Self: LanesAtMost32,
 {
-    /// Construct a mask by setting all lanes to the given value.
+    #[inline]
     pub fn splat(value: bool) -> Self {
         if value {
             Self(u64::MAX >> (64 - LANES))
@@ -20,23 +18,13 @@ where
         }
     }
 
-    /// Tests the value of the specified lane.
-    ///
-    /// # Panics
-    /// Panics if `lane` is greater than or equal to the number of lanes in the vector.
     #[inline]
-    pub fn test(&self, lane: usize) -> bool {
-        assert!(lane < LANES, "lane index out of range");
+    pub unsafe fn test_unchecked(&self, lane: usize) -> bool {
         (self.0 >> lane) & 0x1 > 0
     }
 
-    /// Sets the value of the specified lane.
-    ///
-    /// # Panics
-    /// Panics if `lane` is greater than or equal to the number of lanes in the vector.
     #[inline]
-    pub fn set(&mut self, lane: usize, value: bool) {
-        assert!(lane < LANES, "lane index out of range");
+    pub unsafe fn set_unchecked(&mut self, lane: usize, value: bool) {
         self.0 ^= ((value ^ self.test(lane)) as u64) << lane
     }
 }
diff --git a/crates/core_simd/src/masks/full_masks.rs b/crates/core_simd/src/masks/full_masks.rs
index 60a6cb5fdbe8..6972a4216b68 100644
--- a/crates/core_simd/src/masks/full_masks.rs
+++ b/crates/core_simd/src/masks/full_masks.rs
@@ -1,18 +1,5 @@
 //! Masks that take up full SIMD vector registers.
 
-/// The error type returned when converting an integer to a mask fails.
-#[derive(Debug, Copy, Clone, PartialEq, Eq)]
-pub struct TryFromMaskError(());
-
-impl core::fmt::Display for TryFromMaskError {
-    fn fmt(&self, f: &mut core::fmt::Formatter) -> core::fmt::Result {
-        write!(
-            f,
-            "mask vector must have all bits set or unset in each lane"
-        )
-    }
-}
-
 macro_rules! define_mask {
     {
         $(#[$attr:meta])*
@@ -27,6 +14,8 @@ macro_rules! define_mask {
         where
             crate::$type<LANES>: crate::LanesAtMost32;
 
+        impl_full_mask_reductions! { $name, $type }
+
         impl<const LANES: usize> Copy for $name<LANES>
         where
             crate::$type<LANES>: crate::LanesAtMost32,
@@ -46,7 +35,6 @@ macro_rules! define_mask {
         where
             crate::$type<LANES>: crate::LanesAtMost32,
         {
-            /// Construct a mask by setting all lanes to the given value.
             pub fn splat(value: bool) -> Self {
                 Self(<crate::$type<LANES>>::splat(
                     if value {
@@ -57,20 +45,12 @@ macro_rules! define_mask {
                 ))
             }
 
-            /// Tests the value of the specified lane.
-            ///
-            /// # Panics
-            /// Panics if `lane` is greater than or equal to the number of lanes in the vector.
             #[inline]
             pub fn test(&self, lane: usize) -> bool {
                 assert!(lane < LANES, "lane index out of range");
                 self.0[lane] == -1
             }
 
-            /// Sets the value of the specified lane.
-            ///
-            /// # Panics
-            /// Panics if `lane` is greater than or equal to the number of lanes in the vector.
             #[inline]
             pub fn set(&mut self, lane: usize, value: bool) {
                 assert!(lane < LANES, "lane index out of range");
@@ -81,56 +61,15 @@ macro_rules! define_mask {
                 }
             }
 
-            /// Converts the mask to the equivalent integer representation, where -1 represents
-            /// "set" and 0 represents "unset".
             #[inline]
             pub fn to_int(self) -> crate::$type<LANES> {
                 self.0
             }
 
-            /// Creates a  mask from the equivalent integer representation, where -1 represents
-            /// "set" and 0 represents "unset".
-            ///
-            /// Each provided lane must be either 0 or -1.
             #[inline]
             pub unsafe fn from_int_unchecked(value: crate::$type<LANES>) -> Self {
                 Self(value)
             }
-
-            /// Creates a mask from the equivalent integer representation, where -1 represents
-            /// "set" and 0 represents "unset".
-            ///
-            /// # Panics
-            /// Panics if any lane is not 0 or -1.
-            #[inline]
-            pub fn from_int(value: crate::$type<LANES>) -> Self {
-                use core::convert::TryInto;
-                value.try_into().unwrap()
-            }
-        }
-
-        impl<const LANES: usize> core::convert::From<bool> for $name<LANES>
-        where
-            crate::$type<LANES>: crate::LanesAtMost32,
-        {
-            fn from(value: bool) -> Self {
-                Self::splat(value)
-            }
-        }
-
-        impl<const LANES: usize> core::convert::TryFrom<crate::$type<LANES>> for $name<LANES>
-        where
-            crate::$type<LANES>: crate::LanesAtMost32,
-        {
-            type Error = TryFromMaskError;
-            fn try_from(value: crate::$type<LANES>) -> Result<Self, Self::Error> {
-                let valid = (value.lanes_eq(crate::$type::<LANES>::splat(0)) | value.lanes_eq(crate::$type::<LANES>::splat(-1))).all();
-                if valid {
-                    Ok(Self(value))
-                } else {
-                    Err(TryFromMaskError(()))
-                }
-            }
         }
 
         impl<const LANES: usize> core::convert::From<$name<LANES>> for crate::$type<LANES>
@@ -142,36 +81,6 @@ macro_rules! define_mask {
             }
         }
 
-        impl<const LANES: usize> core::convert::From<crate::BitMask<LANES>> for $name<LANES>
-        where
-            crate::$type<LANES>: crate::LanesAtMost32,
-            crate::BitMask<LANES>: crate::LanesAtMost32,
-        {
-            fn from(value: crate::BitMask<LANES>) -> Self {
-                // TODO use an intrinsic to do this efficiently (with LLVM's sext instruction)
-                let mut mask = Self::splat(false);
-                for lane in 0..LANES {
-                    mask.set(lane, value.test(lane));
-                }
-                mask
-            }
-        }
-
-        impl<const LANES: usize> core::convert::From<$name<LANES>> for crate::BitMask<LANES>
-        where
-            crate::$type<LANES>: crate::LanesAtMost32,
-            crate::BitMask<LANES>: crate::LanesAtMost32,
-        {
-            fn from(value: $name<$lanes>) -> Self {
-                // TODO use an intrinsic to do this efficiently (with LLVM's trunc instruction)
-                let mut mask = Self::splat(false);
-                for lane in 0..LANES {
-                    mask.set(lane, value.test(lane));
-                }
-                mask
-            }
-        }
-
         impl<const LANES: usize> core::fmt::Debug for $name<LANES>
         where
             crate::$type<LANES>: crate::LanesAtMost32,
@@ -230,28 +139,6 @@ macro_rules! define_mask {
             }
         }
 
-        impl<const LANES: usize> core::ops::BitAnd<bool> for $name<LANES>
-        where
-            crate::$type<LANES>: crate::LanesAtMost32,
-        {
-            type Output = Self;
-            #[inline]
-            fn bitand(self, rhs: bool) -> Self {
-                self & Self::splat(rhs)
-            }
-        }
-
-        impl<const LANES: usize> core::ops::BitAnd<$name<LANES>> for bool
-        where
-            crate::$type<LANES>: crate::LanesAtMost32,
-        {
-            type Output = $name<LANES>;
-            #[inline]
-            fn bitand(self, rhs: $name<LANES>) -> $name<LANES> {
-                $name::<LANES>::splat(self) & rhs
-            }
-        }
-
         impl<const LANES: usize> core::ops::BitOr for $name<LANES>
         where
             crate::$type<LANES>: crate::LanesAtMost32,
@@ -263,28 +150,6 @@ macro_rules! define_mask {
             }
         }
 
-        impl<const LANES: usize> core::ops::BitOr<bool> for $name<LANES>
-        where
-            crate::$type<LANES>: crate::LanesAtMost32,
-        {
-            type Output = Self;
-            #[inline]
-            fn bitor(self, rhs: bool) -> Self {
-                self | Self::splat(rhs)
-            }
-        }
-
-        impl<const LANES: usize> core::ops::BitOr<$name<LANES>> for bool
-        where
-            crate::$type<LANES>: crate::LanesAtMost32,
-        {
-            type Output = $name<LANES>;
-            #[inline]
-            fn bitor(self, rhs: $name<LANES>) -> $name<LANES> {
-                $name::<LANES>::splat(self) | rhs
-            }
-        }
-
         impl<const LANES: usize> core::ops::BitXor for $name<LANES>
         where
             crate::$type<LANES>: crate::LanesAtMost32,
@@ -296,28 +161,6 @@ macro_rules! define_mask {
             }
         }
 
-        impl<const LANES: usize> core::ops::BitXor<bool> for $name<LANES>
-        where
-            crate::$type<LANES>: crate::LanesAtMost32,
-        {
-            type Output = Self;
-            #[inline]
-            fn bitxor(self, rhs: bool) -> Self::Output {
-                self ^ Self::splat(rhs)
-            }
-        }
-
-        impl<const LANES: usize> core::ops::BitXor<$name<LANES>> for bool
-        where
-            crate::$type<LANES>: crate::LanesAtMost32,
-        {
-            type Output = $name<LANES>;
-            #[inline]
-            fn bitxor(self, rhs: $name<LANES>) -> Self::Output {
-                $name::<LANES>::splat(self) ^ rhs
-            }
-        }
-
         impl<const LANES: usize> core::ops::Not for $name<LANES>
         where
             crate::$type<LANES>: crate::LanesAtMost32,
@@ -339,16 +182,6 @@ macro_rules! define_mask {
             }
         }
 
-        impl<const LANES: usize> core::ops::BitAndAssign<bool> for $name<LANES>
-        where
-            crate::$type<LANES>: crate::LanesAtMost32,
-        {
-            #[inline]
-            fn bitand_assign(&mut self, rhs: bool) {
-                *self &= Self::splat(rhs);
-            }
-        }
-
         impl<const LANES: usize> core::ops::BitOrAssign for $name<LANES>
         where
             crate::$type<LANES>: crate::LanesAtMost32,
@@ -359,16 +192,6 @@ macro_rules! define_mask {
             }
         }
 
-        impl<const LANES: usize> core::ops::BitOrAssign<bool> for $name<LANES>
-        where
-            crate::$type<LANES>: crate::LanesAtMost32,
-        {
-            #[inline]
-            fn bitor_assign(&mut self, rhs: bool) {
-                *self |= Self::splat(rhs);
-            }
-        }
-
         impl<const LANES: usize> core::ops::BitXorAssign for $name<LANES>
         where
             crate::$type<LANES>: crate::LanesAtMost32,
@@ -378,47 +201,35 @@ macro_rules! define_mask {
                 self.0 ^= rhs.0;
             }
         }
-
-        impl<const LANES: usize> core::ops::BitXorAssign<bool> for $name<LANES>
-        where
-            crate::$type<LANES>: crate::LanesAtMost32,
-        {
-            #[inline]
-            fn bitxor_assign(&mut self, rhs: bool) {
-                *self ^= Self::splat(rhs);
-            }
-        }
-
-        impl_full_mask_reductions! { $name, $type }
     }
 }
 
 define_mask! {
     /// A mask equivalent to [SimdI8](crate::SimdI8), where all bits in the lane must be either set
     /// or unset.
-    struct SimdMask8<const LANES: usize>(crate::SimdI8<LANES>);
+    struct Mask8<const LANES: usize>(crate::SimdI8<LANES>);
 }
 
 define_mask! {
     /// A mask equivalent to [SimdI16](crate::SimdI16), where all bits in the lane must be either set
     /// or unset.
-    struct SimdMask16<const LANES: usize>(crate::SimdI16<LANES>);
+    struct Mask16<const LANES: usize>(crate::SimdI16<LANES>);
 }
 
 define_mask! {
     /// A mask equivalent to [SimdI32](crate::SimdI32), where all bits in the lane must be either set
     /// or unset.
-    struct SimdMask32<const LANES: usize>(crate::SimdI32<LANES>);
+    struct Mask32<const LANES: usize>(crate::SimdI32<LANES>);
 }
 
 define_mask! {
     /// A mask equivalent to [SimdI64](crate::SimdI64), where all bits in the lane must be either set
     /// or unset.
-    struct SimdMask64<const LANES: usize>(crate::SimdI64<LANES>);
+    struct Mask64<const LANES: usize>(crate::SimdI64<LANES>);
 }
 
 define_mask! {
     /// A mask equivalent to [SimdIsize](crate::SimdIsize), where all bits in the lane must be either set
     /// or unset.
-    struct SimdMaskSize<const LANES: usize>(crate::SimdIsize<LANES>);
+    struct MaskSize<const LANES: usize>(crate::SimdIsize<LANES>);
 }
diff --git a/crates/core_simd/src/masks/mod.rs b/crates/core_simd/src/masks/mod.rs
index c394c7003a35..fbb934b96424 100644
--- a/crates/core_simd/src/masks/mod.rs
+++ b/crates/core_simd/src/masks/mod.rs
@@ -1,25 +1,24 @@
 //! Types and traits associated with masking lanes of vectors.
+//! Types representing
 #![allow(non_camel_case_types)]
 
-mod full_masks;
-pub use full_masks::*;
-
-mod bitmask;
-pub use bitmask::*;
+#[cfg_attr(not(all(target_arch = "x86_64", target_feature = "avx512f")), path = "full_masks.rs")]
+#[cfg_attr(all(target_arch = "x86_64", target_feature = "avx512f"), path = "bitmask.rs")]
+mod mask_impl;
 
 use crate::{LanesAtMost32, SimdI16, SimdI32, SimdI64, SimdI8, SimdIsize};
 
 macro_rules! define_opaque_mask {
     {
         $(#[$attr:meta])*
-        struct $name:ident<const $lanes:ident: usize>($inner_ty:ident<$lanes2:ident>);
+        struct $name:ident<const $lanes:ident: usize>($inner_ty:ty);
         @bits $bits_ty:ident
     } => {
         $(#[$attr])*
         #[allow(non_camel_case_types)]
-        pub struct $name<const LANES: usize>($inner_ty<LANES>) where $bits_ty<LANES>: LanesAtMost32;
+        pub struct $name<const LANES: usize>($inner_ty) where $bits_ty<LANES>: LanesAtMost32;
 
-        impl_opaque_mask_reductions! { $name, $inner_ty, $bits_ty }
+        impl_opaque_mask_reductions! { $name, $bits_ty }
 
         impl<const LANES: usize> $name<LANES>
         where
@@ -27,7 +26,7 @@ macro_rules! define_opaque_mask {
         {
             /// Construct a mask by setting all lanes to the given value.
             pub fn splat(value: bool) -> Self {
-                Self(<$inner_ty<LANES>>::splat(value))
+                Self(<$inner_ty>::splat(value))
             }
 
             /// Converts an array to a SIMD vector.
@@ -52,6 +51,16 @@ macro_rules! define_opaque_mask {
                 array
             }
 
+            /// Converts a vector of integers to a mask, where 0 represents `false` and -1
+            /// represents `true`.
+            ///
+            /// # Safety
+            /// All lanes must be either 0 or -1.
+            #[inline]
+            pub unsafe fn from_int_unchecked(value: $bits_ty<LANES>) -> Self {
+                Self(<$inner_ty>::from_int_unchecked(value))
+            }
+
             /// Tests the value of the specified lane.
             ///
             /// # Panics
@@ -71,44 +80,6 @@ macro_rules! define_opaque_mask {
             }
         }
 
-        impl<const LANES: usize> From<BitMask<LANES>> for $name<LANES>
-        where
-            $bits_ty<LANES>: LanesAtMost32,
-            BitMask<LANES>: LanesAtMost32,
-        {
-            fn from(value: BitMask<LANES>) -> Self {
-                Self(value.into())
-            }
-        }
-
-        impl<const LANES: usize> From<$name<LANES>> for crate::BitMask<LANES>
-        where
-            $bits_ty<LANES>: LanesAtMost32,
-            BitMask<LANES>: LanesAtMost32,
-        {
-            fn from(value: $name<LANES>) -> Self {
-                value.0.into()
-            }
-        }
-
-        impl<const LANES: usize> From<$inner_ty<LANES>> for $name<LANES>
-        where
-            $bits_ty<LANES>: LanesAtMost32,
-        {
-            fn from(value: $inner_ty<LANES>) -> Self {
-                Self(value)
-            }
-        }
-
-        impl<const LANES: usize> From<$name<LANES>> for $inner_ty<LANES>
-        where
-            $bits_ty<LANES>: LanesAtMost32,
-        {
-            fn from(value: $name<LANES>) -> Self {
-                value.0
-            }
-        }
-
         // vector/array conversion
         impl<const LANES: usize> From<[bool; LANES]> for $name<LANES>
         where
@@ -130,7 +101,7 @@ macro_rules! define_opaque_mask {
 
         impl<const LANES: usize> Copy for $name<LANES>
         where
-            $inner_ty<LANES>: Copy,
+            $inner_ty: Copy,
             $bits_ty<LANES>: LanesAtMost32,
         {}
 
@@ -359,7 +330,7 @@ define_opaque_mask! {
     /// Mask for vectors with `LANES` 8-bit elements.
     ///
     /// The layout of this type is unspecified.
-    struct Mask8<const LANES: usize>(SimdMask8<LANES>);
+    struct Mask8<const LANES: usize>(mask_impl::Mask8<LANES>);
     @bits SimdI8
 }
 
@@ -367,7 +338,7 @@ define_opaque_mask! {
     /// Mask for vectors with `LANES` 16-bit elements.
     ///
     /// The layout of this type is unspecified.
-    struct Mask16<const LANES: usize>(SimdMask16<LANES>);
+    struct Mask16<const LANES: usize>(mask_impl::Mask16<LANES>);
     @bits SimdI16
 }
 
@@ -375,7 +346,7 @@ define_opaque_mask! {
     /// Mask for vectors with `LANES` 32-bit elements.
     ///
     /// The layout of this type is unspecified.
-    struct Mask32<const LANES: usize>(SimdMask32<LANES>);
+    struct Mask32<const LANES: usize>(mask_impl::Mask32<LANES>);
     @bits SimdI32
 }
 
@@ -383,7 +354,7 @@ define_opaque_mask! {
     /// Mask for vectors with `LANES` 64-bit elements.
     ///
     /// The layout of this type is unspecified.
-    struct Mask64<const LANES: usize>(SimdMask64<LANES>);
+    struct Mask64<const LANES: usize>(mask_impl::Mask64<LANES>);
     @bits SimdI64
 }
 
@@ -391,7 +362,7 @@ define_opaque_mask! {
     /// Mask for vectors with `LANES` pointer-width elements.
     ///
     /// The layout of this type is unspecified.
-    struct MaskSize<const LANES: usize>(SimdMaskSize<LANES>);
+    struct MaskSize<const LANES: usize>(mask_impl::MaskSize<LANES>);
     @bits SimdIsize
 }
 
diff --git a/crates/core_simd/src/reduction.rs b/crates/core_simd/src/reduction.rs
index 382d366dd3d1..2d4f1bca2647 100644
--- a/crates/core_simd/src/reduction.rs
+++ b/crates/core_simd/src/reduction.rs
@@ -103,18 +103,16 @@ macro_rules! impl_float_reductions {
 }
 
 macro_rules! impl_full_mask_reductions {
-    { $name:ident, $inner:ident } => {
-        impl<const LANES: usize> crate::$name<LANES>
+    { $name:ident, $bits_ty:ident } => {
+        impl<const LANES: usize> $name<LANES>
         where
-            crate::$inner<LANES>: crate::LanesAtMost32
+            crate::$bits_ty<LANES>: crate::LanesAtMost32
         {
-            /// Returns true if any lane is set, or false otherwise.
             #[inline]
             pub fn any(self) -> bool {
                 unsafe { crate::intrinsics::simd_reduce_any(self.to_int()) }
             }
 
-            /// Returns true if all lanes are set, or false otherwise.
             #[inline]
             pub fn all(self) -> bool {
                 unsafe { crate::intrinsics::simd_reduce_all(self.to_int()) }
@@ -124,10 +122,10 @@ macro_rules! impl_full_mask_reductions {
 }
 
 macro_rules! impl_opaque_mask_reductions {
-    { $name:ident, $inner:ident, $bits_ty:ident } => {
+    { $name:ident, $bits_ty:ident } => {
         impl<const LANES: usize> $name<LANES>
         where
-            $bits_ty<LANES>: crate::LanesAtMost32
+            crate::$bits_ty<LANES>: crate::LanesAtMost32
         {
             /// Returns true if any lane is set, or false otherwise.
             #[inline]
@@ -143,20 +141,3 @@ macro_rules! impl_opaque_mask_reductions {
         }
     }
 }
-
-impl<const LANES: usize> crate::BitMask<LANES>
-where
-    crate::BitMask<LANES>: crate::LanesAtMost32,
-{
-    /// Returns true if any lane is set, or false otherwise.
-    #[inline]
-    pub fn any(self) -> bool {
-        self != Self::splat(false)
-    }
-
-    /// Returns true if all lanes are set, or false otherwise.
-    #[inline]
-    pub fn all(self) -> bool {
-        self == Self::splat(true)
-    }
-}
diff --git a/crates/core_simd/tests/masks.rs b/crates/core_simd/tests/masks.rs
index 59da77de622b..6c3993e39a93 100644
--- a/crates/core_simd/tests/masks.rs
+++ b/crates/core_simd/tests/masks.rs
@@ -1,30 +1,9 @@
-use core::convert::TryFrom;
-use core_simd::{BitMask, Mask8, SimdI8, SimdMask8};
-
 #[cfg(target_arch = "wasm32")]
 use wasm_bindgen_test::*;
 
 #[cfg(target_arch = "wasm32")]
 wasm_bindgen_test_configure!(run_in_browser);
 
-#[test]
-#[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
-fn mask_format_round_trip() {
-    let ints = SimdI8::from_array([-1, 0, 0, -1]);
-
-    let simd_mask = SimdMask8::try_from(ints).unwrap();
-
-    let bitmask = BitMask::from(simd_mask);
-
-    let opaque_mask = Mask8::from(bitmask);
-
-    let simd_mask_returned = SimdMask8::from(opaque_mask);
-
-    let ints_returned = SimdI8::from(simd_mask_returned);
-
-    assert_eq!(ints_returned, ints);
-}
-
 macro_rules! test_mask_api {
     { $name:ident } => {
         #[allow(non_snake_case)]
@@ -83,6 +62,4 @@ macro_rules! test_mask_api {
 
 mod mask_api {
     test_mask_api! { Mask8 }
-    test_mask_api! { SimdMask8 }
-    test_mask_api! { BitMask }
 }
diff --git a/crates/test_helpers/src/lib.rs b/crates/test_helpers/src/lib.rs
index 9e8790842b49..8b56877967cb 100644
--- a/crates/test_helpers/src/lib.rs
+++ b/crates/test_helpers/src/lib.rs
@@ -281,7 +281,6 @@ macro_rules! test_lanes {
                     core_simd::SimdIsize<$lanes>: core_simd::LanesAtMost32,
                     core_simd::SimdF32<$lanes>: core_simd::LanesAtMost32,
                     core_simd::SimdF64<$lanes>: core_simd::LanesAtMost32,
-                    core_simd::BitMask<$lanes>: core_simd::LanesAtMost32,
                 $body
 
                 #[cfg(target_arch = "wasm32")]
@@ -351,7 +350,6 @@ macro_rules! test_lanes_panic {
                     core_simd::SimdIsize<$lanes>: core_simd::LanesAtMost32,
                     core_simd::SimdF32<$lanes>: core_simd::LanesAtMost32,
                     core_simd::SimdF64<$lanes>: core_simd::LanesAtMost32,
-                    core_simd::BitMask<$lanes>: core_simd::LanesAtMost32,
                 $body
 
                 #[test]

From eec42808aa024d354bb40ec890612c37ba4a496c Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Mon, 19 Apr 2021 04:31:43 +0000
Subject: [PATCH 145/249] Update bitmask API

---
 crates/core_simd/src/intrinsics.rs       |   3 +
 crates/core_simd/src/lanes_at_most_32.rs |  38 ++++--
 crates/core_simd/src/masks/bitmask.rs    | 146 ++++++++---------------
 crates/core_simd/src/masks/full_masks.rs |  59 ++-------
 crates/core_simd/src/masks/mod.rs        |  93 ++++++++++++++-
 crates/core_simd/tests/masks.rs          |  17 +++
 6 files changed, 196 insertions(+), 160 deletions(-)

diff --git a/crates/core_simd/src/intrinsics.rs b/crates/core_simd/src/intrinsics.rs
index 665dc1a51d74..1812a9c624dc 100644
--- a/crates/core_simd/src/intrinsics.rs
+++ b/crates/core_simd/src/intrinsics.rs
@@ -76,6 +76,9 @@ extern "platform-intrinsic" {
     pub(crate) fn simd_reduce_and<T, U>(x: T) -> U;
     pub(crate) fn simd_reduce_or<T, U>(x: T) -> U;
     pub(crate) fn simd_reduce_xor<T, U>(x: T) -> U;
+
+    // truncate integer vector to bitmask
+    pub(crate) fn simd_bitmask<T, U>(x: T) -> U;
 }
 
 #[cfg(feature = "std")]
diff --git a/crates/core_simd/src/lanes_at_most_32.rs b/crates/core_simd/src/lanes_at_most_32.rs
index 2fee9ca91891..2d84b1306ea5 100644
--- a/crates/core_simd/src/lanes_at_most_32.rs
+++ b/crates/core_simd/src/lanes_at_most_32.rs
@@ -1,14 +1,38 @@
 /// Implemented for vectors that are supported by the implementation.
-pub trait LanesAtMost32 {}
+pub trait LanesAtMost32: sealed::Sealed {
+    #[doc(hidden)]
+    type BitMask: Into<u64>;
+}
+
+mod sealed {
+    pub trait Sealed {}
+}
 
 macro_rules! impl_for {
     { $name:ident } => {
-        impl LanesAtMost32 for $name<1> {}
-        impl LanesAtMost32 for $name<2> {}
-        impl LanesAtMost32 for $name<4> {}
-        impl LanesAtMost32 for $name<8> {}
-        impl LanesAtMost32 for $name<16> {}
-        impl LanesAtMost32 for $name<32> {}
+        impl<const LANES: usize> sealed::Sealed for $name<LANES>
+        where
+            $name<LANES>: LanesAtMost32,
+        {}
+
+        impl LanesAtMost32 for $name<1> {
+            type BitMask = u8;
+        }
+        impl LanesAtMost32 for $name<2> {
+            type BitMask = u8;
+        }
+        impl LanesAtMost32 for $name<4> {
+            type BitMask = u8;
+        }
+        impl LanesAtMost32 for $name<8> {
+            type BitMask = u8;
+        }
+        impl LanesAtMost32 for $name<16> {
+            type BitMask = u16;
+        }
+        impl LanesAtMost32 for $name<32> {
+            type BitMask = u32;
+        }
     }
 }
 
diff --git a/crates/core_simd/src/masks/bitmask.rs b/crates/core_simd/src/masks/bitmask.rs
index 32e2ffb86153..bf7c70c5a3ad 100644
--- a/crates/core_simd/src/masks/bitmask.rs
+++ b/crates/core_simd/src/masks/bitmask.rs
@@ -1,13 +1,9 @@
-use crate::LanesAtMost32;
-
 /// A mask where each lane is represented by a single bit.
 #[derive(Copy, Clone, Debug, PartialOrd, PartialEq, Ord, Eq, Hash)]
 #[repr(transparent)]
-pub struct BitMask<const LANES: usize>(u64)
+pub struct BitMask<const LANES: usize>(u64);
 
 impl<const LANES: usize> BitMask<LANES>
-where
-    Self: LanesAtMost32,
 {
     #[inline]
     pub fn splat(value: bool) -> Self {
@@ -25,13 +21,50 @@ where
 
     #[inline]
     pub unsafe fn set_unchecked(&mut self, lane: usize, value: bool) {
-        self.0 ^= ((value ^ self.test(lane)) as u64) << lane
+        self.0 ^= ((value ^ self.test_unchecked(lane)) as u64) << lane
+    }
+
+    #[inline]
+    pub fn to_int<V, T>(self) -> V
+    where
+        V: Default + AsMut<[T; LANES]>,
+        T: From<i8>,
+    {
+        // TODO this should be an intrinsic sign-extension
+        let mut v = V::default();
+        for i in 0..LANES {
+            let lane = unsafe { self.test_unchecked(i) };
+            v.as_mut()[i] = (-(lane as i8)).into();
+        }
+        v
+    }
+
+    #[inline]
+    pub unsafe fn from_int_unchecked<V>(value: V) -> Self
+    where
+        V: crate::LanesAtMost32,
+    {
+        let mask: V::BitMask = crate::intrinsics::simd_bitmask(value);
+        Self(mask.into())
+    }
+
+    #[inline]
+    pub fn to_bitmask(self) -> u64 {
+        self.0
+    }
+
+    #[inline]
+    pub fn any(self) -> bool {
+        self != Self::splat(false)
+    }
+
+    #[inline]
+    pub fn all(self) -> bool {
+        self == Self::splat(true)
     }
 }
 
 impl<const LANES: usize> core::ops::BitAnd for BitMask<LANES>
-where
-    Self: LanesAtMost32,
 {
     type Output = Self;
     #[inline]
@@ -41,8 +74,6 @@ where
 }
 
 impl<const LANES: usize> core::ops::BitAnd<bool> for BitMask<LANES>
-where
-    Self: LanesAtMost32,
 {
     type Output = Self;
     #[inline]
@@ -52,8 +83,6 @@ where
 }
 
 impl<const LANES: usize> core::ops::BitAnd<BitMask<LANES>> for bool
-where
-    BitMask<LANES>: LanesAtMost32,
 {
     type Output = BitMask<LANES>;
     #[inline]
@@ -63,8 +92,6 @@ where
 }
 
 impl<const LANES: usize> core::ops::BitOr for BitMask<LANES>
-where
-    Self: LanesAtMost32,
 {
     type Output = Self;
     #[inline]
@@ -73,31 +100,7 @@ where
     }
 }
 
-impl<const LANES: usize> core::ops::BitOr<bool> for BitMask<LANES>
-where
-    Self: LanesAtMost32,
-{
-    type Output = Self;
-    #[inline]
-    fn bitor(self, rhs: bool) -> Self {
-        self | Self::splat(rhs)
-    }
-}
-
-impl<const LANES: usize> core::ops::BitOr<BitMask<LANES>> for bool
-where
-    BitMask<LANES>: LanesAtMost32,
-{
-    type Output = BitMask<LANES>;
-    #[inline]
-    fn bitor(self, rhs: BitMask<LANES>) -> BitMask<LANES> {
-        BitMask::<LANES>::splat(self) | rhs
-    }
-}
-
 impl<const LANES: usize> core::ops::BitXor for BitMask<LANES>
-where
-    Self: LanesAtMost32,
 {
     type Output = Self;
     #[inline]
@@ -106,42 +109,16 @@ where
     }
 }
 
-impl<const LANES: usize> core::ops::BitXor<bool> for BitMask<LANES>
-where
-    Self: LanesAtMost32,
-{
-    type Output = Self;
-    #[inline]
-    fn bitxor(self, rhs: bool) -> Self::Output {
-        self ^ Self::splat(rhs)
-    }
-}
-
-impl<const LANES: usize> core::ops::BitXor<BitMask<LANES>> for bool
-where
-    BitMask<LANES>: LanesAtMost32,
-{
-    type Output = BitMask<LANES>;
-    #[inline]
-    fn bitxor(self, rhs: BitMask<LANES>) -> Self::Output {
-        BitMask::<LANES>::splat(self) ^ rhs
-    }
-}
-
 impl<const LANES: usize> core::ops::Not for BitMask<LANES>
-where
-    Self: LanesAtMost32,
 {
     type Output = BitMask<LANES>;
     #[inline]
     fn not(self) -> Self::Output {
-        Self(!self.0)
+        Self(!self.0) & Self::splat(true)
     }
 }
 
 impl<const LANES: usize> core::ops::BitAndAssign for BitMask<LANES>
-where
-    Self: LanesAtMost32,
 {
     #[inline]
     fn bitand_assign(&mut self, rhs: Self) {
@@ -149,19 +126,7 @@ where
     }
 }
 
-impl<const LANES: usize> core::ops::BitAndAssign<bool> for BitMask<LANES>
-where
-    Self: LanesAtMost32,
-{
-    #[inline]
-    fn bitand_assign(&mut self, rhs: bool) {
-        *self &= Self::splat(rhs);
-    }
-}
-
 impl<const LANES: usize> core::ops::BitOrAssign for BitMask<LANES>
-where
-    Self: LanesAtMost32,
 {
     #[inline]
     fn bitor_assign(&mut self, rhs: Self) {
@@ -169,19 +134,7 @@ where
     }
 }
 
-impl<const LANES: usize> core::ops::BitOrAssign<bool> for BitMask<LANES>
-where
-    Self: LanesAtMost32,
-{
-    #[inline]
-    fn bitor_assign(&mut self, rhs: bool) {
-        *self |= Self::splat(rhs);
-    }
-}
-
 impl<const LANES: usize> core::ops::BitXorAssign for BitMask<LANES>
-where
-    Self: LanesAtMost32,
 {
     #[inline]
     fn bitxor_assign(&mut self, rhs: Self) {
@@ -189,12 +142,9 @@ where
     }
 }
 
-impl<const LANES: usize> core::ops::BitXorAssign<bool> for BitMask<LANES>
-where
-    Self: LanesAtMost32,
-{
-    #[inline]
-    fn bitxor_assign(&mut self, rhs: bool) {
-        *self ^= Self::splat(rhs);
-    }
-}
+pub type Mask8<const LANES: usize> = BitMask<LANES>;
+pub type Mask16<const LANES: usize> = BitMask<LANES>;
+pub type Mask32<const LANES: usize> = BitMask<LANES>;
+pub type Mask64<const LANES: usize> = BitMask<LANES>;
+pub type Mask128<const LANES: usize> = BitMask<LANES>;
+pub type MaskSize<const LANES: usize> = BitMask<LANES>;
diff --git a/crates/core_simd/src/masks/full_masks.rs b/crates/core_simd/src/masks/full_masks.rs
index 6972a4216b68..2d1ddd6dc305 100644
--- a/crates/core_simd/src/masks/full_masks.rs
+++ b/crates/core_simd/src/masks/full_masks.rs
@@ -46,14 +46,12 @@ macro_rules! define_mask {
             }
 
             #[inline]
-            pub fn test(&self, lane: usize) -> bool {
-                assert!(lane < LANES, "lane index out of range");
+            pub unsafe fn test_unchecked(&self, lane: usize) -> bool {
                 self.0[lane] == -1
             }
 
             #[inline]
-            pub fn set(&mut self, lane: usize, value: bool) {
-                assert!(lane < LANES, "lane index out of range");
+            pub unsafe fn set_unchecked(&mut self, lane: usize, value: bool) {
                 self.0[lane] = if value {
                     -1
                 } else {
@@ -70,6 +68,12 @@ macro_rules! define_mask {
             pub unsafe fn from_int_unchecked(value: crate::$type<LANES>) -> Self {
                 Self(value)
             }
+
+            #[inline]
+            pub fn to_bitmask(self) -> u64 {
+                let mask: <crate::$type<LANES> as crate::LanesAtMost32>::BitMask = unsafe { crate::intrinsics::simd_bitmask(self.0) };
+                mask.into()
+            }
         }
 
         impl<const LANES: usize> core::convert::From<$name<LANES>> for crate::$type<LANES>
@@ -81,53 +85,6 @@ macro_rules! define_mask {
             }
         }
 
-        impl<const LANES: usize> core::fmt::Debug for $name<LANES>
-        where
-            crate::$type<LANES>: crate::LanesAtMost32,
-        {
-            fn fmt(&self, f: &mut core::fmt::Formatter) -> core::fmt::Result {
-                f.debug_list()
-                    .entries((0..LANES).map(|lane| self.test(lane)))
-                    .finish()
-            }
-        }
-
-        impl<const LANES: usize> core::fmt::Binary for $name<LANES>
-        where
-            crate::$type<LANES>: crate::LanesAtMost32,
-        {
-            fn fmt(&self, f: &mut core::fmt::Formatter) -> core::fmt::Result {
-                core::fmt::Binary::fmt(&self.0, f)
-            }
-        }
-
-        impl<const LANES: usize> core::fmt::Octal for $name<LANES>
-        where
-            crate::$type<LANES>: crate::LanesAtMost32,
-        {
-            fn fmt(&self, f: &mut core::fmt::Formatter) -> core::fmt::Result {
-                core::fmt::Octal::fmt(&self.0, f)
-            }
-        }
-
-        impl<const LANES: usize> core::fmt::LowerHex for $name<LANES>
-        where
-            crate::$type<LANES>: crate::LanesAtMost32,
-        {
-            fn fmt(&self, f: &mut core::fmt::Formatter) -> core::fmt::Result {
-                core::fmt::LowerHex::fmt(&self.0, f)
-            }
-        }
-
-        impl<const LANES: usize> core::fmt::UpperHex for $name<LANES>
-        where
-            crate::$type<LANES>: crate::LanesAtMost32,
-        {
-            fn fmt(&self, f: &mut core::fmt::Formatter) -> core::fmt::Result {
-                core::fmt::UpperHex::fmt(&self.0, f)
-            }
-        }
-
         impl<const LANES: usize> core::ops::BitAnd for $name<LANES>
         where
             crate::$type<LANES>: crate::LanesAtMost32,
diff --git a/crates/core_simd/src/masks/mod.rs b/crates/core_simd/src/masks/mod.rs
index fbb934b96424..e5352ef4d1a2 100644
--- a/crates/core_simd/src/masks/mod.rs
+++ b/crates/core_simd/src/masks/mod.rs
@@ -8,6 +8,12 @@ mod mask_impl;
 
 use crate::{LanesAtMost32, SimdI16, SimdI32, SimdI64, SimdI8, SimdIsize};
 
+/// Converts masks to bitmasks, with one bit set for each lane.
+pub trait ToBitMask {
+    /// Converts this mask to a bitmask.
+    fn to_bitmask(self) -> u64;
+}
+
 macro_rules! define_opaque_mask {
     {
         $(#[$attr:meta])*
@@ -61,13 +67,53 @@ macro_rules! define_opaque_mask {
                 Self(<$inner_ty>::from_int_unchecked(value))
             }
 
+            /// Converts a vector of integers to a mask, where 0 represents `false` and -1
+            /// represents `true`.
+            ///
+            /// # Panics
+            /// Panics if any lane is not 0 or -1.
+            #[inline]
+            pub fn from_int(value: $bits_ty<LANES>) -> Self {
+                assert!(
+                    (value.lanes_eq($bits_ty::splat(0)) | value.lanes_eq($bits_ty::splat(-1))).all(),
+                    "all values must be either 0 or -1",
+                );
+                unsafe { Self::from_int_unchecked(value) }
+            }
+
+            /// Converts the mask to a vector of integers, where 0 represents `false` and -1
+            /// represents `true`.
+            #[inline]
+            pub fn to_int(self) -> $bits_ty<LANES> {
+                self.0.to_int()
+            }
+
+            /// Tests the value of the specified lane.
+            ///
+            /// # Safety
+            /// `lane` must be less than `LANES`.
+            #[inline]
+            pub unsafe fn test_unchecked(&self, lane: usize) -> bool {
+                self.0.test_unchecked(lane)
+            }
+
             /// Tests the value of the specified lane.
             ///
             /// # Panics
             /// Panics if `lane` is greater than or equal to the number of lanes in the vector.
             #[inline]
             pub fn test(&self, lane: usize) -> bool {
-                self.0.test(lane)
+                assert!(lane < LANES, "lane index out of range");
+                unsafe { self.test_unchecked(lane) }
+            }
+
+            /// Sets the value of the specified lane.
+            ///
+            /// # Safety
+            /// `lane` must be less than `LANES`.
+            #[inline]
+            pub unsafe fn set_unchecked(&mut self, lane: usize, value: bool) {
+                self.0.set_unchecked(lane, value);
             }
 
             /// Sets the value of the specified lane.
@@ -76,7 +122,44 @@ macro_rules! define_opaque_mask {
             /// Panics if `lane` is greater than or equal to the number of lanes in the vector.
             #[inline]
             pub fn set(&mut self, lane: usize, value: bool) {
-                self.0.set(lane, value);
+                assert!(lane < LANES, "lane index out of range");
+                unsafe { self.set_unchecked(lane, value); }
+            }
+        }
+
+        impl ToBitMask for $name<1> {
+            fn to_bitmask(self) -> u64 {
+                self.0.to_bitmask()
+            }
+        }
+
+        impl ToBitMask for $name<2> {
+            fn to_bitmask(self) -> u64 {
+                self.0.to_bitmask()
+            }
+        }
+
+        impl ToBitMask for $name<4> {
+            fn to_bitmask(self) -> u64 {
+                self.0.to_bitmask()
+            }
+        }
+
+        impl ToBitMask for $name<8> {
+            fn to_bitmask(self) -> u64 {
+                self.0.to_bitmask()
+            }
+        }
+
+        impl ToBitMask for $name<16> {
+            fn to_bitmask(self) -> u64 {
+                self.0.to_bitmask()
+            }
+        }
+
+        impl ToBitMask for $name<32> {
+            fn to_bitmask(self) -> u64 {
+                self.0.to_bitmask()
             }
         }
 
@@ -147,10 +230,12 @@ macro_rules! define_opaque_mask {
 
         impl<const LANES: usize> core::fmt::Debug for $name<LANES>
         where
-            $bits_ty<LANES>: LanesAtMost32,
+            $bits_ty<LANES>: crate::LanesAtMost32,
         {
             fn fmt(&self, f: &mut core::fmt::Formatter) -> core::fmt::Result {
-                core::fmt::Debug::fmt(&self.0, f)
+                f.debug_list()
+                    .entries((0..LANES).map(|lane| self.test(lane)))
+                    .finish()
             }
         }
 
diff --git a/crates/core_simd/tests/masks.rs b/crates/core_simd/tests/masks.rs
index 6c3993e39a93..be83f4c2ec77 100644
--- a/crates/core_simd/tests/masks.rs
+++ b/crates/core_simd/tests/masks.rs
@@ -56,6 +56,23 @@ macro_rules! test_mask_api {
                 v.set(2, true);
                 assert!(!v.all());
             }
+
+            #[test]
+            fn roundtrip_int_conversion() {
+                let values = [true, false, false, true, false, false, true, false];
+                let mask = core_simd::$name::<8>::from_array(values);
+                let int = mask.to_int();
+                assert_eq!(int.to_array(), [-1, 0, 0, -1, 0, 0, -1, 0]);
+                assert_eq!(core_simd::$name::<8>::from_int(int), mask);
+            }
+
+            #[test]
+            fn to_bitmask() {
+                use core_simd::ToBitMask;
+                let values = [true, false, false, true, false, false, true, false];
+                let mask = core_simd::$name::<8>::from_array(values);
+                assert_eq!(mask.to_bitmask(), 0b01001001);
+            }
         }
     }
 }

From 98dad135268e0da590a162d24da6f7e2d8781648 Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Thu, 29 Apr 2021 02:41:09 +0000
Subject: [PATCH 146/249] Make implementation more scalable by using a helper
 trait to determine bitmask size.  Improve bitmask to int conversion.

---
 crates/core_simd/src/comparisons.rs      |   1 +
 crates/core_simd/src/intrinsics.rs       |   3 +
 crates/core_simd/src/masks/bitmask.rs    | 191 +++++++++++++----------
 crates/core_simd/src/masks/full_masks.rs | 125 ++++++++-------
 crates/core_simd/src/masks/mod.rs        | 146 ++++++++++-------
 crates/core_simd/src/reduction.rs        |   5 +-
 crates/core_simd/src/vector/float.rs     |   1 +
 crates/core_simd/src/vector/int.rs       |   1 +
 crates/core_simd/src/vector/uint.rs      |   1 -
 crates/core_simd/tests/masks.rs          |  10 +-
 crates/test_helpers/src/array.rs         |   5 +-
 crates/test_helpers/src/lib.rs           |  10 ++
 12 files changed, 293 insertions(+), 206 deletions(-)

diff --git a/crates/core_simd/src/comparisons.rs b/crates/core_simd/src/comparisons.rs
index f3a1954fda22..e8d11406c097 100644
--- a/crates/core_simd/src/comparisons.rs
+++ b/crates/core_simd/src/comparisons.rs
@@ -7,6 +7,7 @@ macro_rules! implement_mask_ops {
             where
                 crate::$vector<LANES>: LanesAtMost32,
                 crate::$inner_ty<LANES>: LanesAtMost32,
+                crate::$mask<LANES>: crate::Mask,
             {
                 /// Test if each lane is equal to the corresponding lane in `other`.
                 #[inline]
diff --git a/crates/core_simd/src/intrinsics.rs b/crates/core_simd/src/intrinsics.rs
index 1812a9c624dc..8cbb0cbccf79 100644
--- a/crates/core_simd/src/intrinsics.rs
+++ b/crates/core_simd/src/intrinsics.rs
@@ -79,6 +79,9 @@ extern "platform-intrinsic" {
 
     // truncate integer vector to bitmask
     pub(crate) fn simd_bitmask<T, U>(x: T) -> U;
+
+    // select
+    pub(crate) fn simd_select_bitmask<T, U>(m: T, a: U, b: U) -> U;
 }
 
 #[cfg(feature = "std")]
diff --git a/crates/core_simd/src/masks/bitmask.rs b/crates/core_simd/src/masks/bitmask.rs
index bf7c70c5a3ad..6bcb08cf9dbb 100644
--- a/crates/core_simd/src/masks/bitmask.rs
+++ b/crates/core_simd/src/masks/bitmask.rs
@@ -1,42 +1,80 @@
-/// A mask where each lane is represented by a single bit.
-#[derive(Copy, Clone, Debug, PartialOrd, PartialEq, Ord, Eq, Hash)]
-#[repr(transparent)]
-pub struct BitMask<const LANES: usize>(u64);
+use crate::Mask;
+use core::marker::PhantomData;
 
-impl<const LANES: usize> BitMask<LANES>
-{
+/// Helper trait for limiting int conversion types
+pub trait ConvertToInt {}
+impl<const LANES: usize> ConvertToInt for crate::SimdI8<LANES> where Self: crate::LanesAtMost32 {}
+impl<const LANES: usize> ConvertToInt for crate::SimdI16<LANES> where Self: crate::LanesAtMost32 {}
+impl<const LANES: usize> ConvertToInt for crate::SimdI32<LANES> where Self: crate::LanesAtMost32 {}
+impl<const LANES: usize> ConvertToInt for crate::SimdI64<LANES> where Self: crate::LanesAtMost32 {}
+impl<const LANES: usize> ConvertToInt for crate::SimdIsize<LANES> where Self: crate::LanesAtMost32 {}
+
+/// A mask where each lane is represented by a single bit.
+#[repr(transparent)]
+pub struct BitMask<T: Mask, const LANES: usize>(T::BitMask, PhantomData<[(); LANES]>);
+
+impl<T: Mask, const LANES: usize> Copy for BitMask<T, LANES> {}
+
+impl<T: Mask, const LANES: usize> Clone for BitMask<T, LANES> {
+    fn clone(&self) -> Self {
+        *self
+    }
+}
+
+impl<T: Mask, const LANES: usize> PartialEq for BitMask<T, LANES> {
+    fn eq(&self, other: &Self) -> bool {
+        self.0.as_ref() == other.0.as_ref()
+    }
+}
+
+impl<T: Mask, const LANES: usize> PartialOrd for BitMask<T, LANES> {
+    fn partial_cmp(&self, other: &Self) -> Option<core::cmp::Ordering> {
+        self.0.as_ref().partial_cmp(other.0.as_ref())
+    }
+}
+
+impl<T: Mask, const LANES: usize> Eq for BitMask<T, LANES> {}
+
+impl<T: Mask, const LANES: usize> Ord for BitMask<T, LANES> {
+    fn cmp(&self, other: &Self) -> core::cmp::Ordering {
+        self.0.as_ref().cmp(other.0.as_ref())
+    }
+}
+
+impl<T: Mask, const LANES: usize> BitMask<T, LANES> {
     #[inline]
     pub fn splat(value: bool) -> Self {
+        let mut mask = T::BitMask::default();
         if value {
-            Self(u64::MAX >> (64 - LANES))
+            mask.as_mut().fill(u8::MAX)
         } else {
-            Self(u64::MIN)
+            mask.as_mut().fill(u8::MIN)
         }
+        if LANES % 8 > 0 {
+            *mask.as_mut().last_mut().unwrap() &= u8::MAX >> (8 - LANES % 8);
+        }
+        Self(mask, PhantomData)
     }
 
     #[inline]
     pub unsafe fn test_unchecked(&self, lane: usize) -> bool {
-        (self.0 >> lane) & 0x1 > 0
+        (self.0.as_ref()[lane / 8] >> lane % 8) & 0x1 > 0
     }
 
     #[inline]
     pub unsafe fn set_unchecked(&mut self, lane: usize, value: bool) {
-        self.0 ^= ((value ^ self.test_unchecked(lane)) as u64) << lane
+        self.0.as_mut()[lane / 8] ^= ((value ^ self.test_unchecked(lane)) as u8) << (lane % 8)
     }
 
     #[inline]
-    pub fn to_int<V, T>(self) -> V
+    pub fn to_int<V>(self) -> V
     where
-        V: Default + AsMut<[T; LANES]>,
-        T: From<i8>,
+        V: ConvertToInt + Default + core::ops::Not<Output = V>,
     {
-        // TODO this should be an intrinsic sign-extension
-        let mut v = V::default();
-        for i in 0..LANES {
-            let lane = unsafe { self.test_unchecked(i) };
-            v.as_mut()[i] = (-(lane as i8)).into();
+        unsafe {
+            let mask: T::IntBitMask = core::mem::transmute_copy(&self);
+            crate::intrinsics::simd_select_bitmask(mask, !V::default(), V::default())
         }
-        v
     }
 
     #[inline]
@@ -44,13 +82,22 @@ impl<const LANES: usize> BitMask<LANES>
     where
         V: crate::LanesAtMost32,
     {
-        let mask: V::BitMask = crate::intrinsics::simd_bitmask(value);
-        Self(mask.into())
+        // TODO remove the transmute when rustc is more flexible
+        assert_eq!(
+            core::mem::size_of::<T::IntBitMask>(),
+            core::mem::size_of::<T::BitMask>()
+        );
+        let mask: T::IntBitMask = crate::intrinsics::simd_bitmask(value);
+        Self(core::mem::transmute_copy(&mask), PhantomData)
     }
 
     #[inline]
-    pub fn to_bitmask(self) -> u64 {
-        self.0
+    pub fn to_bitmask<U: Mask>(self) -> U::BitMask {
+        assert_eq!(
+            core::mem::size_of::<T::BitMask>(),
+            core::mem::size_of::<U::BitMask>()
+        );
+        unsafe { core::mem::transmute_copy(&self.0) }
     }
 
     #[inline]
@@ -64,87 +111,61 @@ impl<const LANES: usize> BitMask<LANES>
     }
 }
 
-impl<const LANES: usize> core::ops::BitAnd for BitMask<LANES>
+impl<T: Mask, const LANES: usize> core::ops::BitAnd for BitMask<T, LANES>
+where
+    T::BitMask: Default + AsRef<[u8]> + AsMut<[u8]>,
 {
     type Output = Self;
     #[inline]
-    fn bitand(self, rhs: Self) -> Self {
-        Self(self.0 & rhs.0)
+    fn bitand(mut self, rhs: Self) -> Self {
+        for (l, r) in self.0.as_mut().iter_mut().zip(rhs.0.as_ref().iter()) {
+            *l &= r;
+        }
+        self
     }
 }
 
-impl<const LANES: usize> core::ops::BitAnd<bool> for BitMask<LANES>
+impl<T: Mask, const LANES: usize> core::ops::BitOr for BitMask<T, LANES>
+where
+    T::BitMask: Default + AsRef<[u8]> + AsMut<[u8]>,
 {
     type Output = Self;
     #[inline]
-    fn bitand(self, rhs: bool) -> Self {
-        self & Self::splat(rhs)
+    fn bitor(mut self, rhs: Self) -> Self {
+        for (l, r) in self.0.as_mut().iter_mut().zip(rhs.0.as_ref().iter()) {
+            *l |= r;
+        }
+        self
     }
 }
 
-impl<const LANES: usize> core::ops::BitAnd<BitMask<LANES>> for bool
-{
-    type Output = BitMask<LANES>;
-    #[inline]
-    fn bitand(self, rhs: BitMask<LANES>) -> BitMask<LANES> {
-        BitMask::<LANES>::splat(self) & rhs
-    }
-}
-
-impl<const LANES: usize> core::ops::BitOr for BitMask<LANES>
-{
+impl<T: Mask, const LANES: usize> core::ops::BitXor for BitMask<T, LANES> {
     type Output = Self;
     #[inline]
-    fn bitor(self, rhs: Self) -> Self {
-        Self(self.0 | rhs.0)
+    fn bitxor(mut self, rhs: Self) -> Self::Output {
+        for (l, r) in self.0.as_mut().iter_mut().zip(rhs.0.as_ref().iter()) {
+            *l ^= r;
+        }
+        self
     }
 }
 
-impl<const LANES: usize> core::ops::BitXor for BitMask<LANES>
-{
+impl<T: Mask, const LANES: usize> core::ops::Not for BitMask<T, LANES> {
     type Output = Self;
     #[inline]
-    fn bitxor(self, rhs: Self) -> Self::Output {
-        Self(self.0 ^ rhs.0)
+    fn not(mut self) -> Self::Output {
+        for x in self.0.as_mut() {
+            *x = !*x;
+        }
+        if LANES % 8 > 0 {
+            *self.0.as_mut().last_mut().unwrap() &= u8::MAX >> (8 - LANES % 8);
+        }
+        self
     }
 }
 
-impl<const LANES: usize> core::ops::Not for BitMask<LANES>
-{
-    type Output = BitMask<LANES>;
-    #[inline]
-    fn not(self) -> Self::Output {
-        Self(!self.0) & Self::splat(true)
-    }
-}
-
-impl<const LANES: usize> core::ops::BitAndAssign for BitMask<LANES>
-{
-    #[inline]
-    fn bitand_assign(&mut self, rhs: Self) {
-        self.0 &= rhs.0;
-    }
-}
-
-impl<const LANES: usize> core::ops::BitOrAssign for BitMask<LANES>
-{
-    #[inline]
-    fn bitor_assign(&mut self, rhs: Self) {
-        self.0 |= rhs.0;
-    }
-}
-
-impl<const LANES: usize> core::ops::BitXorAssign for BitMask<LANES>
-{
-    #[inline]
-    fn bitxor_assign(&mut self, rhs: Self) {
-        self.0 ^= rhs.0;
-    }
-}
-
-pub type Mask8<const LANES: usize> = BitMask<LANES>;
-pub type Mask16<const LANES: usize> = BitMask<LANES>;
-pub type Mask32<const LANES: usize> = BitMask<LANES>;
-pub type Mask64<const LANES: usize> = BitMask<LANES>;
-pub type Mask128<const LANES: usize> = BitMask<LANES>;
-pub type MaskSize<const LANES: usize> = BitMask<LANES>;
+pub type Mask8<T, const LANES: usize> = BitMask<T, LANES>;
+pub type Mask16<T, const LANES: usize> = BitMask<T, LANES>;
+pub type Mask32<T, const LANES: usize> = BitMask<T, LANES>;
+pub type Mask64<T, const LANES: usize> = BitMask<T, LANES>;
+pub type MaskSize<T, const LANES: usize> = BitMask<T, LANES>;
diff --git a/crates/core_simd/src/masks/full_masks.rs b/crates/core_simd/src/masks/full_masks.rs
index 2d1ddd6dc305..bd52a25551e2 100644
--- a/crates/core_simd/src/masks/full_masks.rs
+++ b/crates/core_simd/src/masks/full_masks.rs
@@ -1,5 +1,8 @@
 //! Masks that take up full SIMD vector registers.
 
+use crate::Mask;
+use core::marker::PhantomData;
+
 macro_rules! define_mask {
     {
         $(#[$attr:meta])*
@@ -8,20 +11,19 @@ macro_rules! define_mask {
         );
     } => {
         $(#[$attr])*
-        #[derive(Default, PartialEq, PartialOrd, Eq, Ord, Hash)]
         #[repr(transparent)]
-        pub struct $name<const $lanes: usize>(crate::$type<$lanes2>)
+        pub struct $name<T: Mask, const $lanes: usize>(crate::$type<$lanes2>, PhantomData<T>)
         where
             crate::$type<LANES>: crate::LanesAtMost32;
 
         impl_full_mask_reductions! { $name, $type }
 
-        impl<const LANES: usize> Copy for $name<LANES>
+        impl<T: Mask, const LANES: usize> Copy for $name<T, LANES>
         where
             crate::$type<LANES>: crate::LanesAtMost32,
         {}
 
-        impl<const LANES: usize> Clone for $name<LANES>
+        impl<T: Mask, const LANES: usize> Clone for $name<T, LANES>
         where
             crate::$type<LANES>: crate::LanesAtMost32,
         {
@@ -31,18 +33,53 @@ macro_rules! define_mask {
             }
         }
 
-        impl<const LANES: usize> $name<LANES>
+        impl<T: Mask, const LANES: usize> PartialEq for $name<T, LANES>
+        where
+            crate::$type<LANES>: crate::LanesAtMost32,
+        {
+            fn eq(&self, other: &Self) -> bool {
+                self.0 == other.0
+            }
+        }
+
+        impl<T: Mask, const LANES: usize> PartialOrd for $name<T, LANES>
+        where
+            crate::$type<LANES>: crate::LanesAtMost32,
+        {
+            fn partial_cmp(&self, other: &Self) -> Option<core::cmp::Ordering> {
+                self.0.partial_cmp(&other.0)
+            }
+        }
+
+        impl<T: Mask, const LANES: usize> Eq for $name<T, LANES>
+        where
+            crate::$type<LANES>: crate::LanesAtMost32,
+        {}
+
+        impl<T: Mask, const LANES: usize> Ord for $name<T, LANES>
+        where
+            crate::$type<LANES>: crate::LanesAtMost32,
+        {
+            fn cmp(&self, other: &Self) -> core::cmp::Ordering {
+                self.0.cmp(&other.0)
+            }
+        }
+
+        impl<T: Mask, const LANES: usize> $name<T, LANES>
         where
             crate::$type<LANES>: crate::LanesAtMost32,
         {
             pub fn splat(value: bool) -> Self {
-                Self(<crate::$type<LANES>>::splat(
-                    if value {
-                        -1
-                    } else {
-                        0
-                    }
-                ))
+                Self(
+                    <crate::$type<LANES>>::splat(
+                        if value {
+                            -1
+                        } else {
+                            0
+                        }
+                    ),
+                    PhantomData,
+                )
             }
 
             #[inline]
@@ -66,96 +103,70 @@ macro_rules! define_mask {
 
             #[inline]
             pub unsafe fn from_int_unchecked(value: crate::$type<LANES>) -> Self {
-                Self(value)
+                Self(value, PhantomData)
             }
 
             #[inline]
-            pub fn to_bitmask(self) -> u64 {
-                let mask: <crate::$type<LANES> as crate::LanesAtMost32>::BitMask = unsafe { crate::intrinsics::simd_bitmask(self.0) };
-                mask.into()
+            pub fn to_bitmask<U: crate::Mask>(self) -> U::BitMask {
+                unsafe {
+                    // TODO remove the transmute when rustc is more flexible
+                    assert_eq!(core::mem::size_of::<U::IntBitMask>(), core::mem::size_of::<U::BitMask>());
+                    let mask: U::IntBitMask = crate::intrinsics::simd_bitmask(self.0);
+                    core::mem::transmute_copy(&mask)
+                }
             }
         }
 
-        impl<const LANES: usize> core::convert::From<$name<LANES>> for crate::$type<LANES>
+        impl<T: Mask, const LANES: usize> core::convert::From<$name<T, LANES>> for crate::$type<LANES>
         where
             crate::$type<LANES>: crate::LanesAtMost32,
         {
-            fn from(value: $name<LANES>) -> Self {
+            fn from(value: $name<T, LANES>) -> Self {
                 value.0
             }
         }
 
-        impl<const LANES: usize> core::ops::BitAnd for $name<LANES>
+        impl<T: Mask, const LANES: usize> core::ops::BitAnd for $name<T, LANES>
         where
             crate::$type<LANES>: crate::LanesAtMost32,
         {
             type Output = Self;
             #[inline]
             fn bitand(self, rhs: Self) -> Self {
-                Self(self.0 & rhs.0)
+                Self(self.0 & rhs.0, PhantomData)
             }
         }
 
-        impl<const LANES: usize> core::ops::BitOr for $name<LANES>
+        impl<T: Mask, const LANES: usize> core::ops::BitOr for $name<T, LANES>
         where
             crate::$type<LANES>: crate::LanesAtMost32,
         {
             type Output = Self;
             #[inline]
             fn bitor(self, rhs: Self) -> Self {
-                Self(self.0 | rhs.0)
+                Self(self.0 | rhs.0, PhantomData)
             }
         }
 
-        impl<const LANES: usize> core::ops::BitXor for $name<LANES>
+        impl<T: Mask, const LANES: usize> core::ops::BitXor for $name<T, LANES>
         where
             crate::$type<LANES>: crate::LanesAtMost32,
         {
             type Output = Self;
             #[inline]
             fn bitxor(self, rhs: Self) -> Self::Output {
-                Self(self.0 ^ rhs.0)
+                Self(self.0 ^ rhs.0, PhantomData)
             }
         }
 
-        impl<const LANES: usize> core::ops::Not for $name<LANES>
+        impl<T: Mask, const LANES: usize> core::ops::Not for $name<T, LANES>
         where
             crate::$type<LANES>: crate::LanesAtMost32,
         {
-            type Output = $name<LANES>;
+            type Output = Self;
             #[inline]
             fn not(self) -> Self::Output {
-                Self(!self.0)
-            }
-        }
-
-        impl<const LANES: usize> core::ops::BitAndAssign for $name<LANES>
-        where
-            crate::$type<LANES>: crate::LanesAtMost32,
-        {
-            #[inline]
-            fn bitand_assign(&mut self, rhs: Self) {
-                self.0 &= rhs.0;
-            }
-        }
-
-        impl<const LANES: usize> core::ops::BitOrAssign for $name<LANES>
-        where
-            crate::$type<LANES>: crate::LanesAtMost32,
-        {
-            #[inline]
-            fn bitor_assign(&mut self, rhs: Self) {
-                self.0 |= rhs.0;
-            }
-        }
-
-        impl<const LANES: usize> core::ops::BitXorAssign for $name<LANES>
-        where
-            crate::$type<LANES>: crate::LanesAtMost32,
-        {
-            #[inline]
-            fn bitxor_assign(&mut self, rhs: Self) {
-                self.0 ^= rhs.0;
+                Self(!self.0, PhantomData)
             }
         }
     }
diff --git a/crates/core_simd/src/masks/mod.rs b/crates/core_simd/src/masks/mod.rs
index e5352ef4d1a2..deaf2be5dca4 100644
--- a/crates/core_simd/src/masks/mod.rs
+++ b/crates/core_simd/src/masks/mod.rs
@@ -2,16 +2,30 @@
 //! Types representing
 #![allow(non_camel_case_types)]
 
-#[cfg_attr(not(all(target_arch = "x86_64", target_feature = "avx512f")), path = "full_masks.rs")]
-#[cfg_attr(all(target_arch = "x86_64", target_feature = "avx512f"), path = "bitmask.rs")]
+#[cfg_attr(
+    not(all(target_arch = "x86_64", target_feature = "avx512f")),
+    path = "full_masks.rs"
+)]
+#[cfg_attr(
+    all(target_arch = "x86_64", target_feature = "avx512f"),
+    path = "bitmask.rs"
+)]
 mod mask_impl;
 
 use crate::{LanesAtMost32, SimdI16, SimdI32, SimdI64, SimdI8, SimdIsize};
 
-/// Converts masks to bitmasks, with one bit set for each lane.
-pub trait ToBitMask {
-    /// Converts this mask to a bitmask.
-    fn to_bitmask(self) -> u64;
+mod sealed {
+    pub trait Sealed {}
+}
+
+/// Helper trait for mask types.
+pub trait Mask: sealed::Sealed {
+    /// The bitmask representation of a mask.
+    type BitMask: Copy + Default + AsRef<[u8]> + AsMut<[u8]>;
+
+    // TODO remove this when rustc intrinsics are more flexible
+    #[doc(hidden)]
+    type IntBitMask;
 }
 
 macro_rules! define_opaque_mask {
@@ -22,13 +36,47 @@ macro_rules! define_opaque_mask {
     } => {
         $(#[$attr])*
         #[allow(non_camel_case_types)]
-        pub struct $name<const LANES: usize>($inner_ty) where $bits_ty<LANES>: LanesAtMost32;
+        pub struct $name<const LANES: usize>($inner_ty)
+        where
+            $bits_ty<LANES>: LanesAtMost32,
+            Self: Mask;
+
+        impl<const LANES: usize> sealed::Sealed for $name<LANES>
+        where
+            $bits_ty<LANES>: LanesAtMost32,
+            Self: Mask,
+        {}
+        impl Mask for $name<1> {
+            type BitMask = [u8; 1];
+            type IntBitMask = u8;
+        }
+        impl Mask for $name<2> {
+            type BitMask = [u8; 1];
+            type IntBitMask = u8;
+        }
+        impl Mask for $name<4> {
+            type BitMask = [u8; 1];
+            type IntBitMask = u8;
+        }
+        impl Mask for $name<8> {
+            type BitMask = [u8; 1];
+            type IntBitMask = u8;
+        }
+        impl Mask for $name<16> {
+            type BitMask = [u8; 2];
+            type IntBitMask = u16;
+        }
+        impl Mask for $name<32> {
+            type BitMask = [u8; 4];
+            type IntBitMask = u32;
+        }
 
         impl_opaque_mask_reductions! { $name, $bits_ty }
 
         impl<const LANES: usize> $name<LANES>
         where
-            $bits_ty<LANES>: LanesAtMost32
+            $bits_ty<LANES>: LanesAtMost32,
+            Self: Mask,
         {
             /// Construct a mask by setting all lanes to the given value.
             pub fn splat(value: bool) -> Self {
@@ -125,48 +173,18 @@ macro_rules! define_opaque_mask {
                 assert!(lane < LANES, "lane index out of range");
                 unsafe { self.set_unchecked(lane, value); }
             }
-        }
 
-        impl ToBitMask for $name<1> {
-            fn to_bitmask(self) -> u64 {
-                self.0.to_bitmask()
-            }
-        }
-
-        impl ToBitMask for $name<2> {
-            fn to_bitmask(self) -> u64 {
-                self.0.to_bitmask()
-            }
-        }
-
-        impl ToBitMask for $name<4> {
-            fn to_bitmask(self) -> u64 {
-                self.0.to_bitmask()
-            }
-        }
-
-        impl ToBitMask for $name<8> {
-            fn to_bitmask(self) -> u64 {
-                self.0.to_bitmask()
-            }
-        }
-
-        impl ToBitMask for $name<16> {
-            fn to_bitmask(self) -> u64 {
-                self.0.to_bitmask()
-            }
-        }
-
-        impl ToBitMask for $name<32> {
-            fn to_bitmask(self) -> u64 {
-                self.0.to_bitmask()
+            /// Convert this mask to a bitmask, with one bit set per lane.
+            pub fn to_bitmask(self) -> <Self as Mask>::BitMask {
+                self.0.to_bitmask::<Self>()
             }
         }
 
         // vector/array conversion
         impl<const LANES: usize> From<[bool; LANES]> for $name<LANES>
         where
-            $bits_ty<LANES>: crate::LanesAtMost32
+            $bits_ty<LANES>: crate::LanesAtMost32,
+            Self: Mask,
         {
             fn from(array: [bool; LANES]) -> Self {
                 Self::from_array(array)
@@ -175,7 +193,8 @@ macro_rules! define_opaque_mask {
 
         impl <const LANES: usize> From<$name<LANES>> for [bool; LANES]
         where
-            $bits_ty<LANES>: crate::LanesAtMost32
+            $bits_ty<LANES>: crate::LanesAtMost32,
+            $name<LANES>: Mask,
         {
             fn from(vector: $name<LANES>) -> Self {
                 vector.to_array()
@@ -184,13 +203,14 @@ macro_rules! define_opaque_mask {
 
         impl<const LANES: usize> Copy for $name<LANES>
         where
-            $inner_ty: Copy,
             $bits_ty<LANES>: LanesAtMost32,
+            Self: Mask,
         {}
 
         impl<const LANES: usize> Clone for $name<LANES>
         where
             $bits_ty<LANES>: LanesAtMost32,
+            Self: Mask,
         {
             #[inline]
             fn clone(&self) -> Self {
@@ -201,6 +221,7 @@ macro_rules! define_opaque_mask {
         impl<const LANES: usize> Default for $name<LANES>
         where
             $bits_ty<LANES>: LanesAtMost32,
+            Self: Mask,
         {
             #[inline]
             fn default() -> Self {
@@ -211,6 +232,7 @@ macro_rules! define_opaque_mask {
         impl<const LANES: usize> PartialEq for $name<LANES>
         where
             $bits_ty<LANES>: LanesAtMost32,
+            Self: Mask,
         {
             #[inline]
             fn eq(&self, other: &Self) -> bool {
@@ -221,6 +243,7 @@ macro_rules! define_opaque_mask {
         impl<const LANES: usize> PartialOrd for $name<LANES>
         where
             $bits_ty<LANES>: LanesAtMost32,
+            Self: Mask,
         {
             #[inline]
             fn partial_cmp(&self, other: &Self) -> Option<core::cmp::Ordering> {
@@ -231,6 +254,7 @@ macro_rules! define_opaque_mask {
         impl<const LANES: usize> core::fmt::Debug for $name<LANES>
         where
             $bits_ty<LANES>: crate::LanesAtMost32,
+            Self: Mask,
         {
             fn fmt(&self, f: &mut core::fmt::Formatter) -> core::fmt::Result {
                 f.debug_list()
@@ -242,6 +266,7 @@ macro_rules! define_opaque_mask {
         impl<const LANES: usize> core::ops::BitAnd for $name<LANES>
         where
             $bits_ty<LANES>: LanesAtMost32,
+            Self: Mask,
         {
             type Output = Self;
             #[inline]
@@ -253,6 +278,7 @@ macro_rules! define_opaque_mask {
         impl<const LANES: usize> core::ops::BitAnd<bool> for $name<LANES>
         where
             $bits_ty<LANES>: LanesAtMost32,
+            Self: Mask,
         {
             type Output = Self;
             #[inline]
@@ -264,6 +290,7 @@ macro_rules! define_opaque_mask {
         impl<const LANES: usize> core::ops::BitAnd<$name<LANES>> for bool
         where
             $bits_ty<LANES>: LanesAtMost32,
+            $name<LANES>: Mask,
         {
             type Output = $name<LANES>;
             #[inline]
@@ -275,6 +302,7 @@ macro_rules! define_opaque_mask {
         impl<const LANES: usize> core::ops::BitOr for $name<LANES>
         where
             $bits_ty<LANES>: LanesAtMost32,
+            Self: Mask,
         {
             type Output = Self;
             #[inline]
@@ -286,6 +314,7 @@ macro_rules! define_opaque_mask {
         impl<const LANES: usize> core::ops::BitOr<bool> for $name<LANES>
         where
             $bits_ty<LANES>: LanesAtMost32,
+            Self: Mask,
         {
             type Output = Self;
             #[inline]
@@ -297,6 +326,7 @@ macro_rules! define_opaque_mask {
         impl<const LANES: usize> core::ops::BitOr<$name<LANES>> for bool
         where
             $bits_ty<LANES>: LanesAtMost32,
+            $name<LANES>: Mask,
         {
             type Output = $name<LANES>;
             #[inline]
@@ -308,6 +338,7 @@ macro_rules! define_opaque_mask {
         impl<const LANES: usize> core::ops::BitXor for $name<LANES>
         where
             $bits_ty<LANES>: LanesAtMost32,
+            Self: Mask,
         {
             type Output = Self;
             #[inline]
@@ -319,6 +350,7 @@ macro_rules! define_opaque_mask {
         impl<const LANES: usize> core::ops::BitXor<bool> for $name<LANES>
         where
             $bits_ty<LANES>: LanesAtMost32,
+            Self: Mask,
         {
             type Output = Self;
             #[inline]
@@ -330,6 +362,7 @@ macro_rules! define_opaque_mask {
         impl<const LANES: usize> core::ops::BitXor<$name<LANES>> for bool
         where
             $bits_ty<LANES>: LanesAtMost32,
+            $name<LANES>: Mask,
         {
             type Output = $name<LANES>;
             #[inline]
@@ -341,6 +374,7 @@ macro_rules! define_opaque_mask {
         impl<const LANES: usize> core::ops::Not for $name<LANES>
         where
             $bits_ty<LANES>: LanesAtMost32,
+            Self: Mask,
         {
             type Output = $name<LANES>;
             #[inline]
@@ -352,16 +386,18 @@ macro_rules! define_opaque_mask {
         impl<const LANES: usize> core::ops::BitAndAssign for $name<LANES>
         where
             $bits_ty<LANES>: LanesAtMost32,
+            Self: Mask,
         {
             #[inline]
             fn bitand_assign(&mut self, rhs: Self) {
-                self.0 &= rhs.0;
+                self.0 = self.0 & rhs.0;
             }
         }
 
         impl<const LANES: usize> core::ops::BitAndAssign<bool> for $name<LANES>
         where
             $bits_ty<LANES>: LanesAtMost32,
+            Self: Mask,
         {
             #[inline]
             fn bitand_assign(&mut self, rhs: bool) {
@@ -372,16 +408,18 @@ macro_rules! define_opaque_mask {
         impl<const LANES: usize> core::ops::BitOrAssign for $name<LANES>
         where
             $bits_ty<LANES>: LanesAtMost32,
+            Self: Mask,
         {
             #[inline]
             fn bitor_assign(&mut self, rhs: Self) {
-                self.0 |= rhs.0;
+                self.0 = self.0 | rhs.0;
             }
         }
 
         impl<const LANES: usize> core::ops::BitOrAssign<bool> for $name<LANES>
         where
             $bits_ty<LANES>: LanesAtMost32,
+            Self: Mask,
         {
             #[inline]
             fn bitor_assign(&mut self, rhs: bool) {
@@ -392,16 +430,18 @@ macro_rules! define_opaque_mask {
         impl<const LANES: usize> core::ops::BitXorAssign for $name<LANES>
         where
             $bits_ty<LANES>: LanesAtMost32,
+            Self: Mask,
         {
             #[inline]
             fn bitxor_assign(&mut self, rhs: Self) {
-                self.0 ^= rhs.0;
+                self.0 = self.0 ^ rhs.0;
             }
         }
 
         impl<const LANES: usize> core::ops::BitXorAssign<bool> for $name<LANES>
         where
             $bits_ty<LANES>: LanesAtMost32,
+            Self: Mask,
         {
             #[inline]
             fn bitxor_assign(&mut self, rhs: bool) {
@@ -415,7 +455,7 @@ define_opaque_mask! {
     /// Mask for vectors with `LANES` 8-bit elements.
     ///
     /// The layout of this type is unspecified.
-    struct Mask8<const LANES: usize>(mask_impl::Mask8<LANES>);
+    struct Mask8<const LANES: usize>(mask_impl::Mask8<Self, LANES>);
     @bits SimdI8
 }
 
@@ -423,7 +463,7 @@ define_opaque_mask! {
     /// Mask for vectors with `LANES` 16-bit elements.
     ///
     /// The layout of this type is unspecified.
-    struct Mask16<const LANES: usize>(mask_impl::Mask16<LANES>);
+    struct Mask16<const LANES: usize>(mask_impl::Mask16<Self, LANES>);
     @bits SimdI16
 }
 
@@ -431,7 +471,7 @@ define_opaque_mask! {
     /// Mask for vectors with `LANES` 32-bit elements.
     ///
     /// The layout of this type is unspecified.
-    struct Mask32<const LANES: usize>(mask_impl::Mask32<LANES>);
+    struct Mask32<const LANES: usize>(mask_impl::Mask32<Self, LANES>);
     @bits SimdI32
 }
 
@@ -439,7 +479,7 @@ define_opaque_mask! {
     /// Mask for vectors with `LANES` 64-bit elements.
     ///
     /// The layout of this type is unspecified.
-    struct Mask64<const LANES: usize>(mask_impl::Mask64<LANES>);
+    struct Mask64<const LANES: usize>(mask_impl::Mask64<Self, LANES>);
     @bits SimdI64
 }
 
@@ -447,7 +487,7 @@ define_opaque_mask! {
     /// Mask for vectors with `LANES` pointer-width elements.
     ///
     /// The layout of this type is unspecified.
-    struct MaskSize<const LANES: usize>(mask_impl::MaskSize<LANES>);
+    struct MaskSize<const LANES: usize>(mask_impl::MaskSize<Self, LANES>);
     @bits SimdIsize
 }
 
diff --git a/crates/core_simd/src/reduction.rs b/crates/core_simd/src/reduction.rs
index 2d4f1bca2647..8687d1af5167 100644
--- a/crates/core_simd/src/reduction.rs
+++ b/crates/core_simd/src/reduction.rs
@@ -104,7 +104,7 @@ macro_rules! impl_float_reductions {
 
 macro_rules! impl_full_mask_reductions {
     { $name:ident, $bits_ty:ident } => {
-        impl<const LANES: usize> $name<LANES>
+        impl<T: crate::Mask, const LANES: usize> $name<T, LANES>
         where
             crate::$bits_ty<LANES>: crate::LanesAtMost32
         {
@@ -125,7 +125,8 @@ macro_rules! impl_opaque_mask_reductions {
     { $name:ident, $bits_ty:ident } => {
         impl<const LANES: usize> $name<LANES>
         where
-            crate::$bits_ty<LANES>: crate::LanesAtMost32
+            crate::$bits_ty<LANES>: crate::LanesAtMost32,
+            $name<LANES>: crate::Mask,
         {
             /// Returns true if any lane is set, or false otherwise.
             #[inline]
diff --git a/crates/core_simd/src/vector/float.rs b/crates/core_simd/src/vector/float.rs
index 47013053ae15..6371f88a40a2 100644
--- a/crates/core_simd/src/vector/float.rs
+++ b/crates/core_simd/src/vector/float.rs
@@ -42,6 +42,7 @@ macro_rules! impl_float_vector {
             Self: crate::LanesAtMost32,
             crate::$bits_ty<LANES>: crate::LanesAtMost32,
             crate::$mask_impl_ty<LANES>: crate::LanesAtMost32,
+            crate::$mask_ty<LANES>: crate::Mask,
         {
             /// Returns true for each lane if it has a positive sign, including
             /// `+0.0`, `NaN`s with positive sign bit and positive infinity.
diff --git a/crates/core_simd/src/vector/int.rs b/crates/core_simd/src/vector/int.rs
index 30b09a229e9b..a535fad7bc1d 100644
--- a/crates/core_simd/src/vector/int.rs
+++ b/crates/core_simd/src/vector/int.rs
@@ -30,6 +30,7 @@ macro_rules! impl_integer_vector {
         where
             Self: crate::LanesAtMost32,
             crate::$mask_impl_ty<LANES>: crate::LanesAtMost32,
+            crate::$mask_ty<LANES>: crate::Mask,
         {
             /// Returns true for each positive lane and false if it is zero or negative.
             pub fn is_positive(self) -> crate::$mask_ty<LANES> {
diff --git a/crates/core_simd/src/vector/uint.rs b/crates/core_simd/src/vector/uint.rs
index 53e780520a79..db027b0941f2 100644
--- a/crates/core_simd/src/vector/uint.rs
+++ b/crates/core_simd/src/vector/uint.rs
@@ -1,6 +1,5 @@
 #![allow(non_camel_case_types)]
 
-
 /// Implements additional integer traits (Eq, Ord, Hash) on the specified vector `$name`, holding multiple `$lanes` of `$type`.
 macro_rules! impl_unsigned_vector {
     { $name:ident, $type:ty } => {
diff --git a/crates/core_simd/tests/masks.rs b/crates/core_simd/tests/masks.rs
index be83f4c2ec77..54427ec1ec0d 100644
--- a/crates/core_simd/tests/masks.rs
+++ b/crates/core_simd/tests/masks.rs
@@ -68,10 +68,12 @@ macro_rules! test_mask_api {
 
             #[test]
             fn to_bitmask() {
-                use core_simd::ToBitMask;
-                let values = [true, false, false, true, false, false, true, false];
-                let mask = core_simd::$name::<8>::from_array(values);
-                assert_eq!(mask.to_bitmask(), 0b01001001);
+                let values = [
+                    true, false, false, true, false, false, true, false,
+                    false, false, false, false, false, false, false, false,
+                ];
+                let mask = core_simd::$name::<16>::from_array(values);
+                assert_eq!(mask.to_bitmask(), [0b01001001, 0]);
             }
         }
     }
diff --git a/crates/test_helpers/src/array.rs b/crates/test_helpers/src/array.rs
index c64bfee4f2d1..5ffc92269769 100644
--- a/crates/test_helpers/src/array.rs
+++ b/crates/test_helpers/src/array.rs
@@ -3,14 +3,11 @@
 // Adapted from proptest's array code
 // Copyright 2017 Jason Lingle
 
+use core::{marker::PhantomData, mem::MaybeUninit};
 use proptest::{
     strategy::{NewTree, Strategy, ValueTree},
     test_runner::TestRunner,
 };
-use core::{
-    marker::PhantomData,
-    mem::MaybeUninit,
-};
 
 #[must_use = "strategies do nothing unless used"]
 #[derive(Clone, Copy, Debug)]
diff --git a/crates/test_helpers/src/lib.rs b/crates/test_helpers/src/lib.rs
index 8b56877967cb..fffd088f4da3 100644
--- a/crates/test_helpers/src/lib.rs
+++ b/crates/test_helpers/src/lib.rs
@@ -281,6 +281,11 @@ macro_rules! test_lanes {
                     core_simd::SimdIsize<$lanes>: core_simd::LanesAtMost32,
                     core_simd::SimdF32<$lanes>: core_simd::LanesAtMost32,
                     core_simd::SimdF64<$lanes>: core_simd::LanesAtMost32,
+                    core_simd::Mask8<$lanes>: core_simd::Mask,
+                    core_simd::Mask16<$lanes>: core_simd::Mask,
+                    core_simd::Mask32<$lanes>: core_simd::Mask,
+                    core_simd::Mask64<$lanes>: core_simd::Mask,
+                    core_simd::MaskSize<$lanes>: core_simd::Mask,
                 $body
 
                 #[cfg(target_arch = "wasm32")]
@@ -350,6 +355,11 @@ macro_rules! test_lanes_panic {
                     core_simd::SimdIsize<$lanes>: core_simd::LanesAtMost32,
                     core_simd::SimdF32<$lanes>: core_simd::LanesAtMost32,
                     core_simd::SimdF64<$lanes>: core_simd::LanesAtMost32,
+                    core_simd::Mask8<$lanes>: core_simd::Mask,
+                    core_simd::Mask16<$lanes>: core_simd::Mask,
+                    core_simd::Mask32<$lanes>: core_simd::Mask,
+                    core_simd::Mask64<$lanes>: core_simd::Mask,
+                    core_simd::MaskSize<$lanes>: core_simd::Mask,
                 $body
 
                 #[test]

From 589fce03131225f9167b6b90c6382f40ea22edb6 Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Thu, 29 Apr 2021 03:34:28 +0000
Subject: [PATCH 147/249] Attempt to workaround MIPS bug

---
 crates/core_simd/src/masks/full_masks.rs | 13 ++++++++++++-
 crates/core_simd/tests/masks.rs          |  4 ++--
 2 files changed, 14 insertions(+), 3 deletions(-)

diff --git a/crates/core_simd/src/masks/full_masks.rs b/crates/core_simd/src/masks/full_masks.rs
index bd52a25551e2..f89bbefba631 100644
--- a/crates/core_simd/src/masks/full_masks.rs
+++ b/crates/core_simd/src/masks/full_masks.rs
@@ -112,7 +112,18 @@ macro_rules! define_mask {
                     // TODO remove the transmute when rustc is more flexible
                     assert_eq!(core::mem::size_of::<U::IntBitMask>(), core::mem::size_of::<U::BitMask>());
                     let mask: U::IntBitMask = crate::intrinsics::simd_bitmask(self.0);
-                    core::mem::transmute_copy(&mask)
+                    let mut bitmask: U::BitMask = core::mem::transmute_copy(&mask);
+
+                    // There is a bug where LLVM appears to implement this operation with the wrong
+                    // bit order.
+                    // TODO fix this in a better way
+                    if cfg!(any(target_arch = "mips", target_arch = "mips64")) {
+                        for x in bitmask.as_mut() {
+                            *x = x.reverse_bits();
+                        }
+                    }
+
+                    bitmask
                 }
             }
         }
diff --git a/crates/core_simd/tests/masks.rs b/crates/core_simd/tests/masks.rs
index 54427ec1ec0d..7021d58aa543 100644
--- a/crates/core_simd/tests/masks.rs
+++ b/crates/core_simd/tests/masks.rs
@@ -70,10 +70,10 @@ macro_rules! test_mask_api {
             fn to_bitmask() {
                 let values = [
                     true, false, false, true, false, false, true, false,
-                    false, false, false, false, false, false, false, false,
+                    true, true, false, false, false, false, false, true,
                 ];
                 let mask = core_simd::$name::<16>::from_array(values);
-                assert_eq!(mask.to_bitmask(), [0b01001001, 0]);
+                assert_eq!(mask.to_bitmask(), [0b01001001, 0b10000011]);
             }
         }
     }

From 563d2a2cfce14b5807bb6cacce336e0e87950289 Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Thu, 22 Apr 2021 04:09:56 +0000
Subject: [PATCH 148/249] Add select function

---
 crates/core_simd/src/intrinsics.rs |  1 +
 crates/core_simd/src/lib.rs        |  3 ++
 crates/core_simd/src/select.rs     | 52 ++++++++++++++++++++++++++++++
 3 files changed, 56 insertions(+)
 create mode 100644 crates/core_simd/src/select.rs

diff --git a/crates/core_simd/src/intrinsics.rs b/crates/core_simd/src/intrinsics.rs
index 8cbb0cbccf79..798c4461f79f 100644
--- a/crates/core_simd/src/intrinsics.rs
+++ b/crates/core_simd/src/intrinsics.rs
@@ -81,6 +81,7 @@ extern "platform-intrinsic" {
     pub(crate) fn simd_bitmask<T, U>(x: T) -> U;
 
     // select
+    pub(crate) fn simd_select<T, U>(m: T, a: U, b: U) -> U;
     pub(crate) fn simd_select_bitmask<T, U>(m: T, a: U, b: U) -> U;
 }
 
diff --git a/crates/core_simd/src/lib.rs b/crates/core_simd/src/lib.rs
index 2d4176ce342c..3489dfb5f3ea 100644
--- a/crates/core_simd/src/lib.rs
+++ b/crates/core_simd/src/lib.rs
@@ -14,6 +14,9 @@ mod transmute;
 #[macro_use]
 mod reduction;
 
+mod select;
+pub use select::Select;
+
 mod comparisons;
 mod fmt;
 mod intrinsics;
diff --git a/crates/core_simd/src/select.rs b/crates/core_simd/src/select.rs
new file mode 100644
index 000000000000..66b0839cf248
--- /dev/null
+++ b/crates/core_simd/src/select.rs
@@ -0,0 +1,52 @@
+mod sealed {
+pub trait Sealed {}
+}
+use sealed::Sealed;
+
+/// Supporting trait for vector `select` function
+pub trait Select<Mask>: Sealed {}
+
+macro_rules! impl_select {
+    {
+        $mask:ident ($bits_ty:ident): $($type:ident),*
+    } => {
+        $(
+        impl<const LANES: usize> Sealed for crate::$type<LANES> where Self: crate::LanesAtMost32 {}
+        impl<const LANES: usize> Select<crate::$mask<LANES>> for crate::$type<LANES>
+        where
+            crate::$mask<LANES>: crate::Mask,
+            crate::$bits_ty<LANES>: crate::LanesAtMost32,
+            Self: crate::LanesAtMost32,
+        {}
+        )*
+
+        impl<const LANES: usize> crate::$mask<LANES>
+        where
+            Self: crate::Mask,
+            crate::$bits_ty<LANES>: crate::LanesAtMost32,
+        {
+            /// Choose lanes from two vectors.
+            ///
+            /// For each lane in the mask, choose the corresponding lane from `true_values` if
+            /// that lane mask is true, and `false_values` if that lane mask is false.
+            ///
+            /// ```
+            /// # use core_simd::{Mask32, SimdI32};
+            /// let a = SimdI32::from_array([0, 1, 2, 3]);
+            /// let b = SimdI32::from_array([4, 5, 6, 7]);
+            /// let mask = Mask32::from_array([true, false, false, true]);
+            /// let c = mask.select(a, b);
+            /// assert_eq!(c.to_array(), [0, 5, 6, 3]);
+            /// ```
+            pub fn select<S: Select<Self>>(self, true_values: S, false_values: S) -> S {
+                unsafe { crate::intrinsics::simd_select(self.to_int(), true_values, false_values) }
+            }
+        }
+    }
+}
+
+impl_select! { Mask8 (SimdI8): SimdU8, SimdI8 }
+impl_select! { Mask16 (SimdI16): SimdU16, SimdI16 }
+impl_select! { Mask32 (SimdI32): SimdU32, SimdI32, SimdF32}
+impl_select! { Mask64 (SimdI64): SimdU64, SimdI64, SimdF64}
+impl_select! { MaskSize (SimdIsize): SimdUsize, SimdIsize }

From 0bf5eb5f72f354489ce897a2bd1f6c1e9102fdd2 Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Sat, 8 May 2021 00:07:07 +0000
Subject: [PATCH 149/249] Add select for masks

---
 crates/core_simd/src/intrinsics.rs          |  1 +
 crates/core_simd/src/select.rs              | 45 +++++++++++++++++++--
 crates/core_simd/tests/mask_ops_impl/mod.rs |  2 +-
 3 files changed, 43 insertions(+), 5 deletions(-)

diff --git a/crates/core_simd/src/intrinsics.rs b/crates/core_simd/src/intrinsics.rs
index 798c4461f79f..3779d96a40ec 100644
--- a/crates/core_simd/src/intrinsics.rs
+++ b/crates/core_simd/src/intrinsics.rs
@@ -82,6 +82,7 @@ extern "platform-intrinsic" {
 
     // select
     pub(crate) fn simd_select<T, U>(m: T, a: U, b: U) -> U;
+    #[allow(unused)]
     pub(crate) fn simd_select_bitmask<T, U>(m: T, a: U, b: U) -> U;
 }
 
diff --git a/crates/core_simd/src/select.rs b/crates/core_simd/src/select.rs
index 66b0839cf248..a00af73ef4d4 100644
--- a/crates/core_simd/src/select.rs
+++ b/crates/core_simd/src/select.rs
@@ -1,10 +1,13 @@
 mod sealed {
-pub trait Sealed {}
+    pub trait Sealed {}
 }
 use sealed::Sealed;
 
 /// Supporting trait for vector `select` function
-pub trait Select<Mask>: Sealed {}
+pub trait Select<Mask>: Sealed {
+    #[doc(hidden)]
+    fn select(mask: Mask, true_values: Self, false_values: Self) -> Self;
+}
 
 macro_rules! impl_select {
     {
@@ -17,9 +20,32 @@ macro_rules! impl_select {
             crate::$mask<LANES>: crate::Mask,
             crate::$bits_ty<LANES>: crate::LanesAtMost32,
             Self: crate::LanesAtMost32,
-        {}
+        {
+            #[doc(hidden)]
+            #[inline]
+            fn select(mask: crate::$mask<LANES>, true_values: Self, false_values: Self) -> Self {
+                unsafe { crate::intrinsics::simd_select(mask.to_int(), true_values, false_values) }
+            }
+        }
         )*
 
+        impl<const LANES: usize> Sealed for crate::$mask<LANES>
+        where
+            Self: crate::Mask,
+            crate::$bits_ty<LANES>: crate::LanesAtMost32,
+        {}
+        impl<const LANES: usize> Select<Self> for crate::$mask<LANES>
+        where
+            Self: crate::Mask,
+            crate::$bits_ty<LANES>: crate::LanesAtMost32,
+        {
+            #[doc(hidden)]
+            #[inline]
+            fn select(mask: Self, true_values: Self, false_values: Self) -> Self {
+                mask & true_values | !mask & false_values
+            }
+        }
+
         impl<const LANES: usize> crate::$mask<LANES>
         where
             Self: crate::Mask,
@@ -38,8 +64,19 @@ macro_rules! impl_select {
             /// let c = mask.select(a, b);
             /// assert_eq!(c.to_array(), [0, 5, 6, 3]);
             /// ```
+            ///
+            /// `select` can also be used with masks:
+            /// ```
+            /// # use core_simd::{Mask32};
+            /// let a = Mask32::from_array([true, true, false, false]);
+            /// let b = Mask32::from_array([false, false, true, true]);
+            /// let mask = Mask32::from_array([true, false, false, true]);
+            /// let c = mask.select(a, b);
+            /// assert_eq!(c.to_array(), [true, false, true, false]);
+            /// ```
+            #[inline]
             pub fn select<S: Select<Self>>(self, true_values: S, false_values: S) -> S {
-                unsafe { crate::intrinsics::simd_select(self.to_int(), true_values, false_values) }
+                S::select(self, true_values, false_values)
             }
         }
     }
diff --git a/crates/core_simd/tests/mask_ops_impl/mod.rs b/crates/core_simd/tests/mask_ops_impl/mod.rs
index ff36af956515..b414167866e7 100644
--- a/crates/core_simd/tests/mask_ops_impl/mod.rs
+++ b/crates/core_simd/tests/mask_ops_impl/mod.rs
@@ -1,8 +1,8 @@
 #[macro_use]
 mod mask_macros;
 
-mod mask8;
 mod mask16;
 mod mask32;
 mod mask64;
+mod mask8;
 mod masksize;

From e8cae870fcda4f9c7104123fe352162fbcf3fe9d Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Sat, 8 May 2021 00:11:34 +0000
Subject: [PATCH 150/249] Fix rustfmt

---
 crates/core_simd/tests/mask_ops_impl/mod.rs | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/crates/core_simd/tests/mask_ops_impl/mod.rs b/crates/core_simd/tests/mask_ops_impl/mod.rs
index b414167866e7..b9ec8462a052 100644
--- a/crates/core_simd/tests/mask_ops_impl/mod.rs
+++ b/crates/core_simd/tests/mask_ops_impl/mod.rs
@@ -1,8 +1,9 @@
 #[macro_use]
 mod mask_macros;
 
+#[rustfmt::skip]
+mod mask8;
 mod mask16;
 mod mask32;
 mod mask64;
-mod mask8;
 mod masksize;

From 45d7e80aa88a689ac15a029de2af0ef698465fb4 Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Sat, 8 May 2021 00:13:40 +0000
Subject: [PATCH 151/249] Clarify documentation

---
 crates/core_simd/src/select.rs | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/crates/core_simd/src/select.rs b/crates/core_simd/src/select.rs
index a00af73ef4d4..343fd33a5350 100644
--- a/crates/core_simd/src/select.rs
+++ b/crates/core_simd/src/select.rs
@@ -65,9 +65,9 @@ macro_rules! impl_select {
             /// assert_eq!(c.to_array(), [0, 5, 6, 3]);
             /// ```
             ///
-            /// `select` can also be used with masks:
+            /// `select` can also be used on masks:
             /// ```
-            /// # use core_simd::{Mask32};
+            /// # use core_simd::Mask32;
             /// let a = Mask32::from_array([true, true, false, false]);
             /// let b = Mask32::from_array([false, false, true, true]);
             /// let mask = Mask32::from_array([true, false, false, true]);

From d6795814d402ec47c98c023dd3f298c6a3f5bfa1 Mon Sep 17 00:00:00 2001
From: miguel raz <miguelraz@gmail.com>
Date: Tue, 18 May 2021 09:26:01 -0500
Subject: [PATCH 152/249] add simd_fsqrt intrinsic

---
 crates/core_simd/src/intrinsics.rs   | 3 +++
 crates/core_simd/src/vector/float.rs | 8 ++++++++
 crates/core_simd/tests/ops_macros.rs | 7 +++++++
 3 files changed, 18 insertions(+)

diff --git a/crates/core_simd/src/intrinsics.rs b/crates/core_simd/src/intrinsics.rs
index 3779d96a40ec..7adf4c24e104 100644
--- a/crates/core_simd/src/intrinsics.rs
+++ b/crates/core_simd/src/intrinsics.rs
@@ -45,6 +45,9 @@ extern "platform-intrinsic" {
 
     /// fabs
     pub(crate) fn simd_fabs<T>(x: T) -> T;
+    
+    /// fsqrt
+    pub(crate) fn simd_fsqrt<T>(x: T) -> T;
 
     pub(crate) fn simd_eq<T, U>(x: T, y: T) -> U;
     pub(crate) fn simd_ne<T, U>(x: T, y: T) -> U;
diff --git a/crates/core_simd/src/vector/float.rs b/crates/core_simd/src/vector/float.rs
index 6371f88a40a2..c4565a9dd900 100644
--- a/crates/core_simd/src/vector/float.rs
+++ b/crates/core_simd/src/vector/float.rs
@@ -35,6 +35,14 @@ macro_rules! impl_float_vector {
             pub fn abs(self) -> Self {
                 unsafe { crate::intrinsics::simd_fabs(self) }
             }
+
+            /// Produces a vector where every lane has the square root value
+            /// of the equivalently-indexed lane in `self`
+            #[inline]
+            #[cfg(feature = "std")]
+            pub fn sqrt(self) -> Self {
+                unsafe { crate::intrinsics::simd_fsqrt(self) }
+            }
         }
 
         impl<const LANES: usize> $name<LANES>
diff --git a/crates/core_simd/tests/ops_macros.rs b/crates/core_simd/tests/ops_macros.rs
index 9f9992258770..83c6fec69e88 100644
--- a/crates/core_simd/tests/ops_macros.rs
+++ b/crates/core_simd/tests/ops_macros.rs
@@ -426,6 +426,13 @@ macro_rules! impl_float_tests {
                     )
                 }
 
+                fn sqrt<const LANES: usize>() {
+                    test_helpers::test_unary_elementwise(
+                        &Vector::<LANES>::sqrt,
+                        &Scalar::sqrt,
+                        &|_| true,
+                    )
+                }
                 fn horizontal_sum<const LANES: usize>() {
                     test_helpers::test_1(&|x| {
                         test_helpers::prop_assert_biteq! (

From 1c18f8fd59b9d7e4632393076bf2780d7c97ba12 Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Mon, 24 May 2021 08:37:15 -0400
Subject: [PATCH 153/249] Add byte conversions

---
 crates/core_simd/src/lib.rs        |  3 ++
 crates/core_simd/src/to_bytes.rs   | 68 ++++++++++++++++++++++++++++++
 crates/core_simd/tests/to_bytes.rs | 10 +++++
 3 files changed, 81 insertions(+)
 create mode 100644 crates/core_simd/src/to_bytes.rs
 create mode 100644 crates/core_simd/tests/to_bytes.rs

diff --git a/crates/core_simd/src/lib.rs b/crates/core_simd/src/lib.rs
index 3489dfb5f3ea..0a16b5dacf02 100644
--- a/crates/core_simd/src/lib.rs
+++ b/crates/core_simd/src/lib.rs
@@ -17,6 +17,9 @@ mod reduction;
 mod select;
 pub use select::Select;
 
+mod to_bytes;
+pub use to_bytes::ToBytes;
+
 mod comparisons;
 mod fmt;
 mod intrinsics;
diff --git a/crates/core_simd/src/to_bytes.rs b/crates/core_simd/src/to_bytes.rs
new file mode 100644
index 000000000000..a2d9cc4ef56c
--- /dev/null
+++ b/crates/core_simd/src/to_bytes.rs
@@ -0,0 +1,68 @@
+mod sealed {
+    pub trait Sealed {}
+}
+use sealed::Sealed;
+
+/// Supporting trait for byte conversion functions.
+pub trait ToBytes: Sealed {
+    /// The bytes representation of this type.
+    type Bytes;
+
+    #[doc(hidden)]
+    fn to_bytes_impl(self) -> Self::Bytes;
+
+    #[doc(hidden)]
+    fn from_bytes_impl(bytes: Self::Bytes) -> Self;
+}
+
+macro_rules! impl_to_bytes {
+    { $name:ident, $($int_width:literal -> $byte_width:literal),* } => {
+        $(
+        impl Sealed for crate::$name<$int_width> where Self: crate::LanesAtMost32 {}
+        impl ToBytes for crate::$name<$int_width>
+        where
+            Self: crate::LanesAtMost32,
+            crate::SimdU8<$byte_width>: crate::LanesAtMost32,
+        {
+            type Bytes = crate::SimdU8<$byte_width>;
+            fn to_bytes_impl(self) -> Self::Bytes {
+                unsafe { core::mem::transmute(self) }
+            }
+            fn from_bytes_impl(bytes: Self::Bytes) -> Self {
+                unsafe { core::mem::transmute(bytes) }
+            }
+        }
+        )*
+
+        impl<const LANES: usize> crate::$name<LANES>
+        where
+            Self: ToBytes + crate::LanesAtMost32,
+        {
+            /// Return the memory representation of this integer as a byte array in native byte
+            /// order.
+            pub fn to_ne_bytes(self) -> <Self as ToBytes>::Bytes { self.to_bytes_impl() }
+
+            /// Create a native endian integer value from its memory representation as a byte array
+            /// in native endianness.
+            pub fn from_ne_bytes(bytes: <Self as ToBytes>::Bytes) -> Self { Self::from_bytes_impl(bytes) }
+        }
+    }
+}
+
+impl_to_bytes! { SimdU8, 1 -> 1, 2 -> 2, 4 -> 4, 8 -> 8, 16 -> 16, 32 -> 32 }
+impl_to_bytes! { SimdU16, 1 -> 2, 2 -> 4, 4 -> 8, 8 -> 16, 16 -> 32 }
+impl_to_bytes! { SimdU32, 1 -> 4, 2 -> 8, 4 -> 16, 8 -> 32 }
+impl_to_bytes! { SimdU64, 1 -> 8, 2 -> 16, 4 -> 32 }
+#[cfg(target_pointer_width = "32")]
+impl_to_bytes! { SimdUsize, 1 -> 4, 2 -> 8, 4 -> 16, 8 -> 32 }
+#[cfg(target_pointer_width = "64")]
+impl_to_bytes! { SimdUsize, 1 -> 8, 2 -> 16, 4 -> 32 }
+
+impl_to_bytes! { SimdI8, 1 -> 1, 2 -> 2, 4 -> 4, 8 -> 8, 16 -> 16, 32 -> 32 }
+impl_to_bytes! { SimdI16, 1 -> 2, 2 -> 4, 4 -> 8, 8 -> 16, 16 -> 32 }
+impl_to_bytes! { SimdI32, 1 -> 4, 2 -> 8, 4 -> 16, 8 -> 32 }
+impl_to_bytes! { SimdI64, 1 -> 8, 2 -> 16, 4 -> 32 }
+#[cfg(target_pointer_width = "32")]
+impl_to_bytes! { SimdIsize, 1 -> 4, 2 -> 8, 4 -> 16, 8 -> 32 }
+#[cfg(target_pointer_width = "64")]
+impl_to_bytes! { SimdIsize, 1 -> 8, 2 -> 16, 4 -> 32 }
diff --git a/crates/core_simd/tests/to_bytes.rs b/crates/core_simd/tests/to_bytes.rs
new file mode 100644
index 000000000000..20da1652a6dd
--- /dev/null
+++ b/crates/core_simd/tests/to_bytes.rs
@@ -0,0 +1,10 @@
+use core_simd::SimdU32;
+
+#[test]
+fn byte_convert() {
+    let int = SimdU32::from_array([0xdeadbeef, 0x8badf00d]);
+    let bytes = int.to_ne_bytes();
+    assert_eq!(int[0].to_ne_bytes(), bytes[..4]); 
+    assert_eq!(int[1].to_ne_bytes(), bytes[4..]);
+    assert_eq!(SimdU32::from_ne_bytes(bytes), int);
+}

From e52d51cd45230068a98e5e809dabf1671bc6d630 Mon Sep 17 00:00:00 2001
From: miguel raz <miguelraz@gmail.com>
Date: Tue, 25 May 2021 20:00:01 -0500
Subject: [PATCH 154/249] nbody example

---
 crates/core_simd/examples/nbody.rs | 188 +++++++++++++++++++++++++++++
 1 file changed, 188 insertions(+)
 create mode 100644 crates/core_simd/examples/nbody.rs

diff --git a/crates/core_simd/examples/nbody.rs b/crates/core_simd/examples/nbody.rs
new file mode 100644
index 000000000000..a474bbe39259
--- /dev/null
+++ b/crates/core_simd/examples/nbody.rs
@@ -0,0 +1,188 @@
+use stdsimd::simd::*;
+
+use std::f64::consts::PI;
+const SOLAR_MASS: f64 = 4.0 * PI * PI;
+const DAYS_PER_YEAR: f64 = 365.24;
+
+pub struct Body {
+    pub x: f64x4,
+    pub v: f64x4,
+    pub mass: f64,
+}
+const N_BODIES: usize = 5;
+#[allow(clippy::unreadable_literal)]
+const BODIES: [Body; N_BODIES] = [
+    // sun:
+    Body {
+        x: f64x4::new(0., 0., 0., 0.),
+        v: f64x4::new(0., 0., 0., 0.),
+        mass: SOLAR_MASS,
+    },
+    // jupiter:
+    Body {
+        x: f64x4::new(
+            4.84143144246472090e+00,
+            -1.16032004402742839e+00,
+            -1.03622044471123109e-01,
+            0.,
+        ),
+        v: f64x4::new(
+            1.66007664274403694e-03 * DAYS_PER_YEAR,
+            7.69901118419740425e-03 * DAYS_PER_YEAR,
+            -6.90460016972063023e-05 * DAYS_PER_YEAR,
+            0.,
+        ),
+        mass: 9.54791938424326609e-04 * SOLAR_MASS,
+    },
+    // saturn:
+    Body {
+        x: f64x4::new(
+            8.34336671824457987e+00,
+            4.12479856412430479e+00,
+            -4.03523417114321381e-01,
+            0.,
+        ),
+        v: f64x4::new(
+            -2.76742510726862411e-03 * DAYS_PER_YEAR,
+            4.99852801234917238e-03 * DAYS_PER_YEAR,
+            2.30417297573763929e-05 * DAYS_PER_YEAR,
+            0.,
+        ),
+        mass: 2.85885980666130812e-04 * SOLAR_MASS,
+    },
+    // uranus:
+    Body {
+        x: f64x4::new(
+            1.28943695621391310e+01,
+            -1.51111514016986312e+01,
+            -2.23307578892655734e-01,
+            0.,
+        ),
+        v: f64x4::new(
+            2.96460137564761618e-03 * DAYS_PER_YEAR,
+            2.37847173959480950e-03 * DAYS_PER_YEAR,
+            -2.96589568540237556e-05 * DAYS_PER_YEAR,
+            0.,
+        ),
+        mass: 4.36624404335156298e-05 * SOLAR_MASS,
+    },
+    // neptune:
+    Body {
+        x: f64x4::new(
+            1.53796971148509165e+01,
+            -2.59193146099879641e+01,
+            1.79258772950371181e-01,
+            0.,
+        ),
+        v: f64x4::new(
+            2.68067772490389322e-03 * DAYS_PER_YEAR,
+            1.62824170038242295e-03 * DAYS_PER_YEAR,
+            -9.51592254519715870e-05 * DAYS_PER_YEAR,
+            0.,
+        ),
+        mass: 5.15138902046611451e-05 * SOLAR_MASS,
+    },
+];
+
+pub fn offset_momentum(bodies: &mut [Body; N_BODIES]) {
+    let (sun, rest) = bodies.split_at_mut(1);
+    let sun = &mut sun[0];
+    for body in rest {
+        let m_ratio = body.mass / SOLAR_MASS;
+        sun.v -= body.v * m_ratio;
+    }
+}
+
+pub fn energy(bodies: &[Body; N_BODIES]) -> f64 {
+    let mut e = 0.;
+    for i in 0..N_BODIES {
+        let bi = &bodies[i];
+        e += bi.mass * (bi.v * bi.v).sum() * 0.5;
+        for bj in bodies.iter().take(N_BODIES).skip(i + 1) {
+            let dx = bi.x - bj.x;
+            e -= bi.mass * bj.mass / (dx * dx).sum().sqrt()
+        }
+    }
+    e
+}
+
+pub fn advance(bodies: &mut [Body; N_BODIES], dt: f64) {
+    const N: usize = N_BODIES * (N_BODIES - 1) / 2;
+
+    // compute distance between bodies:
+    let mut r = [f64x4::splat(0.); N];
+    {
+        let mut i = 0;
+        for j in 0..N_BODIES {
+            for k in j + 1..N_BODIES {
+                r[i] = bodies[j].x - bodies[k].x;
+                i += 1;
+            }
+        }
+    }
+
+    let mut mag = [0.0; N];
+    let mut i = 0;
+    while i < N {
+        let d2s = f64x2::new((r[i] * r[i]).sum(), (r[i + 1] * r[i + 1]).sum());
+        let dmags = f64x2::splat(dt) / (d2s * d2s.sqrte());
+        dmags.write_to_slice_unaligned(&mut mag[i..]);
+        i += 2;
+    }
+
+    i = 0;
+    for j in 0..N_BODIES {
+        for k in j + 1..N_BODIES {
+            let f = r[i] * mag[i];
+            bodies[j].v -= f * bodies[k].mass;
+            bodies[k].v += f * bodies[j].mass;
+            i += 1
+        }
+    }
+    for body in bodies {
+        body.x += dt * body.v
+    }
+}
+
+pub fn run_k<K>(n: usize, k: K) -> (f64, f64)
+where
+    K: Fn(&mut [Body; N_BODIES], f64),
+{
+    let mut bodies = BODIES;
+    offset_momentum(&mut bodies);
+    let energy_before = energy(&bodies);
+    for _ in 0..n {
+        k(&mut bodies, 0.01);
+    }
+    let energy_after = energy(&bodies);
+
+    (energy_before, energy_after)
+}
+
+pub fn run(n: usize) -> (f64, f64) {
+    run_k(n, advance)
+}
+
+const OUTPUT: Vec<f64> = vec![-0.169075164, -0.169087605];
+#[cfg(test)]
+mod tests {
+    #[test]
+    fn test() {
+        let mut out: Vec<u8> = Vec::new();
+        run(&mut out, 1000, 0);
+        for &(size, a_e, b_e) in crate::RESULTS {
+            let (a, b) = super::run(size);
+            assert_eq!(format!("{:.9}", a), a_e);
+            assert_eq!(format!("{:.9}", b), b_e);
+        }
+    }
+}
+fn main() {
+    //let n: usize = std::env::args()
+    //.nth(1)
+    //.expect("need one arg")
+    //.parse()
+    //.expect("argument should be a usize");
+    //run(&mut std::io::stdout(), n, alg);
+    println!("{:?}", run_k<10>(10, 10));
+}

From 2591c59ba75f812329c44078b31b3ca140d01055 Mon Sep 17 00:00:00 2001
From: miguel raz <miguelraz@gmail.com>
Date: Tue, 25 May 2021 20:07:20 -0500
Subject: [PATCH 155/249] fix imports

---
 crates/core_simd/examples/nbody.rs | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/crates/core_simd/examples/nbody.rs b/crates/core_simd/examples/nbody.rs
index a474bbe39259..4ca371f44569 100644
--- a/crates/core_simd/examples/nbody.rs
+++ b/crates/core_simd/examples/nbody.rs
@@ -1,4 +1,5 @@
-use stdsimd::simd::*;
+#![feature(platform_intrinsics, repr_simd)]
+use core_simd::*;
 
 use std::f64::consts::PI;
 const SOLAR_MASS: f64 = 4.0 * PI * PI;

From ab6af37f8f9ba27df1cf24789a7c3f50bf3005cf Mon Sep 17 00:00:00 2001
From: miguel raz <miguelraz@gmail.com>
Date: Tue, 25 May 2021 20:10:55 -0500
Subject: [PATCH 156/249] Simdf64 from attempt

---
 crates/core_simd/examples/nbody.rs | 9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)

diff --git a/crates/core_simd/examples/nbody.rs b/crates/core_simd/examples/nbody.rs
index 4ca371f44569..cbdcf848fee3 100644
--- a/crates/core_simd/examples/nbody.rs
+++ b/crates/core_simd/examples/nbody.rs
@@ -5,11 +5,16 @@ use std::f64::consts::PI;
 const SOLAR_MASS: f64 = 4.0 * PI * PI;
 const DAYS_PER_YEAR: f64 = 365.24;
 
+#[derive(Debug)]
+#[repr(simd)]
 pub struct Body {
-    pub x: f64x4,
-    pub v: f64x4,
+    pub x: Simdf64([f64, 4]),
+    pub v: Simdf64([f64, 4]),
     pub mass: f64,
 }
+
+// Translation attempt is this ^^^ far
+//
 const N_BODIES: usize = 5;
 #[allow(clippy::unreadable_literal)]
 const BODIES: [Body; N_BODIES] = [

From 8bea3627cb6f4c9378908599a146e8896034d793 Mon Sep 17 00:00:00 2001
From: miguel raz <miguelraz@gmail.com>
Date: Tue, 25 May 2021 21:25:42 -0500
Subject: [PATCH 157/249] replace sum() with horizontal_sum()

---
 crates/core_simd/examples/nbody.rs | 56 ++++++++++++++----------------
 1 file changed, 27 insertions(+), 29 deletions(-)

diff --git a/crates/core_simd/examples/nbody.rs b/crates/core_simd/examples/nbody.rs
index cbdcf848fee3..93d4c902a4d0 100644
--- a/crates/core_simd/examples/nbody.rs
+++ b/crates/core_simd/examples/nbody.rs
@@ -5,87 +5,85 @@ use std::f64::consts::PI;
 const SOLAR_MASS: f64 = 4.0 * PI * PI;
 const DAYS_PER_YEAR: f64 = 365.24;
 
-#[derive(Debug)]
-#[repr(simd)]
+#[derive(Debug, Clone, Copy)]
 pub struct Body {
-    pub x: Simdf64([f64, 4]),
-    pub v: Simdf64([f64, 4]),
+    pub x: f64x4,
+    pub v: f64x4,
     pub mass: f64,
 }
 
-// Translation attempt is this ^^^ far
-//
+// translation up to here
 const N_BODIES: usize = 5;
 #[allow(clippy::unreadable_literal)]
 const BODIES: [Body; N_BODIES] = [
     // sun:
     Body {
-        x: f64x4::new(0., 0., 0., 0.),
-        v: f64x4::new(0., 0., 0., 0.),
+        x: f64x4::from_array([0., 0., 0., 0.]),
+        v: f64x4::from_array([0., 0., 0., 0.]),
         mass: SOLAR_MASS,
     },
     // jupiter:
     Body {
-        x: f64x4::new(
+        x: f64x4::from_array([
             4.84143144246472090e+00,
             -1.16032004402742839e+00,
             -1.03622044471123109e-01,
             0.,
-        ),
-        v: f64x4::new(
+        ]),
+        v: f64x4::from_array([
             1.66007664274403694e-03 * DAYS_PER_YEAR,
             7.69901118419740425e-03 * DAYS_PER_YEAR,
             -6.90460016972063023e-05 * DAYS_PER_YEAR,
             0.,
-        ),
+        ]),
         mass: 9.54791938424326609e-04 * SOLAR_MASS,
     },
     // saturn:
     Body {
-        x: f64x4::new(
+        x: f64x4::from_array([
             8.34336671824457987e+00,
             4.12479856412430479e+00,
             -4.03523417114321381e-01,
             0.,
-        ),
-        v: f64x4::new(
+        ]),
+        v: f64x4::from_array([
             -2.76742510726862411e-03 * DAYS_PER_YEAR,
             4.99852801234917238e-03 * DAYS_PER_YEAR,
             2.30417297573763929e-05 * DAYS_PER_YEAR,
             0.,
-        ),
+        ]),
         mass: 2.85885980666130812e-04 * SOLAR_MASS,
     },
     // uranus:
     Body {
-        x: f64x4::new(
+        x: f64x4::from_array([
             1.28943695621391310e+01,
             -1.51111514016986312e+01,
             -2.23307578892655734e-01,
             0.,
-        ),
-        v: f64x4::new(
+        ]),
+        v: f64x4::from_array([
             2.96460137564761618e-03 * DAYS_PER_YEAR,
             2.37847173959480950e-03 * DAYS_PER_YEAR,
             -2.96589568540237556e-05 * DAYS_PER_YEAR,
             0.,
-        ),
+        ]),
         mass: 4.36624404335156298e-05 * SOLAR_MASS,
     },
     // neptune:
     Body {
-        x: f64x4::new(
+        x: f64x4::from_array([
             1.53796971148509165e+01,
             -2.59193146099879641e+01,
             1.79258772950371181e-01,
             0.,
-        ),
-        v: f64x4::new(
+        ]),
+        v: f64x4::from_array([
             2.68067772490389322e-03 * DAYS_PER_YEAR,
             1.62824170038242295e-03 * DAYS_PER_YEAR,
             -9.51592254519715870e-05 * DAYS_PER_YEAR,
             0.,
-        ),
+        ]),
         mass: 5.15138902046611451e-05 * SOLAR_MASS,
     },
 ];
@@ -103,10 +101,10 @@ pub fn energy(bodies: &[Body; N_BODIES]) -> f64 {
     let mut e = 0.;
     for i in 0..N_BODIES {
         let bi = &bodies[i];
-        e += bi.mass * (bi.v * bi.v).sum() * 0.5;
+        e += bi.mass * (bi.v * bi.v).horizontal_sum() * 0.5;
         for bj in bodies.iter().take(N_BODIES).skip(i + 1) {
             let dx = bi.x - bj.x;
-            e -= bi.mass * bj.mass / (dx * dx).sum().sqrt()
+            e -= bi.mass * bj.mass / (dx * dx).horizontal_sum().sqrt()
         }
     }
     e
@@ -130,8 +128,8 @@ pub fn advance(bodies: &mut [Body; N_BODIES], dt: f64) {
     let mut mag = [0.0; N];
     let mut i = 0;
     while i < N {
-        let d2s = f64x2::new((r[i] * r[i]).sum(), (r[i + 1] * r[i + 1]).sum());
-        let dmags = f64x2::splat(dt) / (d2s * d2s.sqrte());
+        let d2s = f64x2::from_array([(r[i] * r[i]).horizontal_sum(), (r[i + 1] * r[i + 1]).horizontal_sum()]);
+        let dmags = f64x2::splat(dt) / (d2s * d2s.sqrt());
         dmags.write_to_slice_unaligned(&mut mag[i..]);
         i += 2;
     }
@@ -190,5 +188,5 @@ fn main() {
     //.parse()
     //.expect("argument should be a usize");
     //run(&mut std::io::stdout(), n, alg);
-    println!("{:?}", run_k<10>(10, 10));
+    println!("{:?}", run_k::<10>(10, 10));
 }

From 992621844fa9e5915bef054b095ca2e71b071816 Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Wed, 26 May 2021 10:27:16 -0400
Subject: [PATCH 158/249] Remove extended_key_value_attributes feature

---
 crates/core_simd/src/lib.rs | 1 -
 1 file changed, 1 deletion(-)

diff --git a/crates/core_simd/src/lib.rs b/crates/core_simd/src/lib.rs
index 0a16b5dacf02..7fe7d666e8d8 100644
--- a/crates/core_simd/src/lib.rs
+++ b/crates/core_simd/src/lib.rs
@@ -1,7 +1,6 @@
 #![no_std]
 #![allow(incomplete_features)]
 #![feature(repr_simd, platform_intrinsics, simd_ffi, const_generics)]
-#![feature(extended_key_value_attributes)]
 #![warn(missing_docs)]
 //! Portable SIMD module.
 

From 3c05ceec70a3463a86ab5481b345e550c892641b Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Miguel=20Raz=20Guzm=C3=A1n=20Macedo?=
 <miguelraz@ciencias.unam.mx>
Date: Mon, 31 May 2021 18:03:55 -0500
Subject: [PATCH 159/249] Update crates/core_simd/examples/nbody.rs

Co-authored-by: Jubilee <46493976+workingjubilee@users.noreply.github.com>
---
 crates/core_simd/examples/nbody.rs | 1 -
 1 file changed, 1 deletion(-)

diff --git a/crates/core_simd/examples/nbody.rs b/crates/core_simd/examples/nbody.rs
index 93d4c902a4d0..1d97b6b3ad2e 100644
--- a/crates/core_simd/examples/nbody.rs
+++ b/crates/core_simd/examples/nbody.rs
@@ -1,4 +1,3 @@
-#![feature(platform_intrinsics, repr_simd)]
 use core_simd::*;
 
 use std::f64::consts::PI;

From 83dc5b782b01637be5ba661752ba06da84587277 Mon Sep 17 00:00:00 2001
From: miguel raz <miguelraz@gmail.com>
Date: Tue, 1 Jun 2021 12:09:11 -0500
Subject: [PATCH 160/249] don't need clippy

---
 crates/core_simd/examples/nbody.rs | 1 -
 1 file changed, 1 deletion(-)

diff --git a/crates/core_simd/examples/nbody.rs b/crates/core_simd/examples/nbody.rs
index 1d97b6b3ad2e..27e4634c9d29 100644
--- a/crates/core_simd/examples/nbody.rs
+++ b/crates/core_simd/examples/nbody.rs
@@ -13,7 +13,6 @@ pub struct Body {
 
 // translation up to here
 const N_BODIES: usize = 5;
-#[allow(clippy::unreadable_literal)]
 const BODIES: [Body; N_BODIES] = [
     // sun:
     Body {

From 56050562f1be867c65f38a94498e3ddfda5759a7 Mon Sep 17 00:00:00 2001
From: miguel raz <miguelraz@gmail.com>
Date: Tue, 1 Jun 2021 12:15:37 -0500
Subject: [PATCH 161/249] don't use turbofish on run

---
 crates/core_simd/examples/nbody.rs | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/crates/core_simd/examples/nbody.rs b/crates/core_simd/examples/nbody.rs
index 27e4634c9d29..87bdcf4e7c80 100644
--- a/crates/core_simd/examples/nbody.rs
+++ b/crates/core_simd/examples/nbody.rs
@@ -186,5 +186,5 @@ fn main() {
     //.parse()
     //.expect("argument should be a usize");
     //run(&mut std::io::stdout(), n, alg);
-    println!("{:?}", run_k::<10>(10, 10));
+    println!("{:?}", run(10));
 }

From f24110aa5b59ecbaf457f73b3a3b1560b416322d Mon Sep 17 00:00:00 2001
From: miguel raz <miguelraz@gmail.com>
Date: Tue, 1 Jun 2021 12:26:29 -0500
Subject: [PATCH 162/249] collapse run_k into run

---
 crates/core_simd/examples/nbody.rs | 11 ++---------
 1 file changed, 2 insertions(+), 9 deletions(-)

diff --git a/crates/core_simd/examples/nbody.rs b/crates/core_simd/examples/nbody.rs
index 87bdcf4e7c80..bf045737f82e 100644
--- a/crates/core_simd/examples/nbody.rs
+++ b/crates/core_simd/examples/nbody.rs
@@ -146,25 +146,18 @@ pub fn advance(bodies: &mut [Body; N_BODIES], dt: f64) {
     }
 }
 
-pub fn run_k<K>(n: usize, k: K) -> (f64, f64)
-where
-    K: Fn(&mut [Body; N_BODIES], f64),
-{
+pub fn run(n: usize) -> (f64, f64) {
     let mut bodies = BODIES;
     offset_momentum(&mut bodies);
     let energy_before = energy(&bodies);
     for _ in 0..n {
-        k(&mut bodies, 0.01);
+        advance(&mut bodies, 0.01);
     }
     let energy_after = energy(&bodies);
 
     (energy_before, energy_after)
 }
 
-pub fn run(n: usize) -> (f64, f64) {
-    run_k(n, advance)
-}
-
 const OUTPUT: Vec<f64> = vec![-0.169075164, -0.169087605];
 #[cfg(test)]
 mod tests {

From 5557907098d0c9269d6d6643ead23d9877ee63ab Mon Sep 17 00:00:00 2001
From: miguel raz <miguelraz@gmail.com>
Date: Tue, 1 Jun 2021 15:25:56 -0500
Subject: [PATCH 163/249] rewrite unaligned slice, fix output const array

---
 crates/core_simd/examples/nbody.rs | 14 ++++++++++++--
 1 file changed, 12 insertions(+), 2 deletions(-)

diff --git a/crates/core_simd/examples/nbody.rs b/crates/core_simd/examples/nbody.rs
index bf045737f82e..4fdc02263976 100644
--- a/crates/core_simd/examples/nbody.rs
+++ b/crates/core_simd/examples/nbody.rs
@@ -128,7 +128,9 @@ pub fn advance(bodies: &mut [Body; N_BODIES], dt: f64) {
     while i < N {
         let d2s = f64x2::from_array([(r[i] * r[i]).horizontal_sum(), (r[i + 1] * r[i + 1]).horizontal_sum()]);
         let dmags = f64x2::splat(dt) / (d2s * d2s.sqrt());
-        dmags.write_to_slice_unaligned(&mut mag[i..]);
+        // dmags.write_to_slice_unaligned(&mut mag[i..]);
+        mag[i] = dmags[0];
+        mag[i+1] = dmags[1];
         i += 2;
     }
 
@@ -146,6 +148,12 @@ pub fn advance(bodies: &mut [Body; N_BODIES], dt: f64) {
     }
 }
 
+// #[inline]
+// pub unsafe fn write_to_slice_unaligned(slice: &mut SimdF64::<LANES>) {
+//     let target_ptr = slice.get_unchecked_mut(0) as *mut f64x2;
+//     *(target_ptr as *mut f64x2) = SimdF64;
+// }
+
 pub fn run(n: usize) -> (f64, f64) {
     let mut bodies = BODIES;
     offset_momentum(&mut bodies);
@@ -158,7 +166,7 @@ pub fn run(n: usize) -> (f64, f64) {
     (energy_before, energy_after)
 }
 
-const OUTPUT: Vec<f64> = vec![-0.169075164, -0.169087605];
+const OUTPUT: [f64; 2] = [-0.169075164, -0.169087605];
 #[cfg(test)]
 mod tests {
     #[test]
@@ -172,6 +180,8 @@ mod tests {
         }
     }
 }
+
+
 fn main() {
     //let n: usize = std::env::args()
     //.nth(1)

From 4e86aeb7f9f9c16a3b7d43b7124ae6885bc9b71e Mon Sep 17 00:00:00 2001
From: miguel raz <miguelraz@gmail.com>
Date: Tue, 1 Jun 2021 19:31:34 -0500
Subject: [PATCH 164/249] finish nbody

---
 crates/core_simd/examples/nbody.rs | 39 ++++++++++++++----------------
 1 file changed, 18 insertions(+), 21 deletions(-)

diff --git a/crates/core_simd/examples/nbody.rs b/crates/core_simd/examples/nbody.rs
index 4fdc02263976..5873f311947e 100644
--- a/crates/core_simd/examples/nbody.rs
+++ b/crates/core_simd/examples/nbody.rs
@@ -1,3 +1,6 @@
+/// Benchmarks game nbody code
+/// Taken from the  `packed_simd` crate
+/// Run this benchmark with `cargo test --example body`
 use core_simd::*;
 
 use std::f64::consts::PI;
@@ -126,11 +129,14 @@ pub fn advance(bodies: &mut [Body; N_BODIES], dt: f64) {
     let mut mag = [0.0; N];
     let mut i = 0;
     while i < N {
-        let d2s = f64x2::from_array([(r[i] * r[i]).horizontal_sum(), (r[i + 1] * r[i + 1]).horizontal_sum()]);
+        let d2s = f64x2::from_array([
+            (r[i] * r[i]).horizontal_sum(),
+            (r[i + 1] * r[i + 1]).horizontal_sum(),
+        ]);
         let dmags = f64x2::splat(dt) / (d2s * d2s.sqrt());
         // dmags.write_to_slice_unaligned(&mut mag[i..]);
         mag[i] = dmags[0];
-        mag[i+1] = dmags[1];
+        mag[i + 1] = dmags[1];
         i += 2;
     }
 
@@ -166,28 +172,19 @@ pub fn run(n: usize) -> (f64, f64) {
     (energy_before, energy_after)
 }
 
-const OUTPUT: [f64; 2] = [-0.169075164, -0.169087605];
+fn approx_eq_f32(a: f32, b: f32) -> bool {
+    (a - b).abs() < 0.00000001
+}
+
 #[cfg(test)]
 mod tests {
     #[test]
     fn test() {
-        let mut out: Vec<u8> = Vec::new();
-        run(&mut out, 1000, 0);
-        for &(size, a_e, b_e) in crate::RESULTS {
-            let (a, b) = super::run(size);
-            assert_eq!(format!("{:.9}", a), a_e);
-            assert_eq!(format!("{:.9}", b), b_e);
-        }
+        use super::*;
+        const OUTPUT: [f64; 2] = [-0.169075164, -0.169087605];
+        let (energy_before, energy_after) = super::run(1000);
+        assert!(approx_eq_f32(energy_before as f32, OUTPUT[0] as f32));
+        assert!(approx_eq_f32(energy_after as f32, OUTPUT[1] as f32));
+        // }
     }
 }
-
-
-fn main() {
-    //let n: usize = std::env::args()
-    //.nth(1)
-    //.expect("need one arg")
-    //.parse()
-    //.expect("argument should be a usize");
-    //run(&mut std::io::stdout(), n, alg);
-    println!("{:?}", run(10));
-}

From 70305c5fad7da367d1ff2c443076c378a70b6a0d Mon Sep 17 00:00:00 2001
From: miguel raz <miguelraz@gmail.com>
Date: Tue, 1 Jun 2021 19:46:48 -0500
Subject: [PATCH 165/249] add main to avoid CI crash

---
 crates/core_simd/examples/nbody.rs | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/crates/core_simd/examples/nbody.rs b/crates/core_simd/examples/nbody.rs
index 5873f311947e..9ddf49b4e116 100644
--- a/crates/core_simd/examples/nbody.rs
+++ b/crates/core_simd/examples/nbody.rs
@@ -188,3 +188,7 @@ mod tests {
         // }
     }
 }
+
+fn main () {
+    // empty main to pass CI
+}

From c042f33673bb298c889b10d91c7bed2db1df0cc2 Mon Sep 17 00:00:00 2001
From: miguel raz <miguelraz@gmail.com>
Date: Tue, 1 Jun 2021 20:05:30 -0500
Subject: [PATCH 166/249] clean up code, fudge approx true

---
 crates/core_simd/examples/nbody.rs | 18 +++++-------------
 1 file changed, 5 insertions(+), 13 deletions(-)

diff --git a/crates/core_simd/examples/nbody.rs b/crates/core_simd/examples/nbody.rs
index 9ddf49b4e116..0bad0c2d5579 100644
--- a/crates/core_simd/examples/nbody.rs
+++ b/crates/core_simd/examples/nbody.rs
@@ -14,7 +14,6 @@ pub struct Body {
     pub mass: f64,
 }
 
-// translation up to here
 const N_BODIES: usize = 5;
 const BODIES: [Body; N_BODIES] = [
     // sun:
@@ -134,7 +133,6 @@ pub fn advance(bodies: &mut [Body; N_BODIES], dt: f64) {
             (r[i + 1] * r[i + 1]).horizontal_sum(),
         ]);
         let dmags = f64x2::splat(dt) / (d2s * d2s.sqrt());
-        // dmags.write_to_slice_unaligned(&mut mag[i..]);
         mag[i] = dmags[0];
         mag[i + 1] = dmags[1];
         i += 2;
@@ -154,12 +152,6 @@ pub fn advance(bodies: &mut [Body; N_BODIES], dt: f64) {
     }
 }
 
-// #[inline]
-// pub unsafe fn write_to_slice_unaligned(slice: &mut SimdF64::<LANES>) {
-//     let target_ptr = slice.get_unchecked_mut(0) as *mut f64x2;
-//     *(target_ptr as *mut f64x2) = SimdF64;
-// }
-
 pub fn run(n: usize) -> (f64, f64) {
     let mut bodies = BODIES;
     offset_momentum(&mut bodies);
@@ -172,8 +164,9 @@ pub fn run(n: usize) -> (f64, f64) {
     (energy_before, energy_after)
 }
 
-fn approx_eq_f32(a: f32, b: f32) -> bool {
-    (a - b).abs() < 0.00000001
+// Good enough for demonstration purposes, not going for strictness here.
+fn approx_eq_f64(a: f64, b: f64) -> bool {
+    (a - b).abs() < 0.00001
 }
 
 #[cfg(test)]
@@ -183,9 +176,8 @@ mod tests {
         use super::*;
         const OUTPUT: [f64; 2] = [-0.169075164, -0.169087605];
         let (energy_before, energy_after) = super::run(1000);
-        assert!(approx_eq_f32(energy_before as f32, OUTPUT[0] as f32));
-        assert!(approx_eq_f32(energy_after as f32, OUTPUT[1] as f32));
-        // }
+        assert!(approx_eq_f64(energy_before, OUTPUT[0]));
+        assert!(approx_eq_f64(energy_after, OUTPUT[1]));
     }
 }
 

From 435d1cf7a6985696291048e3b5f2bf7e1b63aa32 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Miguel=20Raz=20Guzm=C3=A1n=20Macedo?=
 <miguelraz@ciencias.unam.mx>
Date: Fri, 4 Jun 2021 10:40:39 -0500
Subject: [PATCH 167/249] Update crates/core_simd/examples/nbody.rs

Co-authored-by: Jubilee <46493976+workingjubilee@users.noreply.github.com>
---
 crates/core_simd/examples/nbody.rs | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/crates/core_simd/examples/nbody.rs b/crates/core_simd/examples/nbody.rs
index 0bad0c2d5579..0aad4935f3a9 100644
--- a/crates/core_simd/examples/nbody.rs
+++ b/crates/core_simd/examples/nbody.rs
@@ -1,6 +1,6 @@
 /// Benchmarks game nbody code
-/// Taken from the  `packed_simd` crate
-/// Run this benchmark with `cargo test --example body`
+/// Taken from the `packed_simd` crate
+/// Run this benchmark with `cargo test --example nbody`
 use core_simd::*;
 
 use std::f64::consts::PI;

From be121c93ffc443a82735321edd793ef1e3ee3a00 Mon Sep 17 00:00:00 2001
From: miguel raz <miguelraz@gmail.com>
Date: Fri, 4 Jun 2021 10:54:08 -0500
Subject: [PATCH 168/249] clean code vis. Jubilee's comments

---
 crates/core_simd/examples/nbody.rs | 18 +++++++-----------
 1 file changed, 7 insertions(+), 11 deletions(-)

diff --git a/crates/core_simd/examples/nbody.rs b/crates/core_simd/examples/nbody.rs
index 0aad4935f3a9..44e1c6e87d0e 100644
--- a/crates/core_simd/examples/nbody.rs
+++ b/crates/core_simd/examples/nbody.rs
@@ -126,8 +126,7 @@ pub fn advance(bodies: &mut [Body; N_BODIES], dt: f64) {
     }
 
     let mut mag = [0.0; N];
-    let mut i = 0;
-    while i < N {
+    for i in (0..N).step_by(2) {
         let d2s = f64x2::from_array([
             (r[i] * r[i]).horizontal_sum(),
             (r[i + 1] * r[i + 1]).horizontal_sum(),
@@ -135,10 +134,9 @@ pub fn advance(bodies: &mut [Body; N_BODIES], dt: f64) {
         let dmags = f64x2::splat(dt) / (d2s * d2s.sqrt());
         mag[i] = dmags[0];
         mag[i + 1] = dmags[1];
-        i += 2;
     }
 
-    i = 0;
+    let mut i = 0;
     for j in 0..N_BODIES {
         for k in j + 1..N_BODIES {
             let f = r[i] * mag[i];
@@ -164,16 +162,14 @@ pub fn run(n: usize) -> (f64, f64) {
     (energy_before, energy_after)
 }
 
-// Good enough for demonstration purposes, not going for strictness here.
-fn approx_eq_f64(a: f64, b: f64) -> bool {
-    (a - b).abs() < 0.00001
-}
-
 #[cfg(test)]
 mod tests {
+    // Good enough for demonstration purposes, not going for strictness here.
+    fn approx_eq_f64(a: f64, b: f64) -> bool {
+        (a - b).abs() < 0.00001
+    }
     #[test]
     fn test() {
-        use super::*;
         const OUTPUT: [f64; 2] = [-0.169075164, -0.169087605];
         let (energy_before, energy_after) = super::run(1000);
         assert!(approx_eq_f64(energy_before, OUTPUT[0]));
@@ -181,6 +177,6 @@ mod tests {
     }
 }
 
-fn main () {
+fn main() {
     // empty main to pass CI
 }

From 3032a62d9310f34388fcff9be86c253e15b4f494 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Miguel=20Raz=20Guzm=C3=A1n=20Macedo?=
 <miguelraz@ciencias.unam.mx>
Date: Mon, 7 Jun 2021 12:56:15 -0500
Subject: [PATCH 169/249] add helloworld to README (#134)

* add helloworld to README
---
 README.md | 34 ++++++++++++++++++++++++++++++++++
 1 file changed, 34 insertions(+)

diff --git a/README.md b/README.md
index fe17108c843f..c6a48c467b70 100644
--- a/README.md
+++ b/README.md
@@ -12,6 +12,40 @@ We can also be found on [Zulip][zulip-project-portable-simd].
 
 If you are interested in support for a specific architecture, you may want [stdarch] instead.
 
+## Hello World
+
+Now we're gonna dip our toes into this world with a small SIMD "Hello, World!" example. Make sure your compiler is up to date and using `nightly`. We can do that by running 
+
+```bash
+rustup update -- nightly
+```
+
+or by setting up `rustup default nightly` or else with `cargo +nihgtly {build,test, run}`. After updating, run 
+```bash
+cargo new hellosimd
+```
+to create a new crate. Edit `hellosimd/Cargo.toml` to be 
+```toml
+[package]
+name = "hellosimd"
+version = "0.1.0"
+edition = "2018"
+[dependencies]
+core_simd = { git = "https://github.com/rust-lang/stdsimd" }
+```
+
+and finally write this in `src/main.rs`:
+```rust
+use core_simd::*;
+fn main() {
+    let a = f32x4::splat(10.0);
+    let b = f32x4::from_array([1.0, 2.0, 3.0, 4.0]);
+    println!("{:?}", a + b);
+}
+```
+
+Explanation: We import all the bindings from the crate with the first line. Then, we construct our SIMD vectors with methods like `splat` or `from_array`. Finally, we can use operators on them like `+` and the appropriate SIMD instructions will be carried out. When we run `cargo run` you should get `[11.0, 12.0, 13.0, 14.0]`.
+
 ## Code Organization
 
 Currently the crate is organized so that each element type is a file, and then the 64-bit, 128-bit, 256-bit, and 512-bit vectors using those types are contained in said file.

From 68393aa594b0d3c5bfecd37d94fde5795de78771 Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Fri, 11 Jun 2021 18:48:05 -0400
Subject: [PATCH 170/249] Add mask width conversion (#127)

---
 crates/core_simd/src/masks/bitmask.rs    | 23 ++++++++++++++++++++
 crates/core_simd/src/masks/full_masks.rs | 27 ++++++++++++++++++++++++
 crates/core_simd/src/masks/mod.rs        | 23 ++++++++++++++++++++
 crates/core_simd/tests/masks.rs          |  9 ++++++++
 4 files changed, 82 insertions(+)

diff --git a/crates/core_simd/src/masks/bitmask.rs b/crates/core_simd/src/masks/bitmask.rs
index 6bcb08cf9dbb..31c7f6e7c289 100644
--- a/crates/core_simd/src/masks/bitmask.rs
+++ b/crates/core_simd/src/masks/bitmask.rs
@@ -169,3 +169,26 @@ pub type Mask16<T, const LANES: usize> = BitMask<T, LANES>;
 pub type Mask32<T, const LANES: usize> = BitMask<T, LANES>;
 pub type Mask64<T, const LANES: usize> = BitMask<T, LANES>;
 pub type MaskSize<T, const LANES: usize> = BitMask<T, LANES>;
+
+macro_rules! impl_from {
+    { $from:ident ($from_inner:ident) => $($to:ident ($to_inner:ident)),* } => {
+        $(
+        impl<const LANES: usize> From<$from<crate::$from<LANES>, LANES>> for $to<crate::$to<LANES>, LANES>
+        where
+            crate::$from_inner<LANES>: crate::LanesAtMost32,
+            crate::$to_inner<LANES>: crate::LanesAtMost32,
+            crate::$from<LANES>: crate::Mask,
+            crate::$to<LANES>: crate::Mask,
+        {
+            fn from(value: $from<crate::$from<LANES>, LANES>) -> Self {
+                unsafe { core::mem::transmute_copy(&value) }
+            }
+        }
+        )*
+    }
+}
+impl_from! { Mask8 (SimdI8) => Mask16 (SimdI16), Mask32 (SimdI32), Mask64 (SimdI64), MaskSize (SimdIsize) }
+impl_from! { Mask16 (SimdI16) => Mask32 (SimdI32), Mask64 (SimdI64), MaskSize (SimdIsize), Mask8 (SimdI8) }
+impl_from! { Mask32 (SimdI32) => Mask64 (SimdI64), MaskSize (SimdIsize), Mask8 (SimdI8), Mask16 (SimdI16) }
+impl_from! { Mask64 (SimdI64) => MaskSize (SimdIsize), Mask8 (SimdI8), Mask16 (SimdI16), Mask32 (SimdI32) }
+impl_from! { MaskSize (SimdIsize) => Mask8 (SimdI8), Mask16 (SimdI16), Mask32 (SimdI32), Mask64 (SimdI64) }
diff --git a/crates/core_simd/src/masks/full_masks.rs b/crates/core_simd/src/masks/full_masks.rs
index f89bbefba631..c2bfa03dfc6e 100644
--- a/crates/core_simd/src/masks/full_masks.rs
+++ b/crates/core_simd/src/masks/full_masks.rs
@@ -212,3 +212,30 @@ define_mask! {
     /// or unset.
     struct MaskSize<const LANES: usize>(crate::SimdIsize<LANES>);
 }
+
+macro_rules! impl_from {
+    { $from:ident ($from_inner:ident) => $($to:ident ($to_inner:ident)),* } => {
+        $(
+        impl<const LANES: usize, T, U> From<$from<T, LANES>> for $to<U, LANES>
+        where
+            crate::$from_inner<LANES>: crate::LanesAtMost32,
+            crate::$to_inner<LANES>: crate::LanesAtMost32,
+            T: crate::Mask,
+            U: crate::Mask,
+        {
+            fn from(value: $from<T, LANES>) -> Self {
+                let mut new = Self::splat(false);
+                for i in 0..LANES {
+                    unsafe { new.set_unchecked(i, value.test_unchecked(i)) }
+                }
+                new
+            }
+        }
+        )*
+    }
+}
+impl_from! { Mask8 (SimdI8) => Mask16 (SimdI16), Mask32 (SimdI32), Mask64 (SimdI64), MaskSize (SimdIsize) }
+impl_from! { Mask16 (SimdI16) => Mask32 (SimdI32), Mask64 (SimdI64), MaskSize (SimdIsize), Mask8 (SimdI8) }
+impl_from! { Mask32 (SimdI32) => Mask64 (SimdI64), MaskSize (SimdIsize), Mask8 (SimdI8), Mask16 (SimdI16) }
+impl_from! { Mask64 (SimdI64) => MaskSize (SimdIsize), Mask8 (SimdI8), Mask16 (SimdI16), Mask32 (SimdI32) }
+impl_from! { MaskSize (SimdIsize) => Mask8 (SimdI8), Mask16 (SimdI16), Mask32 (SimdI32), Mask64 (SimdI64) }
diff --git a/crates/core_simd/src/masks/mod.rs b/crates/core_simd/src/masks/mod.rs
index deaf2be5dca4..81a410de200e 100644
--- a/crates/core_simd/src/masks/mod.rs
+++ b/crates/core_simd/src/masks/mod.rs
@@ -544,3 +544,26 @@ pub type masksizex4 = MaskSize<4>;
 
 /// Vector of eight pointer-width masks
 pub type masksizex8 = MaskSize<8>;
+
+macro_rules! impl_from {
+    { $from:ident ($from_inner:ident) => $($to:ident ($to_inner:ident)),* } => {
+        $(
+        impl<const LANES: usize> From<$from<LANES>> for $to<LANES>
+        where
+            crate::$from_inner<LANES>: crate::LanesAtMost32,
+            crate::$to_inner<LANES>: crate::LanesAtMost32,
+            $from<LANES>: Mask,
+            Self: Mask,
+        {
+            fn from(value: $from<LANES>) -> Self {
+                Self(value.0.into())
+            }
+        }
+        )*
+    }
+}
+impl_from! { Mask8 (SimdI8) => Mask16 (SimdI16), Mask32 (SimdI32), Mask64 (SimdI64), MaskSize (SimdIsize) }
+impl_from! { Mask16 (SimdI16) => Mask32 (SimdI32), Mask64 (SimdI64), MaskSize (SimdIsize), Mask8 (SimdI8) }
+impl_from! { Mask32 (SimdI32) => Mask64 (SimdI64), MaskSize (SimdIsize), Mask8 (SimdI8), Mask16 (SimdI16) }
+impl_from! { Mask64 (SimdI64) => MaskSize (SimdIsize), Mask8 (SimdI8), Mask16 (SimdI16), Mask32 (SimdI32) }
+impl_from! { MaskSize (SimdIsize) => Mask8 (SimdI8), Mask16 (SimdI16), Mask32 (SimdI32), Mask64 (SimdI64) }
diff --git a/crates/core_simd/tests/masks.rs b/crates/core_simd/tests/masks.rs
index 7021d58aa543..5c2e60dd7c82 100644
--- a/crates/core_simd/tests/masks.rs
+++ b/crates/core_simd/tests/masks.rs
@@ -82,3 +82,12 @@ macro_rules! test_mask_api {
 mod mask_api {
     test_mask_api! { Mask8 }
 }
+
+#[test]
+fn convert() {
+    let values = [true, false, false, true, false, false, true, false];
+    assert_eq!(
+        core_simd::Mask8::from_array(values),
+        core_simd::Mask32::from_array(values).into()
+    );
+}

From 57e67c905fe9fb3d45a1714d25baa5dfc143299c Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Miguel=20Raz=20Guzm=C3=A1n=20Macedo?=
 <miguelraz@ciencias.unam.mx>
Date: Fri, 11 Jun 2021 17:48:44 -0500
Subject: [PATCH 171/249] add doctests for shuffle (#130)

* add shuffle doctests/examples
---
 crates/core_simd/src/permute.rs | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/crates/core_simd/src/permute.rs b/crates/core_simd/src/permute.rs
index dd63c69c63db..a64fdbc9dd0a 100644
--- a/crates/core_simd/src/permute.rs
+++ b/crates/core_simd/src/permute.rs
@@ -9,6 +9,15 @@ macro_rules! impl_shuffle_lane {
             /// Some SIMD shuffle instructions can be quite slow, so avoiding them by loading data
             /// into the desired patterns in advance is preferred, but shuffles are still faster
             /// than storing and reloading from memory.
+            ///
+            /// ```
+            /// # use core_simd::*;
+            // let a = f32x4::from_array([1.0, 2.0, 3.0, 4.0]);
+            // let b = f32x4::from_array([5.0, 6.0, 7.0, 8.0]);
+            // const IDXS: [u32; 4] = [4,0,3,7];
+            // let c = f32x4::shuffle::<IDXS>(a,b);
+            // assert_eq!(f32x4::from_array([5.0, 1.0, 4.0, 8.0]), c);
+            /// ```
             #[inline]
             pub fn shuffle<const IDX: [u32; $n]>(self, second: Self) -> Self {
                 unsafe { crate::intrinsics::$fn(self, second, IDX) }

From bdcccba55cddfa2c47ba322efec5dc110865edd1 Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Sun, 13 Jun 2021 17:52:44 +0000
Subject: [PATCH 172/249] Implement Sum/Product traits

---
 crates/core_simd/src/iter.rs | 52 ++++++++++++++++++++++++++++++++++++
 crates/core_simd/src/lib.rs  |  1 +
 2 files changed, 53 insertions(+)
 create mode 100644 crates/core_simd/src/iter.rs

diff --git a/crates/core_simd/src/iter.rs b/crates/core_simd/src/iter.rs
new file mode 100644
index 000000000000..c49715932b48
--- /dev/null
+++ b/crates/core_simd/src/iter.rs
@@ -0,0 +1,52 @@
+macro_rules! impl_traits {
+    { $type:ident, $scalar:ty } => {
+        impl<const LANES: usize> core::iter::Sum<Self> for crate::$type<LANES>
+        where
+            Self: crate::LanesAtMost32,
+        {
+            fn sum<I: core::iter::Iterator<Item = Self>>(iter: I) -> Self {
+                iter.fold(Default::default(), core::ops::Add::add)
+            }
+        }
+
+        impl<const LANES: usize> core::iter::Product<Self> for crate::$type<LANES>
+        where
+            Self: crate::LanesAtMost32,
+        {
+            fn product<I: core::iter::Iterator<Item = Self>>(iter: I) -> Self {
+                iter.fold(Default::default(), core::ops::Mul::mul)
+            }
+        }
+
+        impl<const LANES: usize> core::iter::Sum<crate::$type<LANES>> for $scalar
+        where
+            crate::$type<LANES>: crate::LanesAtMost32,
+        {
+            fn sum<I: core::iter::Iterator<Item = crate::$type<LANES>>>(iter: I) -> Self {
+                iter.sum::<crate::$type<LANES>>().horizontal_sum()
+            }
+        }
+
+        impl<const LANES: usize> core::iter::Product<crate::$type<LANES>> for $scalar
+        where
+            crate::$type<LANES>: crate::LanesAtMost32,
+        {
+            fn product<I: core::iter::Iterator<Item = crate::$type<LANES>>>(iter: I) -> Self {
+                iter.product::<crate::$type<LANES>>().horizontal_product()
+            }
+        }
+    }
+}
+
+impl_traits! { SimdF32, f32 }
+impl_traits! { SimdF64, f64 }
+impl_traits! { SimdU8, u8 }
+impl_traits! { SimdU16, u16 }
+impl_traits! { SimdU32, u32 }
+impl_traits! { SimdU64, u64 }
+impl_traits! { SimdUsize, usize }
+impl_traits! { SimdI8, i8 }
+impl_traits! { SimdI16, i16 }
+impl_traits! { SimdI32, i32 }
+impl_traits! { SimdI64, i64 }
+impl_traits! { SimdIsize, isize }
diff --git a/crates/core_simd/src/lib.rs b/crates/core_simd/src/lib.rs
index 7fe7d666e8d8..5b74d1d574a6 100644
--- a/crates/core_simd/src/lib.rs
+++ b/crates/core_simd/src/lib.rs
@@ -22,6 +22,7 @@ pub use to_bytes::ToBytes;
 mod comparisons;
 mod fmt;
 mod intrinsics;
+mod iter;
 mod ops;
 mod round;
 

From 96f0f5d29f1af3327805ed1d1864c639baef58c2 Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Sun, 13 Jun 2021 18:00:47 +0000
Subject: [PATCH 173/249] Implement Sum/Product over references

---
 crates/core_simd/src/iter.rs | 36 ++++++++++++++++++++++++++++++++++++
 1 file changed, 36 insertions(+)

diff --git a/crates/core_simd/src/iter.rs b/crates/core_simd/src/iter.rs
index c49715932b48..b5f28b05e5c2 100644
--- a/crates/core_simd/src/iter.rs
+++ b/crates/core_simd/src/iter.rs
@@ -35,6 +35,42 @@ macro_rules! impl_traits {
                 iter.product::<crate::$type<LANES>>().horizontal_product()
             }
         }
+
+        impl<'a, const LANES: usize> core::iter::Sum<&'a Self> for crate::$type<LANES>
+        where
+            Self: crate::LanesAtMost32,
+        {
+            fn sum<I: core::iter::Iterator<Item = &'a Self>>(iter: I) -> Self {
+                iter.fold(Default::default(), core::ops::Add::add)
+            }
+        }
+
+        impl<'a, const LANES: usize> core::iter::Product<&'a Self> for crate::$type<LANES>
+        where
+            Self: crate::LanesAtMost32,
+        {
+            fn product<I: core::iter::Iterator<Item = &'a Self>>(iter: I) -> Self {
+                iter.fold(Default::default(), core::ops::Mul::mul)
+            }
+        }
+
+        impl<'a, const LANES: usize> core::iter::Sum<&'a crate::$type<LANES>> for $scalar
+        where
+            crate::$type<LANES>: crate::LanesAtMost32,
+        {
+            fn sum<I: core::iter::Iterator<Item = &'a crate::$type<LANES>>>(iter: I) -> Self {
+                iter.sum::<crate::$type<LANES>>().horizontal_sum()
+            }
+        }
+
+        impl<'a, const LANES: usize> core::iter::Product<&'a crate::$type<LANES>> for $scalar
+        where
+            crate::$type<LANES>: crate::LanesAtMost32,
+        {
+            fn product<I: core::iter::Iterator<Item = &'a crate::$type<LANES>>>(iter: I) -> Self {
+                iter.product::<crate::$type<LANES>>().horizontal_product()
+            }
+        }
     }
 }
 

From b936f34a5c4144db9135baaa07a1c12f73b5d316 Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Sun, 13 Jun 2021 18:45:45 +0000
Subject: [PATCH 174/249] Add various special functions (recip, signum,
 copysign)

---
 crates/core_simd/src/vector/float.rs | 41 ++++++++++++++++++++++-
 crates/core_simd/src/vector/int.rs   | 14 ++++++++
 crates/core_simd/tests/ops_macros.rs | 50 ++++++++++++++++++++++++++++
 3 files changed, 104 insertions(+), 1 deletion(-)

diff --git a/crates/core_simd/src/vector/float.rs b/crates/core_simd/src/vector/float.rs
index c4565a9dd900..5044ac57ec56 100644
--- a/crates/core_simd/src/vector/float.rs
+++ b/crates/core_simd/src/vector/float.rs
@@ -4,7 +4,7 @@
 /// `$lanes` of float `$type`, which uses `$bits_ty` as its binary
 /// representation. Called from `define_float_vector!`.
 macro_rules! impl_float_vector {
-    { $name:ident, $type:ty, $bits_ty:ident, $mask_ty:ident, $mask_impl_ty:ident } => {
+    { $name:ident, $type:ident, $bits_ty:ident, $mask_ty:ident, $mask_impl_ty:ident } => {
         impl_vector! { $name, $type }
         impl_float_reductions! { $name, $type }
 
@@ -43,6 +43,25 @@ macro_rules! impl_float_vector {
             pub fn sqrt(self) -> Self {
                 unsafe { crate::intrinsics::simd_fsqrt(self) }
             }
+
+            /// Takes the reciprocal (inverse) of each lane, `1/x`.
+            #[inline]
+            pub fn recip(self) -> Self {
+                Self::splat(1.0) / self
+            }
+
+            /// Converts each lane from radians to degrees.
+            #[inline]
+            pub fn to_degrees(self) -> Self {
+                // to_degrees uses a special constant for better precision, so extract that constant
+                self * Self::splat($type::to_degrees(1.))
+            }
+
+            /// Converts each lane from degrees to radians.
+            #[inline]
+            pub fn to_radians(self) -> Self {
+                self * Self::splat(core::$type::consts::PI / 180.)
+            }
         }
 
         impl<const LANES: usize> $name<LANES>
@@ -97,6 +116,26 @@ macro_rules! impl_float_vector {
             pub fn is_normal(self) -> crate::$mask_ty<LANES> {
                 !(self.abs().lanes_eq(Self::splat(0.0)) | self.is_nan() | self.is_subnormal() | self.is_infinite())
             }
+
+            /// Replaces each lane with a number that represents its sign.
+            ///
+            /// * `1.0` if the number is positive, `+0.0`, or `INFINITY`
+            /// * `-1.0` if the number is negative, `-0.0`, or `NEG_INFINITY`
+            /// * `NAN` if the number is `NAN`
+            #[inline]
+            pub fn signum(self) -> Self {
+                self.is_nan().select(Self::splat($type::NAN), Self::splat(1.0).copysign(self))
+            }
+
+            /// Returns each lane with the magnitude of `self` and the sign of `sign`.
+            ///
+            /// If any lane is a `NAN`, then a `NAN` with the sign of `sign` is returned.
+            #[inline]
+            pub fn copysign(self, sign: Self) -> Self {
+                let sign_bit = sign.to_bits() & Self::splat(-0.).to_bits();
+                let magnitude = self.to_bits() & !Self::splat(-0.).to_bits();
+                Self::from_bits(sign_bit | magnitude)
+            }
         }
     };
 }
diff --git a/crates/core_simd/src/vector/int.rs b/crates/core_simd/src/vector/int.rs
index a535fad7bc1d..dd7b2225dbd7 100644
--- a/crates/core_simd/src/vector/int.rs
+++ b/crates/core_simd/src/vector/int.rs
@@ -33,14 +33,28 @@ macro_rules! impl_integer_vector {
             crate::$mask_ty<LANES>: crate::Mask,
         {
             /// Returns true for each positive lane and false if it is zero or negative.
+            #[inline]
             pub fn is_positive(self) -> crate::$mask_ty<LANES> {
                 self.lanes_gt(Self::splat(0))
             }
 
             /// Returns true for each negative lane and false if it is zero or positive.
+            #[inline]
             pub fn is_negative(self) -> crate::$mask_ty<LANES> {
                 self.lanes_lt(Self::splat(0))
             }
+
+            /// Returns numbers representing the sign of each lane.
+            /// * `0` if the number is zero
+            /// * `1` if the number is positive
+            /// * `-1` if the number is negative
+            #[inline]
+            pub fn signum(self) -> Self {
+                self.is_positive().select(
+                    Self::splat(1),
+                    self.is_negative().select(Self::splat(-1), Self::splat(0))
+                )
+            }
         }
     }
 }
diff --git a/crates/core_simd/tests/ops_macros.rs b/crates/core_simd/tests/ops_macros.rs
index 83c6fec69e88..9ada95e851e1 100644
--- a/crates/core_simd/tests/ops_macros.rs
+++ b/crates/core_simd/tests/ops_macros.rs
@@ -247,6 +247,15 @@ macro_rules! impl_signed_tests {
                         &|_| true,
                     );
                 }
+
+                fn signum<const LANES: usize>() {
+                    test_helpers::test_unary_elementwise(
+                        &Vector::<LANES>::signum,
+                        &Scalar::signum,
+                        &|_| true,
+                    )
+                }
+
             }
 
             test_helpers::test_lanes_panic! {
@@ -433,6 +442,47 @@ macro_rules! impl_float_tests {
                         &|_| true,
                     )
                 }
+
+                fn recip<const LANES: usize>() {
+                    test_helpers::test_unary_elementwise(
+                        &Vector::<LANES>::recip,
+                        &Scalar::recip,
+                        &|_| true,
+                    )
+                }
+
+                fn to_degrees<const LANES: usize>() {
+                    test_helpers::test_unary_elementwise(
+                        &Vector::<LANES>::to_degrees,
+                        &Scalar::to_degrees,
+                        &|_| true,
+                    )
+                }
+
+                fn to_radians<const LANES: usize>() {
+                    test_helpers::test_unary_elementwise(
+                        &Vector::<LANES>::to_radians,
+                        &Scalar::to_radians,
+                        &|_| true,
+                    )
+                }
+
+                fn signum<const LANES: usize>() {
+                    test_helpers::test_unary_elementwise(
+                        &Vector::<LANES>::signum,
+                        &Scalar::signum,
+                        &|_| true,
+                    )
+                }
+
+                fn copysign<const LANES: usize>() {
+                    test_helpers::test_binary_elementwise(
+                        &Vector::<LANES>::copysign,
+                        &Scalar::copysign,
+                        &|_, _| true,
+                    )
+                }
+
                 fn horizontal_sum<const LANES: usize>() {
                     test_helpers::test_1(&|x| {
                         test_helpers::prop_assert_biteq! (

From 74e6262ce4ad8efb8d0addd461fdf9d25bea9538 Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Sun, 13 Jun 2021 19:47:32 +0000
Subject: [PATCH 175/249] Add min/max/clamp

---
 crates/core_simd/src/vector/float.rs | 41 ++++++++++++++++
 crates/core_simd/tests/ops_macros.rs | 70 ++++++++++++++++++++++++++++
 crates/test_helpers/src/lib.rs       | 21 +++++++++
 3 files changed, 132 insertions(+)

diff --git a/crates/core_simd/src/vector/float.rs b/crates/core_simd/src/vector/float.rs
index 5044ac57ec56..7061b9b06748 100644
--- a/crates/core_simd/src/vector/float.rs
+++ b/crates/core_simd/src/vector/float.rs
@@ -136,6 +136,47 @@ macro_rules! impl_float_vector {
                 let magnitude = self.to_bits() & !Self::splat(-0.).to_bits();
                 Self::from_bits(sign_bit | magnitude)
             }
+
+            /// Returns the minimum of each lane.
+            ///
+            /// If one of the values is `NAN`, then the other value is returned.
+            #[inline]
+            pub fn min(self, other: Self) -> Self {
+                // TODO consider using an intrinsic
+                self.is_nan().select(
+                    other,
+                    self.lanes_ge(other).select(other, self)
+                )
+            }
+
+            /// Returns the maximum of each lane.
+            ///
+            /// If one of the values is `NAN`, then the other value is returned.
+            #[inline]
+            pub fn max(self, other: Self) -> Self {
+                // TODO consider using an intrinsic
+                self.is_nan().select(
+                    other,
+                    self.lanes_le(other).select(other, self)
+                )
+            }
+
+            /// Restrict each lane to a certain interval unless it is NaN.
+            /// 
+            /// For each lane in `self`, returns the corresponding lane in `max` if the lane is
+            /// greater than `max`, and the corresponding lane in `min` if the lane is less
+            /// than `min`.  Otherwise returns the lane in `self`.
+            #[inline]
+            pub fn clamp(self, min: Self, max: Self) -> Self {
+                assert!(
+                    min.lanes_le(max).all(),
+                    "each lane in `min` must be less than or equal to the corresponding lane in `max`",
+                );
+                let mut x = self;
+                x = x.lanes_lt(min).select(min, x);
+                x = x.lanes_gt(max).select(max, x);
+                x
+            }
         }
     };
 }
diff --git a/crates/core_simd/tests/ops_macros.rs b/crates/core_simd/tests/ops_macros.rs
index 9ada95e851e1..8ef2edc8370a 100644
--- a/crates/core_simd/tests/ops_macros.rs
+++ b/crates/core_simd/tests/ops_macros.rs
@@ -483,6 +483,76 @@ macro_rules! impl_float_tests {
                     )
                 }
 
+                fn min<const LANES: usize>() {
+                    // Regular conditions (both values aren't zero)
+                    test_helpers::test_binary_elementwise(
+                        &Vector::<LANES>::min,
+                        &Scalar::min,
+                        // Reject the case where both values are zero with different signs
+                        &|a, b| {
+                            for (a, b) in a.iter().zip(b.iter()) {
+                                if *a == 0. && *b == 0. && a.signum() != b.signum() {
+                                    return false;
+                                }
+                            }
+                            true
+                        }
+                    );
+
+                    // Special case where both values are zero
+                    let p_zero = Vector::<LANES>::splat(0.);
+                    let n_zero = Vector::<LANES>::splat(-0.);
+                    assert!(p_zero.min(n_zero).to_array().iter().all(|x| *x == 0.));
+                    assert!(n_zero.min(p_zero).to_array().iter().all(|x| *x == 0.));
+                }
+
+                fn max<const LANES: usize>() {
+                    // Regular conditions (both values aren't zero)
+                    test_helpers::test_binary_elementwise(
+                        &Vector::<LANES>::max,
+                        &Scalar::max,
+                        // Reject the case where both values are zero with different signs
+                        &|a, b| {
+                            for (a, b) in a.iter().zip(b.iter()) {
+                                if *a == 0. && *b == 0. && a.signum() != b.signum() {
+                                    return false;
+                                }
+                            }
+                            true
+                        }
+                    );
+
+                    // Special case where both values are zero
+                    let p_zero = Vector::<LANES>::splat(0.);
+                    let n_zero = Vector::<LANES>::splat(-0.);
+                    assert!(p_zero.min(n_zero).to_array().iter().all(|x| *x == 0.));
+                    assert!(n_zero.min(p_zero).to_array().iter().all(|x| *x == 0.));
+                }
+
+                fn clamp<const LANES: usize>() {
+                    test_helpers::test_3(&|value: [Scalar; LANES], mut min: [Scalar; LANES], mut max: [Scalar; LANES]| {
+                        for (min, max) in min.iter_mut().zip(max.iter_mut()) {
+                            if max < min {
+                                core::mem::swap(min, max);
+                            }
+                            if min.is_nan() {
+                                *min = Scalar::NEG_INFINITY;
+                            }
+                            if max.is_nan() {
+                                *max = Scalar::INFINITY;
+                            }
+                        }
+
+                        let mut result_scalar = [Scalar::default(); LANES];
+                        for i in 0..LANES {
+                            result_scalar[i] = value[i].clamp(min[i], max[i]);
+                        }
+                        let result_vector = Vector::from_array(value).clamp(min.into(), max.into()).to_array();
+                        test_helpers::prop_assert_biteq!(result_scalar, result_vector);
+                        Ok(())
+                    })
+                }
+
                 fn horizontal_sum<const LANES: usize>() {
                     test_helpers::test_1(&|x| {
                         test_helpers::prop_assert_biteq! (
diff --git a/crates/test_helpers/src/lib.rs b/crates/test_helpers/src/lib.rs
index fffd088f4da3..ff6d30a1afb7 100644
--- a/crates/test_helpers/src/lib.rs
+++ b/crates/test_helpers/src/lib.rs
@@ -97,6 +97,27 @@ pub fn test_2<A: core::fmt::Debug + DefaultStrategy, B: core::fmt::Debug + Defau
         .unwrap();
 }
 
+/// Test a function that takes two values.
+pub fn test_3<
+    A: core::fmt::Debug + DefaultStrategy,
+    B: core::fmt::Debug + DefaultStrategy,
+    C: core::fmt::Debug + DefaultStrategy,
+>(
+    f: &dyn Fn(A, B, C) -> proptest::test_runner::TestCaseResult,
+) {
+    let mut runner = proptest::test_runner::TestRunner::default();
+    runner
+        .run(
+            &(
+                A::default_strategy(),
+                B::default_strategy(),
+                C::default_strategy(),
+            ),
+            |(a, b, c)| f(a, b, c),
+        )
+        .unwrap();
+}
+
 /// Test a unary vector function against a unary scalar function, applied elementwise.
 #[inline(never)]
 pub fn test_unary_elementwise<Scalar, ScalarResult, Vector, VectorResult, const LANES: usize>(

From f102de7c8b2f59bcdc8f27dfe42a94725c91fd36 Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Sun, 13 Jun 2021 19:59:17 +0000
Subject: [PATCH 176/249] Add mul_add

---
 crates/core_simd/src/intrinsics.rs   |  3 ++
 crates/core_simd/src/vector/float.rs | 12 ++++++++
 crates/core_simd/tests/ops_macros.rs |  8 ++++++
 crates/test_helpers/src/lib.rs       | 41 ++++++++++++++++++++++++++++
 4 files changed, 64 insertions(+)

diff --git a/crates/core_simd/src/intrinsics.rs b/crates/core_simd/src/intrinsics.rs
index 7adf4c24e104..3983beb82ecf 100644
--- a/crates/core_simd/src/intrinsics.rs
+++ b/crates/core_simd/src/intrinsics.rs
@@ -49,6 +49,9 @@ extern "platform-intrinsic" {
     /// fsqrt
     pub(crate) fn simd_fsqrt<T>(x: T) -> T;
 
+    /// fma
+    pub(crate) fn simd_fma<T>(x: T, y: T, z: T) -> T;
+
     pub(crate) fn simd_eq<T, U>(x: T, y: T) -> U;
     pub(crate) fn simd_ne<T, U>(x: T, y: T) -> U;
     pub(crate) fn simd_lt<T, U>(x: T, y: T) -> U;
diff --git a/crates/core_simd/src/vector/float.rs b/crates/core_simd/src/vector/float.rs
index 7061b9b06748..4f0888f29f96 100644
--- a/crates/core_simd/src/vector/float.rs
+++ b/crates/core_simd/src/vector/float.rs
@@ -36,6 +36,18 @@ macro_rules! impl_float_vector {
                 unsafe { crate::intrinsics::simd_fabs(self) }
             }
 
+            /// Fused multiply-add.  Computes `(self * a) + b` with only one rounding error,
+            /// yielding a more accurate result than an unfused multiply-add.
+            ///
+            /// Using `mul_add` *may* be more performant than an unfused multiply-add if the target
+            /// architecture has a dedicated `fma` CPU instruction.  However, this is not always
+            /// true, and will be heavily dependent on designing algorithms with specific target
+            /// hardware in mind.
+            #[inline]
+            pub fn mul_add(self, a: Self, b: Self) -> Self {
+                unsafe { crate::intrinsics::simd_fma(self, a, b) }
+            }
+
             /// Produces a vector where every lane has the square root value
             /// of the equivalently-indexed lane in `self`
             #[inline]
diff --git a/crates/core_simd/tests/ops_macros.rs b/crates/core_simd/tests/ops_macros.rs
index 8ef2edc8370a..4057f33d4470 100644
--- a/crates/core_simd/tests/ops_macros.rs
+++ b/crates/core_simd/tests/ops_macros.rs
@@ -435,6 +435,14 @@ macro_rules! impl_float_tests {
                     )
                 }
 
+                fn mul_add<const LANES: usize>() {
+                    test_helpers::test_ternary_elementwise(
+                        &Vector::<LANES>::mul_add,
+                        &Scalar::mul_add,
+                        &|_, _, _| true,
+                    )
+                }
+
                 fn sqrt<const LANES: usize>() {
                     test_helpers::test_unary_elementwise(
                         &Vector::<LANES>::sqrt,
diff --git a/crates/test_helpers/src/lib.rs b/crates/test_helpers/src/lib.rs
index ff6d30a1afb7..4f2380b8e5ba 100644
--- a/crates/test_helpers/src/lib.rs
+++ b/crates/test_helpers/src/lib.rs
@@ -278,6 +278,47 @@ pub fn test_binary_scalar_lhs_elementwise<
     });
 }
 
+/// Test a ternary vector function against a ternary scalar function, applied elementwise.
+#[inline(never)]
+pub fn test_ternary_elementwise<
+    Scalar1,
+    Scalar2,
+    Scalar3,
+    ScalarResult,
+    Vector1,
+    Vector2,
+    Vector3,
+    VectorResult,
+    const LANES: usize,
+>(
+    fv: &dyn Fn(Vector1, Vector2, Vector3) -> VectorResult,
+    fs: &dyn Fn(Scalar1, Scalar2, Scalar3) -> ScalarResult,
+    check: &dyn Fn([Scalar1; LANES], [Scalar2; LANES], [Scalar3; LANES]) -> bool,
+) where
+    Scalar1: Copy + Default + core::fmt::Debug + DefaultStrategy,
+    Scalar2: Copy + Default + core::fmt::Debug + DefaultStrategy,
+    Scalar3: Copy + Default + core::fmt::Debug + DefaultStrategy,
+    ScalarResult: Copy + Default + biteq::BitEq + core::fmt::Debug + DefaultStrategy,
+    Vector1: Into<[Scalar1; LANES]> + From<[Scalar1; LANES]> + Copy,
+    Vector2: Into<[Scalar2; LANES]> + From<[Scalar2; LANES]> + Copy,
+    Vector3: Into<[Scalar3; LANES]> + From<[Scalar3; LANES]> + Copy,
+    VectorResult: Into<[ScalarResult; LANES]> + From<[ScalarResult; LANES]> + Copy,
+{
+    test_3(&|x: [Scalar1; LANES], y: [Scalar2; LANES], z: [Scalar3; LANES]| {
+        proptest::prop_assume!(check(x, y, z));
+        let result_1: [ScalarResult; LANES] = fv(x.into(), y.into(), z.into()).into();
+        let result_2: [ScalarResult; LANES] = {
+            let mut result = [ScalarResult::default(); LANES];
+            for ((i1, (i2, i3)), o) in x.iter().zip(y.iter().zip(z.iter())).zip(result.iter_mut()) {
+                *o = fs(*i1, *i2, *i3);
+            }
+            result
+        };
+        crate::prop_assert_biteq!(result_1, result_2);
+        Ok(())
+    });
+}
+
 /// Expand a const-generic test into separate tests for each possible lane count.
 #[macro_export]
 macro_rules! test_lanes {

From 7b66032ed511229410b0879d384b06cfcacf0538 Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Sun, 13 Jun 2021 20:11:01 +0000
Subject: [PATCH 177/249] Fix test typo

---
 crates/core_simd/tests/ops_macros.rs | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/crates/core_simd/tests/ops_macros.rs b/crates/core_simd/tests/ops_macros.rs
index 4057f33d4470..cb39e7377054 100644
--- a/crates/core_simd/tests/ops_macros.rs
+++ b/crates/core_simd/tests/ops_macros.rs
@@ -533,8 +533,8 @@ macro_rules! impl_float_tests {
                     // Special case where both values are zero
                     let p_zero = Vector::<LANES>::splat(0.);
                     let n_zero = Vector::<LANES>::splat(-0.);
-                    assert!(p_zero.min(n_zero).to_array().iter().all(|x| *x == 0.));
-                    assert!(n_zero.min(p_zero).to_array().iter().all(|x| *x == 0.));
+                    assert!(p_zero.max(n_zero).to_array().iter().all(|x| *x == 0.));
+                    assert!(n_zero.max(p_zero).to_array().iter().all(|x| *x == 0.));
                 }
 
                 fn clamp<const LANES: usize>() {

From 15b4e280049a2b495635027ebfcb3b6930fbc12a Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Mon, 21 Jun 2021 17:05:43 -0400
Subject: [PATCH 178/249] Add from_bitmask (#136)

* Add from_bitmask
* Add mips workaround
---
 crates/core_simd/src/masks/bitmask.rs    |  9 +++++++++
 crates/core_simd/src/masks/full_masks.rs | 24 ++++++++++++++++++++++++
 crates/core_simd/src/masks/mod.rs        |  5 +++++
 crates/core_simd/tests/masks.rs          |  6 ++++--
 4 files changed, 42 insertions(+), 2 deletions(-)

diff --git a/crates/core_simd/src/masks/bitmask.rs b/crates/core_simd/src/masks/bitmask.rs
index 31c7f6e7c289..fab136d2b241 100644
--- a/crates/core_simd/src/masks/bitmask.rs
+++ b/crates/core_simd/src/masks/bitmask.rs
@@ -100,6 +100,15 @@ impl<T: Mask, const LANES: usize> BitMask<T, LANES> {
         unsafe { core::mem::transmute_copy(&self.0) }
     }
 
+    #[inline]
+    pub fn from_bitmask<U: Mask>(bitmask: U::BitMask) -> Self {
+        assert_eq!(
+            core::mem::size_of::<T::BitMask>(),
+            core::mem::size_of::<U::BitMask>()
+        );
+        unsafe { core::mem::transmute_copy(&bitmask) }
+    }
+
     #[inline]
     pub fn any(self) -> bool {
         self != Self::splat(false)
diff --git a/crates/core_simd/src/masks/full_masks.rs b/crates/core_simd/src/masks/full_masks.rs
index c2bfa03dfc6e..7d98333ef607 100644
--- a/crates/core_simd/src/masks/full_masks.rs
+++ b/crates/core_simd/src/masks/full_masks.rs
@@ -126,6 +126,30 @@ macro_rules! define_mask {
                     bitmask
                 }
             }
+
+            #[inline]
+            pub fn from_bitmask<U: crate::Mask>(mut bitmask: U::BitMask) -> Self {
+                unsafe {
+                    // There is a bug where LLVM appears to implement this operation with the wrong
+                    // bit order.
+                    // TODO fix this in a better way
+                    if cfg!(any(target_arch = "mips", target_arch = "mips64")) {
+                        for x in bitmask.as_mut() {
+                            *x = x.reverse_bits();
+                        }
+                    }
+
+                    // TODO remove the transmute when rustc is more flexible
+                    assert_eq!(core::mem::size_of::<U::IntBitMask>(), core::mem::size_of::<U::BitMask>());
+                    let bitmask: U::IntBitMask = core::mem::transmute_copy(&bitmask);
+
+                    Self::from_int_unchecked(crate::intrinsics::simd_select_bitmask(
+                        bitmask,
+                        Self::splat(true).to_int(),
+                        Self::splat(false).to_int(),
+                    ))
+                }
+            }
         }
 
         impl<T: Mask, const LANES: usize> core::convert::From<$name<T, LANES>> for crate::$type<LANES>
diff --git a/crates/core_simd/src/masks/mod.rs b/crates/core_simd/src/masks/mod.rs
index 81a410de200e..1d6b2e45224a 100644
--- a/crates/core_simd/src/masks/mod.rs
+++ b/crates/core_simd/src/masks/mod.rs
@@ -178,6 +178,11 @@ macro_rules! define_opaque_mask {
             pub fn to_bitmask(self) -> <Self as Mask>::BitMask {
                 self.0.to_bitmask::<Self>()
             }
+
+            /// Convert a bitmask to a mask.
+            pub fn from_bitmask(bitmask: <Self as Mask>::BitMask) -> Self {
+                Self(<$inner_ty>::from_bitmask::<Self>(bitmask))
+            }
         }
 
         // vector/array conversion
diff --git a/crates/core_simd/tests/masks.rs b/crates/core_simd/tests/masks.rs
index 5c2e60dd7c82..32dea49729f0 100644
--- a/crates/core_simd/tests/masks.rs
+++ b/crates/core_simd/tests/masks.rs
@@ -67,13 +67,15 @@ macro_rules! test_mask_api {
             }
 
             #[test]
-            fn to_bitmask() {
+            fn roundtrip_bitmask_conversion() {
                 let values = [
                     true, false, false, true, false, false, true, false,
                     true, true, false, false, false, false, false, true,
                 ];
                 let mask = core_simd::$name::<16>::from_array(values);
-                assert_eq!(mask.to_bitmask(), [0b01001001, 0b10000011]);
+                let bitmask = mask.to_bitmask();
+                assert_eq!(bitmask, [0b01001001, 0b10000011]);
+                assert_eq!(core_simd::$name::<16>::from_bitmask(bitmask), mask);
             }
         }
     }

From 708ae618414ca3452ebbb01d94d29105b442b1d8 Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Mon, 21 Jun 2021 21:08:26 +0000
Subject: [PATCH 179/249] Remove scalar Sum/Product over questionable order of
 operations

---
 crates/core_simd/src/iter.rs | 62 ++++++++----------------------------
 1 file changed, 13 insertions(+), 49 deletions(-)

diff --git a/crates/core_simd/src/iter.rs b/crates/core_simd/src/iter.rs
index b5f28b05e5c2..c1c4c645db6b 100644
--- a/crates/core_simd/src/iter.rs
+++ b/crates/core_simd/src/iter.rs
@@ -1,5 +1,5 @@
 macro_rules! impl_traits {
-    { $type:ident, $scalar:ty } => {
+    { $type:ident } => {
         impl<const LANES: usize> core::iter::Sum<Self> for crate::$type<LANES>
         where
             Self: crate::LanesAtMost32,
@@ -18,24 +18,6 @@ macro_rules! impl_traits {
             }
         }
 
-        impl<const LANES: usize> core::iter::Sum<crate::$type<LANES>> for $scalar
-        where
-            crate::$type<LANES>: crate::LanesAtMost32,
-        {
-            fn sum<I: core::iter::Iterator<Item = crate::$type<LANES>>>(iter: I) -> Self {
-                iter.sum::<crate::$type<LANES>>().horizontal_sum()
-            }
-        }
-
-        impl<const LANES: usize> core::iter::Product<crate::$type<LANES>> for $scalar
-        where
-            crate::$type<LANES>: crate::LanesAtMost32,
-        {
-            fn product<I: core::iter::Iterator<Item = crate::$type<LANES>>>(iter: I) -> Self {
-                iter.product::<crate::$type<LANES>>().horizontal_product()
-            }
-        }
-
         impl<'a, const LANES: usize> core::iter::Sum<&'a Self> for crate::$type<LANES>
         where
             Self: crate::LanesAtMost32,
@@ -53,36 +35,18 @@ macro_rules! impl_traits {
                 iter.fold(Default::default(), core::ops::Mul::mul)
             }
         }
-
-        impl<'a, const LANES: usize> core::iter::Sum<&'a crate::$type<LANES>> for $scalar
-        where
-            crate::$type<LANES>: crate::LanesAtMost32,
-        {
-            fn sum<I: core::iter::Iterator<Item = &'a crate::$type<LANES>>>(iter: I) -> Self {
-                iter.sum::<crate::$type<LANES>>().horizontal_sum()
-            }
-        }
-
-        impl<'a, const LANES: usize> core::iter::Product<&'a crate::$type<LANES>> for $scalar
-        where
-            crate::$type<LANES>: crate::LanesAtMost32,
-        {
-            fn product<I: core::iter::Iterator<Item = &'a crate::$type<LANES>>>(iter: I) -> Self {
-                iter.product::<crate::$type<LANES>>().horizontal_product()
-            }
-        }
     }
 }
 
-impl_traits! { SimdF32, f32 }
-impl_traits! { SimdF64, f64 }
-impl_traits! { SimdU8, u8 }
-impl_traits! { SimdU16, u16 }
-impl_traits! { SimdU32, u32 }
-impl_traits! { SimdU64, u64 }
-impl_traits! { SimdUsize, usize }
-impl_traits! { SimdI8, i8 }
-impl_traits! { SimdI16, i16 }
-impl_traits! { SimdI32, i32 }
-impl_traits! { SimdI64, i64 }
-impl_traits! { SimdIsize, isize }
+impl_traits! { SimdF32 }
+impl_traits! { SimdF64 }
+impl_traits! { SimdU8 }
+impl_traits! { SimdU16 }
+impl_traits! { SimdU32 }
+impl_traits! { SimdU64 }
+impl_traits! { SimdUsize }
+impl_traits! { SimdI8 }
+impl_traits! { SimdI16 }
+impl_traits! { SimdI32 }
+impl_traits! { SimdI64 }
+impl_traits! { SimdIsize }

From b0a9fe5d07674fc20ab3dc26f9c3c9c1302d54e8 Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Mon, 21 Jun 2021 21:11:37 +0000
Subject: [PATCH 180/249] Extract constant from scalar to_radians as well

---
 crates/core_simd/src/vector/float.rs | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/crates/core_simd/src/vector/float.rs b/crates/core_simd/src/vector/float.rs
index 4f0888f29f96..4656eb3f3794 100644
--- a/crates/core_simd/src/vector/float.rs
+++ b/crates/core_simd/src/vector/float.rs
@@ -72,7 +72,7 @@ macro_rules! impl_float_vector {
             /// Converts each lane from degrees to radians.
             #[inline]
             pub fn to_radians(self) -> Self {
-                self * Self::splat(core::$type::consts::PI / 180.)
+                self * Self::splat($type::to_radians(1.))
             }
         }
 

From 16765a10218bbdfc06375961d70477460a53fc01 Mon Sep 17 00:00:00 2001
From: Jubilee Young <workingjubilee@gmail.com>
Date: Tue, 15 Jun 2021 10:32:53 -0700
Subject: [PATCH 181/249] Introduce SimdArray trait

This provides a general framework for describing relationships
between vector types and scalar types.
---
 crates/core_simd/src/array.rs | 120 ++++++++++++++++++++++++++++++++++
 crates/core_simd/src/lib.rs   |   3 +
 2 files changed, 123 insertions(+)
 create mode 100644 crates/core_simd/src/array.rs

diff --git a/crates/core_simd/src/array.rs b/crates/core_simd/src/array.rs
new file mode 100644
index 000000000000..d2f944d1e535
--- /dev/null
+++ b/crates/core_simd/src/array.rs
@@ -0,0 +1,120 @@
+use crate::masks::*;
+use crate::vector::*;
+
+/// A representation of a vector as an "array" with indices, implementing
+/// operations applicable to any vector type based solely on "having lanes",
+/// and describing relationships between vector and scalar types.
+pub trait SimdArray<const LANES: usize>: crate::LanesAtMost32
+where
+    SimdUsize<LANES>: crate::LanesAtMost32,
+    SimdIsize<LANES>: crate::LanesAtMost32,
+    MaskSize<LANES>: crate::Mask,
+    Self: Sized,
+{
+    /// The scalar type in every lane of this vector type.
+    type Scalar: Copy + Sized;
+
+    /// Generates a SIMD vector with the same value in every lane.
+    #[must_use]
+    fn splat(val: Self::Scalar) -> Self;
+}
+
+macro_rules! impl_simdarray_for {
+    ($simd:ident {type Scalar = $scalar:ident;}) => {
+        impl<const LANES: usize> SimdArray<LANES> for $simd<LANES>
+            where SimdUsize<LANES>: crate::LanesAtMost32,
+            SimdIsize<LANES>: crate::LanesAtMost32,
+            MaskSize<LANES>: crate::Mask,
+            Self: crate::LanesAtMost32,
+        {
+            type Scalar = $scalar;
+
+            #[must_use]
+            #[inline]
+            fn splat(val: Self::Scalar) -> Self {
+                [val; LANES].into()
+            }
+        }
+    };
+
+    ($simd:ident $impl:tt) => {
+        impl<const LANES: usize> SimdArray<LANES> for $simd<LANES>
+            where SimdUsize<LANES>: crate::LanesAtMost32,
+            SimdIsize<LANES>: crate::LanesAtMost32,
+            MaskSize<LANES>: crate::Mask,
+            Self: crate::LanesAtMost32,
+        $impl
+    }
+}
+
+impl_simdarray_for! {
+    SimdUsize {
+        type Scalar = usize;
+    }
+}
+
+impl_simdarray_for! {
+    SimdIsize {
+        type Scalar = isize;
+    }
+}
+
+impl_simdarray_for! {
+    SimdI8 {
+        type Scalar = i8;
+    }
+}
+
+impl_simdarray_for! {
+    SimdI16 {
+        type Scalar = i16;
+    }
+}
+
+impl_simdarray_for! {
+    SimdI32 {
+        type Scalar = i32;
+    }
+}
+
+impl_simdarray_for! {
+    SimdI64 {
+        type Scalar = i64;
+    }
+}
+
+impl_simdarray_for! {
+    SimdU8 {
+        type Scalar = u8;
+    }
+}
+
+impl_simdarray_for! {
+    SimdU16 {
+        type Scalar = u16;
+    }
+}
+
+impl_simdarray_for! {
+    SimdU32 {
+        type Scalar = u32;
+    }
+}
+
+impl_simdarray_for! {
+    SimdU64 {
+        type Scalar = u64;
+    }
+}
+
+impl_simdarray_for! {
+    SimdF32 {
+        type Scalar = f32;
+    }
+}
+
+impl_simdarray_for! {
+    SimdF64 {
+        type Scalar = f64;
+    }
+}
diff --git a/crates/core_simd/src/lib.rs b/crates/core_simd/src/lib.rs
index 7fe7d666e8d8..1e68cf5d4284 100644
--- a/crates/core_simd/src/lib.rs
+++ b/crates/core_simd/src/lib.rs
@@ -35,3 +35,6 @@ pub use masks::*;
 
 mod vector;
 pub use vector::*;
+
+mod array;
+pub use array::SimdArray;

From 2f99cc80d841781ef84d3f83cb732d0b84db3592 Mon Sep 17 00:00:00 2001
From: Jubilee Young <workingjubilee@gmail.com>
Date: Tue, 15 Jun 2021 10:52:29 -0700
Subject: [PATCH 182/249] Add pointer vectors: SimdConstPtr, SimdMutPtr

---
 crates/core_simd/src/vector.rs     |  3 ++
 crates/core_simd/src/vector/ptr.rs | 55 ++++++++++++++++++++++++++++++
 2 files changed, 58 insertions(+)
 create mode 100644 crates/core_simd/src/vector/ptr.rs

diff --git a/crates/core_simd/src/vector.rs b/crates/core_simd/src/vector.rs
index 95534384b705..95214ea88642 100644
--- a/crates/core_simd/src/vector.rs
+++ b/crates/core_simd/src/vector.rs
@@ -5,3 +5,6 @@ mod uint;
 pub use float::*;
 pub use int::*;
 pub use uint::*;
+
+// Vectors of pointers are not for public use at the current time.
+pub(crate) mod ptr;
diff --git a/crates/core_simd/src/vector/ptr.rs b/crates/core_simd/src/vector/ptr.rs
new file mode 100644
index 000000000000..30bef038b333
--- /dev/null
+++ b/crates/core_simd/src/vector/ptr.rs
@@ -0,0 +1,55 @@
+//! Private implementation details of public gather/scatter APIs.
+use crate::SimdUsize;
+use core::mem;
+
+/// A vector of *const T.
+#[derive(Debug, Copy, Clone)]
+#[repr(simd)]
+pub(crate) struct SimdConstPtr<T, const LANES: usize>([*const T; LANES]);
+
+impl<T, const LANES: usize> SimdConstPtr<T, LANES>
+where
+    SimdUsize<LANES>: crate::LanesAtMost32,
+    T: Sized,
+{
+    #[inline]
+    #[must_use]
+    pub fn splat(ptr: *const T) -> Self {
+        Self([ptr; LANES])
+    }
+
+    #[inline]
+    #[must_use]
+    pub fn wrapping_add(self, addend: SimdUsize<LANES>) -> Self {
+        unsafe {
+            let x: SimdUsize<LANES> = mem::transmute_copy(&self);
+            mem::transmute_copy(&{ x + (addend * mem::size_of::<T>()) })
+        }
+    }
+}
+
+/// A vector of *mut T. Be very careful around potential aliasing.
+#[derive(Debug, Copy, Clone)]
+#[repr(simd)]
+pub(crate) struct SimdMutPtr<T, const LANES: usize>([*mut T; LANES]);
+
+impl<T, const LANES: usize> SimdMutPtr<T, LANES>
+where
+    SimdUsize<LANES>: crate::LanesAtMost32,
+    T: Sized,
+{
+    #[inline]
+    #[must_use]
+    pub fn splat(ptr: *mut T) -> Self {
+        Self([ptr; LANES])
+    }
+
+    #[inline]
+    #[must_use]
+    pub fn wrapping_add(self, addend: SimdUsize<LANES>) -> Self {
+        unsafe {
+            let x: SimdUsize<LANES> = mem::transmute_copy(&self);
+            mem::transmute_copy(&{ x + (addend * mem::size_of::<T>()) })
+        }
+    }
+}

From 128b6f5e22db1aad085627322f1deb7fb71b07ae Mon Sep 17 00:00:00 2001
From: Jubilee Young <workingjubilee@gmail.com>
Date: Tue, 15 Jun 2021 14:38:34 -0700
Subject: [PATCH 183/249] Add SimdArray::gather_{or,or_default,select}

---
 crates/core_simd/src/array.rs      | 66 ++++++++++++++++++++++++++++++
 crates/core_simd/src/intrinsics.rs |  4 +-
 2 files changed, 69 insertions(+), 1 deletion(-)

diff --git a/crates/core_simd/src/array.rs b/crates/core_simd/src/array.rs
index d2f944d1e535..36e1fb59675c 100644
--- a/crates/core_simd/src/array.rs
+++ b/crates/core_simd/src/array.rs
@@ -1,4 +1,6 @@
+use crate::intrinsics;
 use crate::masks::*;
+use crate::vector::ptr::SimdConstPtr;
 use crate::vector::*;
 
 /// A representation of a vector as an "array" with indices, implementing
@@ -17,6 +19,70 @@ where
     /// Generates a SIMD vector with the same value in every lane.
     #[must_use]
     fn splat(val: Self::Scalar) -> Self;
+
+    /// SIMD gather: construct a SIMD vector by reading from a slice, using potentially discontiguous indices.
+    /// If an index is out of bounds, that lane instead selects the value from the "or" vector.
+    /// ```
+    /// # use core_simd::*;
+    /// let vec: Vec<i32> = vec![10, 11, 12, 13, 14, 15, 16, 17, 18];
+    /// let idxs = SimdUsize::<4>::from_array([9, 3, 0, 5]);
+    /// let alt = SimdI32::from_array([-5, -4, -3, -2]);
+    ///
+    /// let result = SimdI32::<4>::gather_or(&vec, idxs, alt); // Note the lane that is out-of-bounds.
+    /// assert_eq!(result, SimdI32::from_array([-5, 13, 10, 15]));
+    /// ```
+    #[must_use]
+    #[inline]
+    fn gather_or(slice: &[Self::Scalar], idxs: SimdUsize<LANES>, or: Self) -> Self {
+        Self::gather_select(slice, MaskSize::splat(true), idxs, or)
+    }
+
+    /// SIMD gather: construct a SIMD vector by reading from a slice, using potentially discontiguous indices.
+    /// Out-of-bounds indices instead use the default value for that lane (0).
+    /// ```
+    /// # use core_simd::*;
+    /// let vec: Vec<i32> = vec![10, 11, 12, 13, 14, 15, 16, 17, 18];
+    /// let idxs = SimdUsize::<4>::from_array([9, 3, 0, 5]);
+    ///
+    /// let result = SimdI32::<4>::gather_or_default(&vec, idxs); // Note the lane that is out-of-bounds.
+    /// assert_eq!(result, SimdI32::from_array([0, 13, 10, 15]));
+    /// ```
+    #[must_use]
+    #[inline]
+    fn gather_or_default(slice: &[Self::Scalar], idxs: SimdUsize<LANES>) -> Self
+    where
+        Self::Scalar: Default,
+    {
+        Self::gather_or(slice, idxs, Self::splat(Self::Scalar::default()))
+    }
+
+    /// SIMD gather: construct a SIMD vector by reading from a slice, using potentially discontiguous indices.
+    /// Out-of-bounds or masked indices instead select the value from the "or" vector.
+    /// ```
+    /// # use core_simd::*;
+    /// let vec: Vec<i32> = vec![10, 11, 12, 13, 14, 15, 16, 17, 18];
+    /// let idxs = SimdUsize::<4>::from_array([9, 3, 0, 5]);
+    /// let alt = SimdI32::from_array([-5, -4, -3, -2]);
+    /// let mask = MaskSize::from_array([true, true, true, false]); // Note the mask of the last lane.
+    ///
+    /// let result = SimdI32::<4>::gather_select(&vec, mask, idxs, alt); // Note the lane that is out-of-bounds.
+    /// assert_eq!(result, SimdI32::from_array([-5, 13, 10, -2]));
+    /// ```
+    #[must_use]
+    #[inline]
+    fn gather_select(
+        slice: &[Self::Scalar],
+        mask: MaskSize<LANES>,
+        idxs: SimdUsize<LANES>,
+        or: Self,
+    ) -> Self {
+        let mask = (mask & idxs.lanes_lt(SimdUsize::splat(slice.len()))).to_int();
+        let base_ptr = SimdConstPtr::splat(slice.as_ptr());
+        // Ferris forgive me, I have done pointer arithmetic here.
+        let ptrs = base_ptr.wrapping_add(idxs);
+        // SAFETY: The ptrs have been bounds-masked to prevent memory-unsafe reads insha'allah
+        unsafe { intrinsics::simd_gather(or, ptrs, mask) }
+    }
 }
 
 macro_rules! impl_simdarray_for {
diff --git a/crates/core_simd/src/intrinsics.rs b/crates/core_simd/src/intrinsics.rs
index 7adf4c24e104..e69696e515b6 100644
--- a/crates/core_simd/src/intrinsics.rs
+++ b/crates/core_simd/src/intrinsics.rs
@@ -45,7 +45,7 @@ extern "platform-intrinsic" {
 
     /// fabs
     pub(crate) fn simd_fabs<T>(x: T) -> T;
-    
+
     /// fsqrt
     pub(crate) fn simd_fsqrt<T>(x: T) -> T;
 
@@ -63,6 +63,8 @@ extern "platform-intrinsic" {
     pub(crate) fn simd_shuffle16<T, U>(x: T, y: T, idx: [u32; 16]) -> U;
     pub(crate) fn simd_shuffle32<T, U>(x: T, y: T, idx: [u32; 32]) -> U;
 
+    pub(crate) fn simd_gather<T, U, V>(val: T, ptr: U, mask: V) -> T;
+
     // {s,u}add.sat
     pub(crate) fn simd_saturating_add<T>(x: T, y: T) -> T;
 

From 81ceda8c5b93df524fca5f2a14f93d044c0dc6da Mon Sep 17 00:00:00 2001
From: Jubilee Young <workingjubilee@gmail.com>
Date: Mon, 21 Jun 2021 18:04:39 -0700
Subject: [PATCH 184/249] Add SimdArray::scatter{,_select}

---
 crates/core_simd/src/array.rs      | 60 +++++++++++++++++++++++++++++-
 crates/core_simd/src/intrinsics.rs |  1 +
 2 files changed, 60 insertions(+), 1 deletion(-)

diff --git a/crates/core_simd/src/array.rs b/crates/core_simd/src/array.rs
index 36e1fb59675c..202a44f41184 100644
--- a/crates/core_simd/src/array.rs
+++ b/crates/core_simd/src/array.rs
@@ -1,6 +1,6 @@
 use crate::intrinsics;
 use crate::masks::*;
-use crate::vector::ptr::SimdConstPtr;
+use crate::vector::ptr::{SimdConstPtr, SimdMutPtr};
 use crate::vector::*;
 
 /// A representation of a vector as an "array" with indices, implementing
@@ -83,6 +83,64 @@ where
         // SAFETY: The ptrs have been bounds-masked to prevent memory-unsafe reads insha'allah
         unsafe { intrinsics::simd_gather(or, ptrs, mask) }
     }
+
+    /// SIMD scatter: write a SIMD vector's values into a slice, using potentially discontiguous indices.
+    /// Out-of-bounds indices are not written.
+    /// ```
+    /// # use core_simd::*;
+    /// let mut vec: Vec<i32> = vec![10, 11, 12, 13, 14, 15, 16, 17, 18];
+    /// let idxs = SimdUsize::<4>::from_array([9, 3, 0, 5]);
+    /// let vals = SimdI32::from_array([-5, -4, -3, -2]);
+    ///
+    /// vals.scatter(&mut vec, idxs);
+    /// assert_eq!(vec, vec![-3, 11, 12, -4, 14, -2, 16, 17, 18]);
+    /// ```
+    #[inline]
+    fn scatter(self, slice: &mut [Self::Scalar], idxs: SimdUsize<LANES>) {
+        self.scatter_select(slice, MaskSize::splat(true), idxs)
+    }
+
+    /// SIMD scatter: write a SIMD vector's values into a slice, using potentially discontiguous indices.
+    /// Out-of-bounds or masked indices are not written.
+    /// ```
+    /// # use core_simd::*;
+    /// let mut vec: Vec<i32> = vec![10, 11, 12, 13, 14, 15, 16, 17, 18];
+    /// let idxs = SimdUsize::<4>::from_array([9, 3, 0, 5]);
+    /// let vals = SimdI32::from_array([-5, -4, -3, -2]);
+    /// let mask = MaskSize::from_array([true, true, true, false]); // Note the mask of the last lane.
+    ///
+    /// vals.scatter_select(&mut vec, mask, idxs);
+    /// assert_eq!(vec, vec![-3, 11, 12, -4, 14, 15, 16, 17, 18]);
+    /// ```
+    #[inline]
+    fn scatter_select(
+        self,
+        slice: &mut [Self::Scalar],
+        mask: MaskSize<LANES>,
+        idxs: SimdUsize<LANES>,
+    ) {
+        // We must construct our scatter mask before we derive a pointer!
+        let mask = (mask & idxs.lanes_lt(SimdUsize::splat(slice.len()))).to_int();
+        // SAFETY: This block works with *mut T derived from &mut 'a [T],
+        // which means it is delicate in Rust's borrowing model, circa 2021:
+        // &mut 'a [T] asserts uniqueness, so deriving &'a [T] invalidates live *mut Ts!
+        // Even though this block is largely safe methods, it must be almost exactly this way
+        // to prevent invalidating the raw ptrs while they're live.
+        // Thus, entering this block requires all values to use being already ready:
+        // 0. idxs we want to write to, which are used to construct the mask.
+        // 1. mask, which depends on an initial &'a [T] and the idxs.
+        // 2. actual values to scatter (self).
+        // 3. &mut [T] which will become our base ptr.
+        unsafe {
+            // Now Entering ☢️ *mut T Zone
+            let base_ptr = SimdMutPtr::splat(slice.as_mut_ptr());
+            // Ferris forgive me, I have done pointer arithmetic here.
+            let ptrs = base_ptr.wrapping_add(idxs);
+            // The ptrs have been bounds-masked to prevent memory-unsafe writes insha'allah
+            intrinsics::simd_scatter(self, ptrs, mask)
+            // Cleared ☢️ *mut T Zone
+        }
+    }
 }
 
 macro_rules! impl_simdarray_for {
diff --git a/crates/core_simd/src/intrinsics.rs b/crates/core_simd/src/intrinsics.rs
index e69696e515b6..ee9726e0fec4 100644
--- a/crates/core_simd/src/intrinsics.rs
+++ b/crates/core_simd/src/intrinsics.rs
@@ -64,6 +64,7 @@ extern "platform-intrinsic" {
     pub(crate) fn simd_shuffle32<T, U>(x: T, y: T, idx: [u32; 32]) -> U;
 
     pub(crate) fn simd_gather<T, U, V>(val: T, ptr: U, mask: V) -> T;
+    pub(crate) fn simd_scatter<T, U, V>(val: T, ptr: U, mask: V);
 
     // {s,u}add.sat
     pub(crate) fn simd_saturating_add<T>(x: T, y: T) -> T;

From f38659a46c039e0561cb86886b173869fbc45528 Mon Sep 17 00:00:00 2001
From: Jubilee Young <workingjubilee@gmail.com>
Date: Tue, 22 Jun 2021 15:17:01 -0700
Subject: [PATCH 185/249] Add assoc const SimdArray::LANES

---
 crates/core_simd/src/array.rs | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/crates/core_simd/src/array.rs b/crates/core_simd/src/array.rs
index 202a44f41184..83fd3878a35f 100644
--- a/crates/core_simd/src/array.rs
+++ b/crates/core_simd/src/array.rs
@@ -15,6 +15,8 @@ where
 {
     /// The scalar type in every lane of this vector type.
     type Scalar: Copy + Sized;
+    /// The number of lanes for this vector.
+    const LANES: usize = LANES;
 
     /// Generates a SIMD vector with the same value in every lane.
     #[must_use]

From 1529ed43d8b029c0406d636a6d2ae84bf5346aca Mon Sep 17 00:00:00 2001
From: Jubilee Young <workingjubilee@gmail.com>
Date: Wed, 23 Jun 2021 12:13:10 -0700
Subject: [PATCH 186/249] Document and test doubled writes in scatter

---
 crates/core_simd/src/array.rs | 18 ++++++++++--------
 1 file changed, 10 insertions(+), 8 deletions(-)

diff --git a/crates/core_simd/src/array.rs b/crates/core_simd/src/array.rs
index 83fd3878a35f..0a52876e55b5 100644
--- a/crates/core_simd/src/array.rs
+++ b/crates/core_simd/src/array.rs
@@ -88,14 +88,15 @@ where
 
     /// SIMD scatter: write a SIMD vector's values into a slice, using potentially discontiguous indices.
     /// Out-of-bounds indices are not written.
+    /// `scatter` writes "in order", so if an index receives two writes, only the last is guaranteed.
     /// ```
     /// # use core_simd::*;
     /// let mut vec: Vec<i32> = vec![10, 11, 12, 13, 14, 15, 16, 17, 18];
-    /// let idxs = SimdUsize::<4>::from_array([9, 3, 0, 5]);
-    /// let vals = SimdI32::from_array([-5, -4, -3, -2]);
+    /// let idxs = SimdUsize::<4>::from_array([9, 3, 0, 0]);
+    /// let vals = SimdI32::from_array([-27, 82, -41, 124]);
     ///
-    /// vals.scatter(&mut vec, idxs);
-    /// assert_eq!(vec, vec![-3, 11, 12, -4, 14, -2, 16, 17, 18]);
+    /// vals.scatter(&mut vec, idxs); // index 0 receives two writes.
+    /// assert_eq!(vec, vec![124, 11, 12, 82, 14, 15, 16, 17, 18]);
     /// ```
     #[inline]
     fn scatter(self, slice: &mut [Self::Scalar], idxs: SimdUsize<LANES>) {
@@ -104,15 +105,16 @@ where
 
     /// SIMD scatter: write a SIMD vector's values into a slice, using potentially discontiguous indices.
     /// Out-of-bounds or masked indices are not written.
+    /// `scatter_select` writes "in order", so if an index receives two writes, only the last is guaranteed.
     /// ```
     /// # use core_simd::*;
     /// let mut vec: Vec<i32> = vec![10, 11, 12, 13, 14, 15, 16, 17, 18];
-    /// let idxs = SimdUsize::<4>::from_array([9, 3, 0, 5]);
-    /// let vals = SimdI32::from_array([-5, -4, -3, -2]);
+    /// let idxs = SimdUsize::<4>::from_array([9, 3, 0, 0]);
+    /// let vals = SimdI32::from_array([-27, 82, -41, 124]);
     /// let mask = MaskSize::from_array([true, true, true, false]); // Note the mask of the last lane.
     ///
-    /// vals.scatter_select(&mut vec, mask, idxs);
-    /// assert_eq!(vec, vec![-3, 11, 12, -4, 14, 15, 16, 17, 18]);
+    /// vals.scatter_select(&mut vec, mask, idxs); // index 0's second write is masked, thus omitted.
+    /// assert_eq!(vec, vec![-41, 11, 12, 82, 14, 15, 16, 17, 18]);
     /// ```
     #[inline]
     fn scatter_select(

From 715f9ac4e36ee303c3d464121ebb65df8f92416e Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Sat, 26 Jun 2021 17:08:40 +0000
Subject: [PATCH 187/249] Fix typo. Closes #140

---
 README.md | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/README.md b/README.md
index c6a48c467b70..ee28297668e9 100644
--- a/README.md
+++ b/README.md
@@ -20,7 +20,7 @@ Now we're gonna dip our toes into this world with a small SIMD "Hello, World!" e
 rustup update -- nightly
 ```
 
-or by setting up `rustup default nightly` or else with `cargo +nihgtly {build,test, run}`. After updating, run 
+or by setting up `rustup default nightly` or else with `cargo +nightly {build,test,run}`. After updating, run 
 ```bash
 cargo new hellosimd
 ```

From 871d588ec44ec21008df54233dc60615a76e2638 Mon Sep 17 00:00:00 2001
From: Adam Greig <adam@adamgreig.com>
Date: Mon, 19 Jul 2021 22:18:29 +0100
Subject: [PATCH 188/249] Add 32-bit SIMD types i/u16x2 and i/u8x4. (#145)

These types are useful for the "SIMD32" instructions available on ARMv6
(except M-class), ARMv7 (M-class with DSP), and ARMv8.
---
 crates/core_simd/src/vector/int.rs  | 6 ++++++
 crates/core_simd/src/vector/uint.rs | 6 ++++++
 2 files changed, 12 insertions(+)

diff --git a/crates/core_simd/src/vector/int.rs b/crates/core_simd/src/vector/int.rs
index dd7b2225dbd7..75b75d7a15c0 100644
--- a/crates/core_simd/src/vector/int.rs
+++ b/crates/core_simd/src/vector/int.rs
@@ -136,6 +136,9 @@ pub type isizex4 = SimdIsize<4>;
 /// Vector of eight `isize` values
 pub type isizex8 = SimdIsize<8>;
 
+/// Vector of two `i16` values
+pub type i16x2 = SimdI16<2>;
+
 /// Vector of four `i16` values
 pub type i16x4 = SimdI16<4>;
 
@@ -169,6 +172,9 @@ pub type i64x4 = SimdI64<4>;
 /// Vector of eight `i64` values
 pub type i64x8 = SimdI64<8>;
 
+/// Vector of four `i8` values
+pub type i8x4 = SimdI8<4>;
+
 /// Vector of eight `i8` values
 pub type i8x8 = SimdI8<8>;
 
diff --git a/crates/core_simd/src/vector/uint.rs b/crates/core_simd/src/vector/uint.rs
index db027b0941f2..e7d0fa79c70e 100644
--- a/crates/core_simd/src/vector/uint.rs
+++ b/crates/core_simd/src/vector/uint.rs
@@ -105,6 +105,9 @@ pub type usizex4 = SimdUsize<4>;
 /// Vector of eight `usize` values
 pub type usizex8 = SimdUsize<8>;
 
+/// Vector of two `u16` values
+pub type u16x2 = SimdU16<2>;
+
 /// Vector of four `u16` values
 pub type u16x4 = SimdU16<4>;
 
@@ -138,6 +141,9 @@ pub type u64x4 = SimdU64<4>;
 /// Vector of eight `u64` values
 pub type u64x8 = SimdU64<8>;
 
+/// Vector of four `u8` values
+pub type u8x4 = SimdU8<4>;
+
 /// Vector of eight `u8` values
 pub type u8x8 = SimdU8<8>;
 

From ac749a180bfd10a1a1c210f047cc679b12fcb1f1 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Miguel=20Raz=20Guzm=C3=A1n=20Macedo?=
 <miguelraz@ciencias.unam.mx>
Date: Mon, 19 Jul 2021 16:58:13 -0500
Subject: [PATCH 189/249] add matrix_inversion example (#131)

* add matrix_inversion example
---
 crates/core_simd/examples/matrix_inversion.rs | 316 ++++++++++++++++++
 1 file changed, 316 insertions(+)
 create mode 100644 crates/core_simd/examples/matrix_inversion.rs

diff --git a/crates/core_simd/examples/matrix_inversion.rs b/crates/core_simd/examples/matrix_inversion.rs
new file mode 100644
index 000000000000..001187124d84
--- /dev/null
+++ b/crates/core_simd/examples/matrix_inversion.rs
@@ -0,0 +1,316 @@
+//! 4x4 matrix inverse
+// Code ported from the `packed_simd` crate
+// Run this code with `cargo test --example matrix_inversion`
+#![feature(array_chunks)]
+use core_simd::*;
+
+// Gotta define our own 4x4 matrix since Rust doesn't ship multidim arrays yet :^)
+#[derive(Copy, Clone, Debug, PartialEq, PartialOrd)]
+pub struct Matrix4x4([[f32; 4]; 4]);
+
+#[allow(clippy::too_many_lines)]
+pub fn scalar_inv4x4(m: Matrix4x4) -> Option<Matrix4x4> {
+    let m = m.0;
+
+    let mut inv = [
+        [ // row 0:
+            // 0,0:
+            m[1][1]  * m[2][2] * m[3][3] -
+            m[1][1]  * m[2][3] * m[3][2] -
+            m[2][1]  * m[1][2]  * m[3][3] +
+            m[2][1]  * m[1][3]  * m[3][2] +
+            m[3][1] * m[1][2]  * m[2][3] -
+            m[3][1] * m[1][3]  * m[2][2],
+            // 0,1:
+           -m[0][1]  * m[2][2] * m[3][3] +
+            m[0][1]  * m[2][3] * m[3][2] +
+            m[2][1]  * m[0][2] * m[3][3] -
+            m[2][1]  * m[0][3] * m[3][2] -
+            m[3][1] * m[0][2] * m[2][3] +
+            m[3][1] * m[0][3] * m[2][2],
+            // 0,2:
+            m[0][1]  * m[1][2] * m[3][3] -
+            m[0][1]  * m[1][3] * m[3][2] -
+            m[1][1]  * m[0][2] * m[3][3] +
+            m[1][1]  * m[0][3] * m[3][2] +
+            m[3][1] * m[0][2] * m[1][3] -
+            m[3][1] * m[0][3] * m[1][2],
+            // 0,3:
+           -m[0][1] * m[1][2] * m[2][3] +
+            m[0][1] * m[1][3] * m[2][2] +
+            m[1][1] * m[0][2] * m[2][3] -
+            m[1][1] * m[0][3] * m[2][2] -
+            m[2][1] * m[0][2] * m[1][3] +
+            m[2][1] * m[0][3] * m[1][2],
+        ],
+        [ // row 1
+            // 1,0:
+           -m[1][0]  * m[2][2] * m[3][3] +
+            m[1][0]  * m[2][3] * m[3][2] +
+            m[2][0]  * m[1][2]  * m[3][3] -
+            m[2][0]  * m[1][3]  * m[3][2] -
+            m[3][0] * m[1][2]  * m[2][3] +
+            m[3][0] * m[1][3]  * m[2][2],
+            // 1,1:
+            m[0][0]  * m[2][2] * m[3][3] -
+            m[0][0]  * m[2][3] * m[3][2] -
+            m[2][0]  * m[0][2] * m[3][3] +
+            m[2][0]  * m[0][3] * m[3][2] +
+            m[3][0] * m[0][2] * m[2][3] -
+            m[3][0] * m[0][3] * m[2][2],
+            // 1,2:
+           -m[0][0]  * m[1][2] * m[3][3] +
+            m[0][0]  * m[1][3] * m[3][2] +
+            m[1][0]  * m[0][2] * m[3][3] -
+            m[1][0]  * m[0][3] * m[3][2] -
+            m[3][0] * m[0][2] * m[1][3] +
+            m[3][0] * m[0][3] * m[1][2],
+            // 1,3:
+            m[0][0] * m[1][2] * m[2][3] -
+            m[0][0] * m[1][3] * m[2][2] -
+            m[1][0] * m[0][2] * m[2][3] +
+            m[1][0] * m[0][3] * m[2][2] +
+            m[2][0] * m[0][2] * m[1][3] -
+            m[2][0] * m[0][3] * m[1][2],
+        ],
+        [ // row 2
+            // 2,0:
+            m[1][0]  * m[2][1] * m[3][3] -
+            m[1][0]  * m[2][3] * m[3][1] -
+            m[2][0]  * m[1][1] * m[3][3] +
+            m[2][0]  * m[1][3] * m[3][1] +
+            m[3][0] * m[1][1] * m[2][3] -
+            m[3][0] * m[1][3] * m[2][1],
+            // 2,1:
+           -m[0][0]  * m[2][1] * m[3][3] +
+            m[0][0]  * m[2][3] * m[3][1] +
+            m[2][0]  * m[0][1] * m[3][3] -
+            m[2][0]  * m[0][3] * m[3][1] -
+            m[3][0] * m[0][1] * m[2][3] +
+            m[3][0] * m[0][3] * m[2][1],
+            // 2,2:
+            m[0][0]  * m[1][1] * m[3][3] -
+            m[0][0]  * m[1][3] * m[3][1] -
+            m[1][0]  * m[0][1] * m[3][3] +
+            m[1][0]  * m[0][3] * m[3][1] +
+            m[3][0] * m[0][1] * m[1][3] -
+            m[3][0] * m[0][3] * m[1][1],
+            // 2,3:
+           -m[0][0] * m[1][1] * m[2][3] +
+            m[0][0] * m[1][3] * m[2][1] +
+            m[1][0] * m[0][1] * m[2][3] -
+            m[1][0] * m[0][3] * m[2][1] -
+            m[2][0] * m[0][1] * m[1][3] +
+            m[2][0] * m[0][3] * m[1][1],
+        ],
+        [ // row 3
+            // 3,0:
+           -m[1][0]  * m[2][1] * m[3][2] +
+            m[1][0]  * m[2][2] * m[3][1] +
+            m[2][0]  * m[1][1] * m[3][2] -
+            m[2][0]  * m[1][2] * m[3][1] -
+            m[3][0] * m[1][1] * m[2][2] +
+            m[3][0] * m[1][2] * m[2][1],
+            // 3,1:
+            m[0][0]  * m[2][1] * m[3][2] -
+            m[0][0]  * m[2][2] * m[3][1] -
+            m[2][0]  * m[0][1] * m[3][2] +
+            m[2][0]  * m[0][2] * m[3][1] +
+            m[3][0] * m[0][1] * m[2][2] -
+            m[3][0] * m[0][2] * m[2][1],
+            // 3,2:
+           -m[0][0]  * m[1][1] * m[3][2] +
+            m[0][0]  * m[1][2] * m[3][1] +
+            m[1][0]  * m[0][1] * m[3][2] -
+            m[1][0]  * m[0][2] * m[3][1] -
+            m[3][0] * m[0][1] * m[1][2] +
+            m[3][0] * m[0][2] * m[1][1],
+            // 3,3:
+            m[0][0] * m[1][1] * m[2][2] -
+            m[0][0] * m[1][2] * m[2][1] -
+            m[1][0] * m[0][1] * m[2][2] +
+            m[1][0] * m[0][2] * m[2][1] +
+            m[2][0] * m[0][1] * m[1][2] -
+            m[2][0] * m[0][2] * m[1][1],
+        ],
+    ];
+
+    let det = m[0][0] * inv[0][0] + m[0][1] * inv[1][0] +
+              m[0][2] * inv[2][0] + m[0][3] * inv[3][0];
+    if det == 0. { return None; }
+
+    let det_inv = 1. / det;
+
+    for row in &mut inv {
+        for elem in row.iter_mut() {
+            *elem *= det_inv;
+        }
+    }
+
+    Some(Matrix4x4(inv))
+}
+
+pub fn simd_inv4x4(m: Matrix4x4) -> Option<Matrix4x4> {
+    let m = m.0;
+    let m_0 = f32x4::from_array(m[0]);
+    let m_1 = f32x4::from_array(m[1]);
+    let m_2 = f32x4::from_array(m[2]);
+    let m_3 = f32x4::from_array(m[3]);
+
+    // 2 argument shuffle, returns an f32x4
+    // the first f32x4 is indexes 0..=3
+    // the second f32x4 is indexed 4..=7
+    let tmp1 = f32x4::shuffle::<{[0, 1, 4, 5]}>(m_0, m_1);
+    let row1 = f32x4::shuffle::<{[0, 1, 4, 5]}>(m_2, m_3,);
+
+    let row0 = f32x4::shuffle::<{[0, 2, 4, 6]}>(tmp1, row1);
+    let row1 = f32x4::shuffle::<{[1, 3, 5, 7]}>(row1, tmp1);
+
+    let tmp1 = f32x4::shuffle::<{[2, 3, 6, 7]}>(m_0, m_1);
+    let row3 = f32x4::shuffle::<{[2, 3, 6, 7]}>(m_2, m_3);
+    let row2 = f32x4::shuffle::<{[0, 2, 4, 6]}>(tmp1, row3);
+    let row3 = f32x4::shuffle::<{[1, 3, 5, 7]}>(row3, tmp1);
+
+    let tmp1 = row2 * row3;
+    // there's no syntax for a 1 arg shuffle yet,
+    // so we just pass the same f32x4 twice
+    let tmp1 = f32x4::shuffle::<{[1, 0, 3, 2]}>(tmp1, tmp1);
+
+    let minor0 = row1 * tmp1;
+    let minor1 = row0 * tmp1;
+    let tmp1   = f32x4::shuffle::<{[2, 3, 0, 1]}>(tmp1, tmp1);
+    let minor0 = (row1 * tmp1) - minor0;
+    let minor1 = (row0 * tmp1) - minor1;
+    let minor1 = f32x4::shuffle::<{[2, 3, 0, 1]}>(minor1, minor1);
+
+    let tmp1 = row1 * row2;
+    let tmp1 = f32x4::shuffle::<{[1, 0, 3, 2]}>(tmp1, tmp1);
+    let minor0 = (row3 * tmp1) + minor0;
+    let minor3 = row0 * tmp1;
+    let tmp1 = f32x4::shuffle::<{[2, 3, 0, 1]}>(tmp1, tmp1);
+
+    let minor0 = minor0 - row3 * tmp1;
+    let minor3 = row0 * tmp1 - minor3;
+    let minor3 = f32x4::shuffle::<{[2, 3, 0, 1]}>(minor3, minor3);
+
+    let tmp1 = row3 * f32x4::shuffle::<{[2, 3, 0, 1]}>(row1, row1);
+    let tmp1 = f32x4::shuffle::<{[1, 0, 3, 2]}>(tmp1, tmp1);
+    let row2 = f32x4::shuffle::<{[2, 3, 0, 1]}>(row2, row2);
+    let minor0 = row2 * tmp1 + minor0;
+    let minor2 = row0 * tmp1;
+    let tmp1 = f32x4::shuffle::<{[2, 3, 0, 1]}>(tmp1, tmp1);
+    let minor0 = minor0 - row2 * tmp1;
+    let minor2 = row0 * tmp1 - minor2;
+    let minor2 = f32x4::shuffle::<{[2, 3, 0, 1]}>(minor2, minor2);
+
+    let tmp1 = row0 * row1;
+    let tmp1 = f32x4::shuffle::<{[1, 0, 3, 2]}>(tmp1, tmp1);
+    let minor2 = minor2 + row3 * tmp1;
+    let minor3 = row2 * tmp1 - minor3;
+    let tmp1 = f32x4::shuffle::<{[2, 3, 0, 1]}>(tmp1, tmp1);
+    let minor2 = row3 * tmp1 - minor2;
+    let minor3 = minor3 - row2 * tmp1;
+
+    let tmp1 = row0 * row3;
+    let tmp1 = f32x4::shuffle::<{[1, 0, 3, 2]}>(tmp1, tmp1);
+    let minor1 = minor1 - row2 * tmp1;
+    let minor2 = row1 * tmp1 + minor2;
+    let tmp1 = f32x4::shuffle::<{[2, 3, 0, 1]}>(tmp1, tmp1);
+    let minor1 = row2 * tmp1 + minor1;
+    let minor2 = minor2 - row1 * tmp1;
+
+    let tmp1 = row0 * row2;
+    let tmp1 = f32x4::shuffle::<{[1, 0, 3, 2]}>(tmp1, tmp1);
+    let minor1 = row3 * tmp1 + minor1;
+    let minor3 = minor3 - row1 * tmp1;
+    let tmp1 = f32x4::shuffle::<{[2, 3, 0, 1]}>(tmp1, tmp1);
+    let minor1 = minor1 - row3 * tmp1;
+    let minor3 = row1 * tmp1 + minor3;
+
+    let det = row0 * minor0;
+    let det = f32x4::shuffle::<{[2, 3, 0, 1]}>(det, det) + det;
+    let det = f32x4::shuffle::<{[1, 0, 3, 2]}>(det, det) + det;
+
+    if det.horizontal_sum() == 0. {
+        return None;
+    }
+    // calculate the reciprocal
+    let tmp1 = f32x4::splat(1.0) / det;
+    let det = tmp1 + tmp1 - det * tmp1 * tmp1;
+
+    let res0 = minor0 * det;
+    let res1 = minor1 * det;
+    let res2 = minor2 * det;
+    let res3 = minor3 * det;
+
+    let mut m = m;
+
+    m[0] = res0.to_array();
+    m[1] = res1.to_array();
+    m[2] = res2.to_array();
+    m[3] = res3.to_array();
+
+    Some(Matrix4x4(m))
+}
+
+
+#[cfg(test)]
+#[rustfmt::skip]
+mod tests {
+    use super::*;
+
+    #[test]
+    fn test() {
+    let tests: &[(Matrix4x4, Option<Matrix4x4>)] = &[
+        // Identity:
+        (Matrix4x4([
+            [1., 0., 0., 0.],
+            [0., 1., 0., 0.],
+            [0., 0., 1., 0.],
+            [0., 0., 0., 1.],
+         ]),
+         Some(Matrix4x4([
+             [1., 0., 0., 0.],
+             [0., 1., 0., 0.],
+             [0., 0., 1., 0.],
+             [0., 0., 0., 1.],
+         ]))
+        ),
+        // None:
+        (Matrix4x4([
+            [1., 2., 3., 4.],
+            [12., 11., 10., 9.],
+            [5., 6., 7., 8.],
+            [16., 15., 14., 13.],
+        ]),
+         None
+        ),
+        // Other:
+        (Matrix4x4([
+            [1., 1., 1., 0.],
+            [0., 3., 1., 2.],
+            [2., 3., 1., 0.],
+            [1., 0., 2., 1.],
+        ]),
+         Some(Matrix4x4([
+             [-3., -0.5,   1.5,  1.0],
+             [ 1., 0.25, -0.25, -0.5],
+             [ 3., 0.25, -1.25, -0.5],
+             [-3., 0.0,    1.0,  1.0],
+         ]))
+        ),
+
+
+    ];
+
+        for &(input, output) in tests {
+            assert_eq!(scalar_inv4x4(input), output);
+            assert_eq!(simd_inv4x4(input), output);
+        }
+    }
+}
+
+
+fn main() {
+    // Empty main to make cargo happy
+}

From 3954b27787ad3c3d51b45a89503d0f96adb52eb8 Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Mon, 19 Jul 2021 18:01:28 -0400
Subject: [PATCH 190/249] Add conversions between vendor intrinsics (#144)

* Add x86 vendor conversions
* Add wasm32 vendor types
* Add arm vendor types
* Add powerpc vendor types
---
 crates/core_simd/src/lib.rs            |  5 +-
 crates/core_simd/src/transmute.rs      | 28 ------------
 crates/core_simd/src/vector/float.rs   |  8 ----
 crates/core_simd/src/vector/int.rs     | 28 ------------
 crates/core_simd/src/vector/uint.rs    | 28 ------------
 crates/core_simd/src/vendor.rs         | 27 +++++++++++
 crates/core_simd/src/vendor/arm.rs     | 53 ++++++++++++++++++++++
 crates/core_simd/src/vendor/powerpc.rs | 11 +++++
 crates/core_simd/src/vendor/wasm32.rs  | 30 ++++++++++++
 crates/core_simd/src/vendor/x86.rs     | 63 ++++++++++++++++++++++++++
 10 files changed, 186 insertions(+), 95 deletions(-)
 create mode 100644 crates/core_simd/src/vendor.rs
 create mode 100644 crates/core_simd/src/vendor/arm.rs
 create mode 100644 crates/core_simd/src/vendor/powerpc.rs
 create mode 100644 crates/core_simd/src/vendor/wasm32.rs
 create mode 100644 crates/core_simd/src/vendor/x86.rs

diff --git a/crates/core_simd/src/lib.rs b/crates/core_simd/src/lib.rs
index 8d963e86d12e..235733b34909 100644
--- a/crates/core_simd/src/lib.rs
+++ b/crates/core_simd/src/lib.rs
@@ -1,6 +1,6 @@
 #![no_std]
 #![allow(incomplete_features)]
-#![feature(repr_simd, platform_intrinsics, simd_ffi, const_generics)]
+#![feature(repr_simd, platform_intrinsics, simd_ffi, const_generics, stdsimd)]
 #![warn(missing_docs)]
 //! Portable SIMD module.
 
@@ -9,8 +9,6 @@ mod first;
 #[macro_use]
 mod permute;
 #[macro_use]
-mod transmute;
-#[macro_use]
 mod reduction;
 
 mod select;
@@ -25,6 +23,7 @@ mod intrinsics;
 mod iter;
 mod ops;
 mod round;
+mod vendor;
 
 mod math;
 
diff --git a/crates/core_simd/src/transmute.rs b/crates/core_simd/src/transmute.rs
index b7072b334fdd..e69de29bb2d1 100644
--- a/crates/core_simd/src/transmute.rs
+++ b/crates/core_simd/src/transmute.rs
@@ -1,28 +0,0 @@
-/// Provides implementations of `From<$a> for $b` and `From<$b> for $a` that transmutes the value.
-#[cfg(any(target_arch = "x86", target_arch = "x86_64"))]
-macro_rules! from_transmute {
-    { unsafe $a:ty => $b:ty } => {
-        from_transmute!{ @impl $a => $b }
-        from_transmute!{ @impl $b => $a }
-    };
-    { @impl $from:ty => $to:ty } => {
-        impl core::convert::From<$from> for $to {
-            #[inline]
-            fn from(value: $from) -> $to {
-                unsafe { core::mem::transmute(value) }
-            }
-        }
-    };
-}
-
-/// Provides implementations of `From<$generic> for core::arch::{x86, x86_64}::$intel` and
-/// vice-versa that transmutes the value.
-macro_rules! from_transmute_x86 {
-    { unsafe $generic:ty => $intel:ident } => {
-        #[cfg(target_arch = "x86")]
-        from_transmute! { unsafe $generic => core::arch::x86::$intel }
-
-        #[cfg(target_arch = "x86_64")]
-        from_transmute! { unsafe $generic => core::arch::x86_64::$intel }
-    }
-}
diff --git a/crates/core_simd/src/vector/float.rs b/crates/core_simd/src/vector/float.rs
index 4656eb3f3794..91087740c450 100644
--- a/crates/core_simd/src/vector/float.rs
+++ b/crates/core_simd/src/vector/float.rs
@@ -201,10 +201,6 @@ where
 
 impl_float_vector! { SimdF32, f32, SimdU32, Mask32, SimdI32 }
 
-from_transmute_x86! { unsafe f32x4 => __m128 }
-from_transmute_x86! { unsafe f32x8 => __m256 }
-//from_transmute_x86! { unsafe f32x16 => __m512 }
-
 /// A SIMD vector of containing `LANES` `f64` values.
 #[repr(simd)]
 pub struct SimdF64<const LANES: usize>([f64; LANES])
@@ -213,10 +209,6 @@ where
 
 impl_float_vector! { SimdF64, f64, SimdU64, Mask64, SimdI64 }
 
-from_transmute_x86! { unsafe f64x2 => __m128d }
-from_transmute_x86! { unsafe f64x4 => __m256d }
-//from_transmute_x86! { unsafe f64x8 => __m512d }
-
 /// Vector of two `f32` values
 pub type f32x2 = SimdF32<2>;
 
diff --git a/crates/core_simd/src/vector/int.rs b/crates/core_simd/src/vector/int.rs
index 75b75d7a15c0..15ad1a7193af 100644
--- a/crates/core_simd/src/vector/int.rs
+++ b/crates/core_simd/src/vector/int.rs
@@ -67,18 +67,6 @@ where
 
 impl_integer_vector! { SimdIsize, isize, MaskSize, SimdIsize }
 
-#[cfg(target_pointer_width = "32")]
-from_transmute_x86! { unsafe isizex4 => __m128i }
-#[cfg(target_pointer_width = "32")]
-from_transmute_x86! { unsafe isizex8 => __m256i }
-
-#[cfg(target_pointer_width = "64")]
-from_transmute_x86! { unsafe isizex2 => __m128i }
-#[cfg(target_pointer_width = "64")]
-from_transmute_x86! { unsafe isizex4 => __m256i }
-//#[cfg(target_pointer_width = "64")]
-//from_transmute_x86! { unsafe isizex8 => __m512i }
-
 /// A SIMD vector of containing `LANES` `i16` values.
 #[repr(simd)]
 pub struct SimdI16<const LANES: usize>([i16; LANES])
@@ -87,10 +75,6 @@ where
 
 impl_integer_vector! { SimdI16, i16, Mask16, SimdI16 }
 
-from_transmute_x86! { unsafe i16x8 => __m128i }
-from_transmute_x86! { unsafe i16x16 => __m256i }
-//from_transmute_x86! { unsafe i16x32 => __m512i }
-
 /// A SIMD vector of containing `LANES` `i32` values.
 #[repr(simd)]
 pub struct SimdI32<const LANES: usize>([i32; LANES])
@@ -99,10 +83,6 @@ where
 
 impl_integer_vector! { SimdI32, i32, Mask32, SimdI32 }
 
-from_transmute_x86! { unsafe i32x4 => __m128i }
-from_transmute_x86! { unsafe i32x8 => __m256i }
-//from_transmute_x86! { unsafe i32x16 => __m512i }
-
 /// A SIMD vector of containing `LANES` `i64` values.
 #[repr(simd)]
 pub struct SimdI64<const LANES: usize>([i64; LANES])
@@ -111,10 +91,6 @@ where
 
 impl_integer_vector! { SimdI64, i64, Mask64, SimdI64 }
 
-from_transmute_x86! { unsafe i64x2 => __m128i }
-from_transmute_x86! { unsafe i64x4 => __m256i }
-//from_transmute_x86! { unsafe i64x8 => __m512i }
-
 /// A SIMD vector of containing `LANES` `i8` values.
 #[repr(simd)]
 pub struct SimdI8<const LANES: usize>([i8; LANES])
@@ -123,10 +99,6 @@ where
 
 impl_integer_vector! { SimdI8, i8, Mask8, SimdI8 }
 
-from_transmute_x86! { unsafe i8x16 => __m128i }
-from_transmute_x86! { unsafe i8x32 => __m256i }
-//from_transmute_x86! { unsafe i8x64 => __m512i }
-
 /// Vector of two `isize` values
 pub type isizex2 = SimdIsize<2>;
 
diff --git a/crates/core_simd/src/vector/uint.rs b/crates/core_simd/src/vector/uint.rs
index e7d0fa79c70e..0429410ed6d6 100644
--- a/crates/core_simd/src/vector/uint.rs
+++ b/crates/core_simd/src/vector/uint.rs
@@ -36,18 +36,6 @@ where
 
 impl_unsigned_vector! { SimdUsize, usize }
 
-#[cfg(target_pointer_width = "32")]
-from_transmute_x86! { unsafe usizex4 => __m128i }
-#[cfg(target_pointer_width = "32")]
-from_transmute_x86! { unsafe usizex8 => __m256i }
-
-#[cfg(target_pointer_width = "64")]
-from_transmute_x86! { unsafe usizex2 => __m128i }
-#[cfg(target_pointer_width = "64")]
-from_transmute_x86! { unsafe usizex4 => __m256i }
-//#[cfg(target_pointer_width = "64")]
-//from_transmute_x86! { unsafe usizex8 => __m512i }
-
 /// A SIMD vector of containing `LANES` `u16` values.
 #[repr(simd)]
 pub struct SimdU16<const LANES: usize>([u16; LANES])
@@ -56,10 +44,6 @@ where
 
 impl_unsigned_vector! { SimdU16, u16 }
 
-from_transmute_x86! { unsafe u16x8 => __m128i }
-from_transmute_x86! { unsafe u16x16 => __m256i }
-//from_transmute_x86! { unsafe u16x32 => __m512i }
-
 /// A SIMD vector of containing `LANES` `u32` values.
 #[repr(simd)]
 pub struct SimdU32<const LANES: usize>([u32; LANES])
@@ -68,10 +52,6 @@ where
 
 impl_unsigned_vector! { SimdU32, u32 }
 
-from_transmute_x86! { unsafe u32x4 => __m128i }
-from_transmute_x86! { unsafe u32x8 => __m256i }
-//from_transmute_x86! { unsafe u32x16 => __m512i }
-
 /// A SIMD vector of containing `LANES` `u64` values.
 #[repr(simd)]
 pub struct SimdU64<const LANES: usize>([u64; LANES])
@@ -80,10 +60,6 @@ where
 
 impl_unsigned_vector! { SimdU64, u64 }
 
-from_transmute_x86! { unsafe u64x2 => __m128i }
-from_transmute_x86! { unsafe u64x4 => __m256i }
-//from_transmute_x86! { unsafe u64x8 => __m512i }
-
 /// A SIMD vector of containing `LANES` `u8` values.
 #[repr(simd)]
 pub struct SimdU8<const LANES: usize>([u8; LANES])
@@ -92,10 +68,6 @@ where
 
 impl_unsigned_vector! { SimdU8, u8 }
 
-from_transmute_x86! { unsafe u8x16 => __m128i }
-from_transmute_x86! { unsafe u8x32 => __m256i }
-//from_transmute_x86! { unsafe u8x64 => __m512i }
-
 /// Vector of two `usize` values
 pub type usizex2 = SimdUsize<2>;
 
diff --git a/crates/core_simd/src/vendor.rs b/crates/core_simd/src/vendor.rs
new file mode 100644
index 000000000000..8c8af43bf133
--- /dev/null
+++ b/crates/core_simd/src/vendor.rs
@@ -0,0 +1,27 @@
+/// Provides implementations of `From<$a> for $b` and `From<$b> for $a` that transmutes the value.
+macro_rules! from_transmute {
+    { unsafe $a:ty => $b:ty } => {
+        from_transmute!{ @impl $a => $b }
+        from_transmute!{ @impl $b => $a }
+    };
+    { @impl $from:ty => $to:ty } => {
+        impl core::convert::From<$from> for $to {
+            #[inline]
+            fn from(value: $from) -> $to {
+                unsafe { core::mem::transmute(value) }
+            }
+        }
+    };
+}
+
+#[cfg(any(target_arch = "x86", target_arch = "x86_64"))]
+mod x86;
+
+#[cfg(any(target_arch = "wasm32"))]
+mod wasm32;
+
+#[cfg(any(target_arch = "arm", target_arch = "aarch64"))]
+mod arm;
+
+#[cfg(any(target_arch = "powerpc", target_arch = "powerpc64"))]
+mod powerpc;
diff --git a/crates/core_simd/src/vendor/arm.rs b/crates/core_simd/src/vendor/arm.rs
new file mode 100644
index 000000000000..1a1e9bed1e1d
--- /dev/null
+++ b/crates/core_simd/src/vendor/arm.rs
@@ -0,0 +1,53 @@
+use crate::*;
+
+#[cfg(target_arch = "arm")]
+use core::arch::arm::*;
+
+#[cfg(target_arch = "aarch64")]
+use core::arch::aarch64::*;
+
+from_transmute! { unsafe f32x2 => float32x2_t }
+from_transmute! { unsafe f32x4 => float32x4_t }
+
+from_transmute! { unsafe u8x8 => uint8x8_t }
+from_transmute! { unsafe u8x16 => uint8x16_t }
+from_transmute! { unsafe i8x8 => int8x8_t }
+from_transmute! { unsafe i8x16 => int8x16_t }
+from_transmute! { unsafe u8x8 => poly8x8_t }
+from_transmute! { unsafe u8x16 => poly8x16_t }
+
+from_transmute! { unsafe u16x4 => uint16x4_t }
+from_transmute! { unsafe u16x8 => uint16x8_t }
+from_transmute! { unsafe i16x4 => int16x4_t }
+from_transmute! { unsafe i16x8 => int16x8_t }
+from_transmute! { unsafe u16x4 => poly16x4_t }
+from_transmute! { unsafe u16x8 => poly16x8_t }
+
+from_transmute! { unsafe u32x2 => uint32x2_t }
+from_transmute! { unsafe u32x4 => uint32x4_t }
+from_transmute! { unsafe i32x2 => int32x2_t }
+from_transmute! { unsafe i32x4 => int32x4_t }
+
+from_transmute! { unsafe SimdU64<1> => uint64x1_t }
+from_transmute! { unsafe u64x2 => uint64x2_t }
+from_transmute! { unsafe SimdI64<1> => int64x1_t }
+from_transmute! { unsafe i64x2 => int64x2_t }
+from_transmute! { unsafe SimdU64<1> => poly64x1_t }
+from_transmute! { unsafe u64x2 => poly64x2_t }
+
+#[cfg(target_arch = "arm")]
+mod arm {
+    use super::*;
+    from_transmute! { unsafe SimdU8<4> => uint8x4_t }
+    from_transmute! { unsafe SimdI8<4> => int8x4_t }
+
+    from_transmute! { unsafe SimdU16<2> => uint16x2_t }
+    from_transmute! { unsafe SimdI16<2> => int16x2_t }
+}
+
+#[cfg(target_arch = "aarch64")]
+mod aarch64 {
+    use super::*;
+    from_transmute! { unsafe SimdF64<1> => float64x1_t }
+    from_transmute! { unsafe f64x2 => float64x2_t }
+}
diff --git a/crates/core_simd/src/vendor/powerpc.rs b/crates/core_simd/src/vendor/powerpc.rs
new file mode 100644
index 000000000000..248764efd516
--- /dev/null
+++ b/crates/core_simd/src/vendor/powerpc.rs
@@ -0,0 +1,11 @@
+use crate::*;
+
+#[cfg(target_arch = "powerpc")]
+use core::arch::powerpc::*;
+
+#[cfg(target_arch = "powerpc64")]
+use core::arch::powerpc64::*;
+
+from_transmute! { unsafe f64x2 => vector_double }
+from_transmute! { unsafe i64x2 => vector_signed_long }
+from_transmute! { unsafe u64x2 => vector_unsigned_long }
diff --git a/crates/core_simd/src/vendor/wasm32.rs b/crates/core_simd/src/vendor/wasm32.rs
new file mode 100644
index 000000000000..ef67572b534d
--- /dev/null
+++ b/crates/core_simd/src/vendor/wasm32.rs
@@ -0,0 +1,30 @@
+use crate::*;
+use core::arch::wasm32::v128;
+
+from_transmute! { unsafe u8x16 => v128 }
+from_transmute! { unsafe i8x16 => v128 }
+
+from_transmute! { unsafe u16x8 => v128 }
+from_transmute! { unsafe i16x8 => v128 }
+
+from_transmute! { unsafe u32x4 => v128 }
+from_transmute! { unsafe i32x4 => v128 }
+from_transmute! { unsafe f32x4 => v128 }
+
+from_transmute! { unsafe u64x2 => v128 }
+from_transmute! { unsafe i64x2 => v128 }
+from_transmute! { unsafe f64x2 => v128 }
+
+#[cfg(target_pointer_width = "32")]
+mod p32 {
+    use super::*;
+    from_transmute! { unsafe usizex4 => v128 }
+    from_transmute! { unsafe isizex4 => v128 }
+}
+
+#[cfg(target_pointer_width = "64")]
+mod p64 {
+    use super::*;
+    from_transmute! { unsafe usizex2 => v128 }
+    from_transmute! { unsafe isizex2 => v128 }
+}
diff --git a/crates/core_simd/src/vendor/x86.rs b/crates/core_simd/src/vendor/x86.rs
new file mode 100644
index 000000000000..4de57de057e5
--- /dev/null
+++ b/crates/core_simd/src/vendor/x86.rs
@@ -0,0 +1,63 @@
+use crate::*;
+
+#[cfg(any(target_arch = "x86"))]
+use core::arch::x86::*;
+
+#[cfg(target_arch = "x86_64")]
+use core::arch::x86_64::*;
+
+from_transmute! { unsafe u8x16 => __m128i }
+from_transmute! { unsafe u8x32 => __m256i }
+//from_transmute! { unsafe u8x64 => __m512i }
+from_transmute! { unsafe i8x16 => __m128i }
+from_transmute! { unsafe i8x32 => __m256i }
+//from_transmute! { unsafe i8x64 => __m512i }
+
+from_transmute! { unsafe u16x8 => __m128i }
+from_transmute! { unsafe u16x16 => __m256i }
+from_transmute! { unsafe u16x32 => __m512i }
+from_transmute! { unsafe i16x8 => __m128i }
+from_transmute! { unsafe i16x16 => __m256i }
+from_transmute! { unsafe i16x32 => __m512i }
+
+from_transmute! { unsafe u32x4 => __m128i }
+from_transmute! { unsafe u32x8 => __m256i }
+from_transmute! { unsafe u32x16 => __m512i }
+from_transmute! { unsafe i32x4 => __m128i }
+from_transmute! { unsafe i32x8 => __m256i }
+from_transmute! { unsafe i32x16 => __m512i }
+from_transmute! { unsafe f32x4 => __m128 }
+from_transmute! { unsafe f32x8 => __m256 }
+from_transmute! { unsafe f32x16 => __m512 }
+
+from_transmute! { unsafe u64x2 => __m128i }
+from_transmute! { unsafe u64x4 => __m256i }
+from_transmute! { unsafe u64x8 => __m512i }
+from_transmute! { unsafe i64x2 => __m128i }
+from_transmute! { unsafe i64x4 => __m256i }
+from_transmute! { unsafe i64x8 => __m512i }
+from_transmute! { unsafe f64x2 => __m128d }
+from_transmute! { unsafe f64x4 => __m256d }
+from_transmute! { unsafe f64x8 => __m512d }
+
+#[cfg(target_pointer_width = "32")]
+mod p32 {
+    use super::*;
+    from_transmute! { unsafe usizex4 => __m128i }
+    from_transmute! { unsafe usizex8 => __m256i }
+    from_transmute! { unsafe SimdUsize<16> => __m512i }
+    from_transmute! { unsafe isizex4 => __m128i }
+    from_transmute! { unsafe isizex8 => __m256i }
+    from_transmute! { unsafe SimdIsize<16> => __m512i }
+}
+
+#[cfg(target_pointer_width = "64")]
+mod p64 {
+    use super::*;
+    from_transmute! { unsafe usizex2 => __m128i }
+    from_transmute! { unsafe usizex4 => __m256i }
+    from_transmute! { unsafe usizex8 => __m512i }
+    from_transmute! { unsafe isizex2 => __m128i }
+    from_transmute! { unsafe isizex4 => __m256i }
+    from_transmute! { unsafe isizex8 => __m512i }
+}

From be96995d8ddec03fac9a0caf4d4c51c7fbc33507 Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Mon, 19 Jul 2021 19:13:24 -0400
Subject: [PATCH 191/249] Add portable_simd unstable feature gate (#141)

---
 crates/core_simd/examples/matrix_inversion.rs |  5 ++++-
 crates/core_simd/examples/nbody.rs            |  2 ++
 crates/core_simd/src/array.rs                 |  5 +++++
 crates/core_simd/src/lib.rs                   | 10 +++++++++-
 crates/core_simd/src/math.rs                  |  7 +++++++
 crates/core_simd/src/permute.rs               | 13 ++++++++-----
 crates/core_simd/src/select.rs                |  2 ++
 crates/core_simd/tests/f32_ops.rs             |  2 ++
 crates/core_simd/tests/f64_ops.rs             |  2 ++
 crates/core_simd/tests/i16_ops.rs             |  2 ++
 crates/core_simd/tests/i32_ops.rs             |  2 ++
 crates/core_simd/tests/i64_ops.rs             |  2 ++
 crates/core_simd/tests/i8_ops.rs              |  2 ++
 crates/core_simd/tests/isize_ops.rs           |  2 ++
 crates/core_simd/tests/mask_ops.rs            |  2 ++
 crates/core_simd/tests/masks.rs               |  2 ++
 crates/core_simd/tests/permute.rs             |  2 ++
 crates/core_simd/tests/round.rs               |  2 ++
 crates/core_simd/tests/to_bytes.rs            |  2 ++
 crates/core_simd/tests/u16_ops.rs             |  2 ++
 crates/core_simd/tests/u32_ops.rs             |  2 ++
 crates/core_simd/tests/u64_ops.rs             |  2 ++
 crates/core_simd/tests/u8_ops.rs              |  2 ++
 crates/core_simd/tests/usize_ops.rs           |  2 ++
 24 files changed, 71 insertions(+), 7 deletions(-)

diff --git a/crates/core_simd/examples/matrix_inversion.rs b/crates/core_simd/examples/matrix_inversion.rs
index 001187124d84..5c2d4390ad08 100644
--- a/crates/core_simd/examples/matrix_inversion.rs
+++ b/crates/core_simd/examples/matrix_inversion.rs
@@ -1,7 +1,10 @@
 //! 4x4 matrix inverse
 // Code ported from the `packed_simd` crate
 // Run this code with `cargo test --example matrix_inversion`
-#![feature(array_chunks)]
+#![feature(
+    array_chunks,
+    portable_simd,
+)]
 use core_simd::*;
 
 // Gotta define our own 4x4 matrix since Rust doesn't ship multidim arrays yet :^)
diff --git a/crates/core_simd/examples/nbody.rs b/crates/core_simd/examples/nbody.rs
index 44e1c6e87d0e..40e4e18b0264 100644
--- a/crates/core_simd/examples/nbody.rs
+++ b/crates/core_simd/examples/nbody.rs
@@ -1,3 +1,5 @@
+#![feature(portable_simd)]
+
 /// Benchmarks game nbody code
 /// Taken from the `packed_simd` crate
 /// Run this benchmark with `cargo test --example nbody`
diff --git a/crates/core_simd/src/array.rs b/crates/core_simd/src/array.rs
index 0a52876e55b5..25c53097bebe 100644
--- a/crates/core_simd/src/array.rs
+++ b/crates/core_simd/src/array.rs
@@ -25,6 +25,7 @@ where
     /// SIMD gather: construct a SIMD vector by reading from a slice, using potentially discontiguous indices.
     /// If an index is out of bounds, that lane instead selects the value from the "or" vector.
     /// ```
+    /// # #![feature(portable_simd)]
     /// # use core_simd::*;
     /// let vec: Vec<i32> = vec![10, 11, 12, 13, 14, 15, 16, 17, 18];
     /// let idxs = SimdUsize::<4>::from_array([9, 3, 0, 5]);
@@ -42,6 +43,7 @@ where
     /// SIMD gather: construct a SIMD vector by reading from a slice, using potentially discontiguous indices.
     /// Out-of-bounds indices instead use the default value for that lane (0).
     /// ```
+    /// # #![feature(portable_simd)]
     /// # use core_simd::*;
     /// let vec: Vec<i32> = vec![10, 11, 12, 13, 14, 15, 16, 17, 18];
     /// let idxs = SimdUsize::<4>::from_array([9, 3, 0, 5]);
@@ -61,6 +63,7 @@ where
     /// SIMD gather: construct a SIMD vector by reading from a slice, using potentially discontiguous indices.
     /// Out-of-bounds or masked indices instead select the value from the "or" vector.
     /// ```
+    /// # #![feature(portable_simd)]
     /// # use core_simd::*;
     /// let vec: Vec<i32> = vec![10, 11, 12, 13, 14, 15, 16, 17, 18];
     /// let idxs = SimdUsize::<4>::from_array([9, 3, 0, 5]);
@@ -90,6 +93,7 @@ where
     /// Out-of-bounds indices are not written.
     /// `scatter` writes "in order", so if an index receives two writes, only the last is guaranteed.
     /// ```
+    /// # #![feature(portable_simd)]
     /// # use core_simd::*;
     /// let mut vec: Vec<i32> = vec![10, 11, 12, 13, 14, 15, 16, 17, 18];
     /// let idxs = SimdUsize::<4>::from_array([9, 3, 0, 0]);
@@ -107,6 +111,7 @@ where
     /// Out-of-bounds or masked indices are not written.
     /// `scatter_select` writes "in order", so if an index receives two writes, only the last is guaranteed.
     /// ```
+    /// # #![feature(portable_simd)]
     /// # use core_simd::*;
     /// let mut vec: Vec<i32> = vec![10, 11, 12, 13, 14, 15, 16, 17, 18];
     /// let idxs = SimdUsize::<4>::from_array([9, 3, 0, 0]);
diff --git a/crates/core_simd/src/lib.rs b/crates/core_simd/src/lib.rs
index 235733b34909..a64904dee304 100644
--- a/crates/core_simd/src/lib.rs
+++ b/crates/core_simd/src/lib.rs
@@ -1,7 +1,15 @@
 #![no_std]
 #![allow(incomplete_features)]
-#![feature(repr_simd, platform_intrinsics, simd_ffi, const_generics, stdsimd)]
+#![feature(
+    const_generics, 
+    platform_intrinsics,
+    repr_simd,
+    simd_ffi,
+    staged_api,
+    stdsimd,
+)]
 #![warn(missing_docs)]
+#![unstable(feature = "portable_simd", issue = "86656")]
 //! Portable SIMD module.
 
 #[macro_use]
diff --git a/crates/core_simd/src/math.rs b/crates/core_simd/src/math.rs
index 6a243dbd1962..7290a28362f9 100644
--- a/crates/core_simd/src/math.rs
+++ b/crates/core_simd/src/math.rs
@@ -6,6 +6,7 @@ macro_rules! impl_uint_arith {
             ///
             /// # Examples
             /// ```
+            /// # #![feature(portable_simd)]
             /// # use core_simd::*;
             #[doc = concat!("# use core::", stringify!($n), "::MAX;")]
             #[doc = concat!("let x = ", stringify!($name), "::from_array([2, 1, 0, MAX]);")]
@@ -24,6 +25,7 @@ macro_rules! impl_uint_arith {
             ///
             /// # Examples
             /// ```
+            /// # #![feature(portable_simd)]
             /// # use core_simd::*;
             #[doc = concat!("# use core::", stringify!($n), "::MAX;")]
             #[doc = concat!("let x = ", stringify!($name), "::from_array([2, 1, 0, MAX]);")]
@@ -48,6 +50,7 @@ macro_rules! impl_int_arith {
             ///
             /// # Examples
             /// ```
+            /// # #![feature(portable_simd)]
             /// # use core_simd::*;
             #[doc = concat!("# use core::", stringify!($n), "::{MIN, MAX};")]
             #[doc = concat!("let x = ", stringify!($name), "::from_array([MIN, 0, 1, MAX]);")]
@@ -66,6 +69,7 @@ macro_rules! impl_int_arith {
             ///
             /// # Examples
             /// ```
+            /// # #![feature(portable_simd)]
             /// # use core_simd::*;
             #[doc = concat!("# use core::", stringify!($n), "::{MIN, MAX};")]
             #[doc = concat!("let x = ", stringify!($name), "::from_array([MIN, -2, -1, MAX]);")]
@@ -84,6 +88,7 @@ macro_rules! impl_int_arith {
             ///
             /// # Examples
             /// ```
+            /// # #![feature(portable_simd)]
             /// # use core_simd::*;
             #[doc = concat!("# use core::", stringify!($n), "::{MIN, MAX};")]
             #[doc = concat!("let xs = ", stringify!($name), "::from_array([MIN, MIN +1, -5, 0]);")]
@@ -101,6 +106,7 @@ macro_rules! impl_int_arith {
             ///
             /// # Examples
             /// ```
+            /// # #![feature(portable_simd)]
             /// # use core_simd::*;
             #[doc = concat!("# use core::", stringify!($n), "::{MIN, MAX};")]
             #[doc = concat!("let xs = ", stringify!($name), "::from_array([MIN, -2, 0, 3]);")]
@@ -122,6 +128,7 @@ macro_rules! impl_int_arith {
             ///
             /// # Examples
             /// ```
+            /// # #![feature(portable_simd)]
             /// # use core_simd::*;
             #[doc = concat!("# use core::", stringify!($n), "::{MIN, MAX};")]
             #[doc = concat!("let x = ", stringify!($name), "::from_array([MIN, -2, 3, MAX]);")]
diff --git a/crates/core_simd/src/permute.rs b/crates/core_simd/src/permute.rs
index a64fdbc9dd0a..01148a26bad4 100644
--- a/crates/core_simd/src/permute.rs
+++ b/crates/core_simd/src/permute.rs
@@ -11,12 +11,13 @@ macro_rules! impl_shuffle_lane {
             /// than storing and reloading from memory.
             ///
             /// ```
+            /// #![feature(portable_simd)]
             /// # use core_simd::*;
-            // let a = f32x4::from_array([1.0, 2.0, 3.0, 4.0]);
-            // let b = f32x4::from_array([5.0, 6.0, 7.0, 8.0]);
-            // const IDXS: [u32; 4] = [4,0,3,7];
-            // let c = f32x4::shuffle::<IDXS>(a,b);
-            // assert_eq!(f32x4::from_array([5.0, 1.0, 4.0, 8.0]), c);
+            /// let a = f32x4::from_array([1.0, 2.0, 3.0, 4.0]);
+            /// let b = f32x4::from_array([5.0, 6.0, 7.0, 8.0]);
+            /// const IDXS: [u32; 4] = [4,0,3,7];
+            /// let c = f32x4::shuffle::<IDXS>(a,b);
+            /// assert_eq!(f32x4::from_array([5.0, 1.0, 4.0, 8.0]), c);
             /// ```
             #[inline]
             pub fn shuffle<const IDX: [u32; $n]>(self, second: Self) -> Self {
@@ -51,6 +52,7 @@ macro_rules! impl_shuffle_lane {
             /// This particular permutation is efficient on many architectures.
             ///
             /// ```
+            /// #![feature(portable_simd)]
             /// # use core_simd::SimdU32;
             /// let a = SimdU32::from_array([0, 1, 2, 3]);
             /// let b = SimdU32::from_array([4, 5, 6, 7]);
@@ -102,6 +104,7 @@ macro_rules! impl_shuffle_lane {
             /// This particular permutation is efficient on many architectures.
             ///
             /// ```
+            /// #![feature(portable_simd)]
             /// # use core_simd::SimdU32;
             /// let a = SimdU32::from_array([0, 4, 1, 5]);
             /// let b = SimdU32::from_array([2, 6, 3, 7]);
diff --git a/crates/core_simd/src/select.rs b/crates/core_simd/src/select.rs
index 343fd33a5350..dee1d775eb82 100644
--- a/crates/core_simd/src/select.rs
+++ b/crates/core_simd/src/select.rs
@@ -57,6 +57,7 @@ macro_rules! impl_select {
             /// that lane mask is true, and `false_values` if that lane mask is false.
             ///
             /// ```
+            /// # #![feature(portable_simd)]
             /// # use core_simd::{Mask32, SimdI32};
             /// let a = SimdI32::from_array([0, 1, 2, 3]);
             /// let b = SimdI32::from_array([4, 5, 6, 7]);
@@ -67,6 +68,7 @@ macro_rules! impl_select {
             ///
             /// `select` can also be used on masks:
             /// ```
+            /// # #![feature(portable_simd)]
             /// # use core_simd::Mask32;
             /// let a = Mask32::from_array([true, true, false, false]);
             /// let b = Mask32::from_array([false, false, true, true]);
diff --git a/crates/core_simd/tests/f32_ops.rs b/crates/core_simd/tests/f32_ops.rs
index ac5499b7ffeb..98283110097e 100644
--- a/crates/core_simd/tests/f32_ops.rs
+++ b/crates/core_simd/tests/f32_ops.rs
@@ -1,3 +1,5 @@
+#![feature(portable_simd)]
+
 #[macro_use]
 mod ops_macros;
 impl_float_tests! { SimdF32, f32, i32 }
diff --git a/crates/core_simd/tests/f64_ops.rs b/crates/core_simd/tests/f64_ops.rs
index dcdb2aa31522..0818b0c5c5a9 100644
--- a/crates/core_simd/tests/f64_ops.rs
+++ b/crates/core_simd/tests/f64_ops.rs
@@ -1,3 +1,5 @@
+#![feature(portable_simd)]
+
 #[macro_use]
 mod ops_macros;
 impl_float_tests! { SimdF64, f64, i64 }
diff --git a/crates/core_simd/tests/i16_ops.rs b/crates/core_simd/tests/i16_ops.rs
index 4d2a7b053b5c..33d92faa5956 100644
--- a/crates/core_simd/tests/i16_ops.rs
+++ b/crates/core_simd/tests/i16_ops.rs
@@ -1,3 +1,5 @@
+#![feature(portable_simd)]
+
 #[macro_use]
 mod ops_macros;
 impl_signed_tests! { SimdI16, i16 }
diff --git a/crates/core_simd/tests/i32_ops.rs b/crates/core_simd/tests/i32_ops.rs
index 90079d727e4b..481bca23e834 100644
--- a/crates/core_simd/tests/i32_ops.rs
+++ b/crates/core_simd/tests/i32_ops.rs
@@ -1,3 +1,5 @@
+#![feature(portable_simd)]
+
 #[macro_use]
 mod ops_macros;
 impl_signed_tests! { SimdI32, i32 }
diff --git a/crates/core_simd/tests/i64_ops.rs b/crates/core_simd/tests/i64_ops.rs
index ebc3e194974b..5ab0614c8480 100644
--- a/crates/core_simd/tests/i64_ops.rs
+++ b/crates/core_simd/tests/i64_ops.rs
@@ -1,3 +1,5 @@
+#![feature(portable_simd)]
+
 #[macro_use]
 mod ops_macros;
 impl_signed_tests! { SimdI64, i64 }
diff --git a/crates/core_simd/tests/i8_ops.rs b/crates/core_simd/tests/i8_ops.rs
index 082422b86d29..0db9ee47a9e2 100644
--- a/crates/core_simd/tests/i8_ops.rs
+++ b/crates/core_simd/tests/i8_ops.rs
@@ -1,3 +1,5 @@
+#![feature(portable_simd)]
+
 #[macro_use]
 mod ops_macros;
 impl_signed_tests! { SimdI8, i8 }
diff --git a/crates/core_simd/tests/isize_ops.rs b/crates/core_simd/tests/isize_ops.rs
index 1509d701c29a..8f5470b685c9 100644
--- a/crates/core_simd/tests/isize_ops.rs
+++ b/crates/core_simd/tests/isize_ops.rs
@@ -1,3 +1,5 @@
+#![feature(portable_simd)]
+
 #[macro_use]
 mod ops_macros;
 impl_signed_tests! { SimdIsize, isize }
diff --git a/crates/core_simd/tests/mask_ops.rs b/crates/core_simd/tests/mask_ops.rs
index 96330550b40b..f113b50cb769 100644
--- a/crates/core_simd/tests/mask_ops.rs
+++ b/crates/core_simd/tests/mask_ops.rs
@@ -1 +1,3 @@
+#![feature(portable_simd)]
+
 mod mask_ops_impl;
diff --git a/crates/core_simd/tests/masks.rs b/crates/core_simd/tests/masks.rs
index 32dea49729f0..61d8e449744b 100644
--- a/crates/core_simd/tests/masks.rs
+++ b/crates/core_simd/tests/masks.rs
@@ -1,3 +1,5 @@
+#![feature(portable_simd)]
+
 #[cfg(target_arch = "wasm32")]
 use wasm_bindgen_test::*;
 
diff --git a/crates/core_simd/tests/permute.rs b/crates/core_simd/tests/permute.rs
index 2be43c9cf3ce..4c771002528f 100644
--- a/crates/core_simd/tests/permute.rs
+++ b/crates/core_simd/tests/permute.rs
@@ -1,3 +1,5 @@
+#![feature(portable_simd)]
+
 use core_simd::SimdU32;
 
 #[cfg(target_arch = "wasm32")]
diff --git a/crates/core_simd/tests/round.rs b/crates/core_simd/tests/round.rs
index 85853c0e8778..37044a751125 100644
--- a/crates/core_simd/tests/round.rs
+++ b/crates/core_simd/tests/round.rs
@@ -1,3 +1,5 @@
+#![feature(portable_simd)]
+
 macro_rules! float_rounding_test {
     { $vector:ident, $scalar:tt, $int_scalar:tt } => {
         mod $scalar {
diff --git a/crates/core_simd/tests/to_bytes.rs b/crates/core_simd/tests/to_bytes.rs
index 20da1652a6dd..11228680dde7 100644
--- a/crates/core_simd/tests/to_bytes.rs
+++ b/crates/core_simd/tests/to_bytes.rs
@@ -1,3 +1,5 @@
+#![feature(portable_simd)]
+
 use core_simd::SimdU32;
 
 #[test]
diff --git a/crates/core_simd/tests/u16_ops.rs b/crates/core_simd/tests/u16_ops.rs
index 488e703d54fb..d220dae64568 100644
--- a/crates/core_simd/tests/u16_ops.rs
+++ b/crates/core_simd/tests/u16_ops.rs
@@ -1,3 +1,5 @@
+#![feature(portable_simd)]
+
 #[macro_use]
 mod ops_macros;
 impl_unsigned_tests! { SimdU16, u16 }
diff --git a/crates/core_simd/tests/u32_ops.rs b/crates/core_simd/tests/u32_ops.rs
index bf0631029e37..f27cc30a17fc 100644
--- a/crates/core_simd/tests/u32_ops.rs
+++ b/crates/core_simd/tests/u32_ops.rs
@@ -1,3 +1,5 @@
+#![feature(portable_simd)]
+
 #[macro_use]
 mod ops_macros;
 impl_unsigned_tests! { SimdU32, u32 }
diff --git a/crates/core_simd/tests/u64_ops.rs b/crates/core_simd/tests/u64_ops.rs
index e52fc3cfce1f..ec3df39c53c3 100644
--- a/crates/core_simd/tests/u64_ops.rs
+++ b/crates/core_simd/tests/u64_ops.rs
@@ -1,3 +1,5 @@
+#![feature(portable_simd)]
+
 #[macro_use]
 mod ops_macros;
 impl_unsigned_tests! { SimdU64, u64 }
diff --git a/crates/core_simd/tests/u8_ops.rs b/crates/core_simd/tests/u8_ops.rs
index 45be3580ec39..2c52a52b9216 100644
--- a/crates/core_simd/tests/u8_ops.rs
+++ b/crates/core_simd/tests/u8_ops.rs
@@ -1,3 +1,5 @@
+#![feature(portable_simd)]
+
 #[macro_use]
 mod ops_macros;
 impl_unsigned_tests! { SimdU8, u8 }
diff --git a/crates/core_simd/tests/usize_ops.rs b/crates/core_simd/tests/usize_ops.rs
index 1ce6e7180045..070edc4e266f 100644
--- a/crates/core_simd/tests/usize_ops.rs
+++ b/crates/core_simd/tests/usize_ops.rs
@@ -1,3 +1,5 @@
+#![feature(portable_simd)]
+
 #[macro_use]
 mod ops_macros;
 impl_unsigned_tests! { SimdUsize, usize }

From 732b7edfab46b33e3861172eb867b139a9425574 Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Fri, 23 Jul 2021 20:43:53 -0400
Subject: [PATCH 192/249] Add fmt and clippy to CI (#147)

* Add fmt and clippy to CI
* Add rust components
* Fix formatting
---
 .github/workflows/ci.yml                      |  47 +++++
 crates/core_simd/examples/matrix_inversion.rs | 185 +++++++++---------
 crates/core_simd/src/lib.rs                   |   4 +-
 crates/core_simd/src/vector/float.rs          |   2 +-
 crates/core_simd/tests/to_bytes.rs            |   2 +-
 crates/test_helpers/src/lib.rs                |  30 +--
 6 files changed, 161 insertions(+), 109 deletions(-)

diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
index 2104c74a4d92..9c62a6d40aee 100644
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -11,6 +11,53 @@ env:
   RUSTUP_MAX_RETRIES: 10
 
 jobs:
+  rustfmt:
+    name: "rustfmt"
+    runs-on: ubuntu-latest
+
+    steps:
+      - uses: actions/checkout@v2
+      - name: Setup Rust
+        run: |
+          rustup update nightly --no-self-update
+          rustup default nightly
+          rustup component add rustfmt
+      - name: Run rustfmt
+        run: cargo fmt --all -- --check
+
+  clippy:
+    name: "clippy on ${{ matrix.target }}"
+    runs-on: ubuntu-latest
+    strategy:
+      fail-fast: false
+      matrix:
+        target:
+          # We shouldn't really have any OS-specific code, so think of this as a list of architectures
+          - x86_64-unknown-linux-gnu
+          - i686-unknown-linux-gnu
+          - i586-unknown-linux-gnu
+          - aarch64-unknown-linux-gnu
+          - armv7-unknown-linux-gnueabihf
+          - mips-unknown-linux-gnu
+          - mips64-unknown-linux-gnuabi64
+          - powerpc-unknown-linux-gnu
+          - powerpc64-unknown-linux-gnu
+          - riscv64gc-unknown-linux-gnu
+          - s390x-unknown-linux-gnu
+          - sparc64-unknown-linux-gnu
+          - wasm32-unknown-unknown
+
+    steps:
+      - uses: actions/checkout@v2
+      - name: Setup Rust
+        run: |
+          rustup update nightly --no-self-update
+          rustup default nightly
+          rustup target add ${{ matrix.target }}
+          rustup component add clippy
+      - name: Run Clippy
+        run: cargo clippy --all-targets --target ${{ matrix.target }}
+
   x86-tests:
     name: "${{ matrix.target_feature }} on ${{ matrix.target }}"
     runs-on: ${{ matrix.os }}
diff --git a/crates/core_simd/examples/matrix_inversion.rs b/crates/core_simd/examples/matrix_inversion.rs
index 5c2d4390ad08..29bdc512d77d 100644
--- a/crates/core_simd/examples/matrix_inversion.rs
+++ b/crates/core_simd/examples/matrix_inversion.rs
@@ -1,10 +1,7 @@
 //! 4x4 matrix inverse
 // Code ported from the `packed_simd` crate
 // Run this code with `cargo test --example matrix_inversion`
-#![feature(
-    array_chunks,
-    portable_simd,
-)]
+#![feature(array_chunks, portable_simd)]
 use core_simd::*;
 
 // Gotta define our own 4x4 matrix since Rust doesn't ship multidim arrays yet :^)
@@ -15,27 +12,29 @@ pub struct Matrix4x4([[f32; 4]; 4]);
 pub fn scalar_inv4x4(m: Matrix4x4) -> Option<Matrix4x4> {
     let m = m.0;
 
+    #[rustfmt::skip]
     let mut inv = [
-        [ // row 0:
+        // row 0:
+        [
             // 0,0:
-            m[1][1]  * m[2][2] * m[3][3] -
-            m[1][1]  * m[2][3] * m[3][2] -
-            m[2][1]  * m[1][2]  * m[3][3] +
-            m[2][1]  * m[1][3]  * m[3][2] +
-            m[3][1] * m[1][2]  * m[2][3] -
-            m[3][1] * m[1][3]  * m[2][2],
+            m[1][1] * m[2][2] * m[3][3] -
+            m[1][1] * m[2][3] * m[3][2] -
+            m[2][1] * m[1][2] * m[3][3] +
+            m[2][1] * m[1][3] * m[3][2] +
+            m[3][1] * m[1][2] * m[2][3] -
+            m[3][1] * m[1][3] * m[2][2],
             // 0,1:
-           -m[0][1]  * m[2][2] * m[3][3] +
-            m[0][1]  * m[2][3] * m[3][2] +
-            m[2][1]  * m[0][2] * m[3][3] -
-            m[2][1]  * m[0][3] * m[3][2] -
+           -m[0][1] * m[2][2] * m[3][3] +
+            m[0][1] * m[2][3] * m[3][2] +
+            m[2][1] * m[0][2] * m[3][3] -
+            m[2][1] * m[0][3] * m[3][2] -
             m[3][1] * m[0][2] * m[2][3] +
             m[3][1] * m[0][3] * m[2][2],
             // 0,2:
-            m[0][1]  * m[1][2] * m[3][3] -
-            m[0][1]  * m[1][3] * m[3][2] -
-            m[1][1]  * m[0][2] * m[3][3] +
-            m[1][1]  * m[0][3] * m[3][2] +
+            m[0][1] * m[1][2] * m[3][3] -
+            m[0][1] * m[1][3] * m[3][2] -
+            m[1][1] * m[0][2] * m[3][3] +
+            m[1][1] * m[0][3] * m[3][2] +
             m[3][1] * m[0][2] * m[1][3] -
             m[3][1] * m[0][3] * m[1][2],
             // 0,3:
@@ -46,26 +45,27 @@ pub fn scalar_inv4x4(m: Matrix4x4) -> Option<Matrix4x4> {
             m[2][1] * m[0][2] * m[1][3] +
             m[2][1] * m[0][3] * m[1][2],
         ],
-        [ // row 1
+        // row 1
+        [
             // 1,0:
-           -m[1][0]  * m[2][2] * m[3][3] +
-            m[1][0]  * m[2][3] * m[3][2] +
-            m[2][0]  * m[1][2]  * m[3][3] -
-            m[2][0]  * m[1][3]  * m[3][2] -
-            m[3][0] * m[1][2]  * m[2][3] +
-            m[3][0] * m[1][3]  * m[2][2],
+           -m[1][0] * m[2][2] * m[3][3] +
+            m[1][0] * m[2][3] * m[3][2] +
+            m[2][0] * m[1][2] * m[3][3] -
+            m[2][0] * m[1][3] * m[3][2] -
+            m[3][0] * m[1][2] * m[2][3] +
+            m[3][0] * m[1][3] * m[2][2],
             // 1,1:
-            m[0][0]  * m[2][2] * m[3][3] -
-            m[0][0]  * m[2][3] * m[3][2] -
-            m[2][0]  * m[0][2] * m[3][3] +
-            m[2][0]  * m[0][3] * m[3][2] +
+            m[0][0] * m[2][2] * m[3][3] -
+            m[0][0] * m[2][3] * m[3][2] -
+            m[2][0] * m[0][2] * m[3][3] +
+            m[2][0] * m[0][3] * m[3][2] +
             m[3][0] * m[0][2] * m[2][3] -
             m[3][0] * m[0][3] * m[2][2],
             // 1,2:
-           -m[0][0]  * m[1][2] * m[3][3] +
-            m[0][0]  * m[1][3] * m[3][2] +
-            m[1][0]  * m[0][2] * m[3][3] -
-            m[1][0]  * m[0][3] * m[3][2] -
+           -m[0][0] * m[1][2] * m[3][3] +
+            m[0][0] * m[1][3] * m[3][2] +
+            m[1][0] * m[0][2] * m[3][3] -
+            m[1][0] * m[0][3] * m[3][2] -
             m[3][0] * m[0][2] * m[1][3] +
             m[3][0] * m[0][3] * m[1][2],
             // 1,3:
@@ -76,26 +76,27 @@ pub fn scalar_inv4x4(m: Matrix4x4) -> Option<Matrix4x4> {
             m[2][0] * m[0][2] * m[1][3] -
             m[2][0] * m[0][3] * m[1][2],
         ],
-        [ // row 2
+        // row 2
+        [
             // 2,0:
-            m[1][0]  * m[2][1] * m[3][3] -
-            m[1][0]  * m[2][3] * m[3][1] -
-            m[2][0]  * m[1][1] * m[3][3] +
-            m[2][0]  * m[1][3] * m[3][1] +
+            m[1][0] * m[2][1] * m[3][3] -
+            m[1][0] * m[2][3] * m[3][1] -
+            m[2][0] * m[1][1] * m[3][3] +
+            m[2][0] * m[1][3] * m[3][1] +
             m[3][0] * m[1][1] * m[2][3] -
             m[3][0] * m[1][3] * m[2][1],
             // 2,1:
-           -m[0][0]  * m[2][1] * m[3][3] +
-            m[0][0]  * m[2][3] * m[3][1] +
-            m[2][0]  * m[0][1] * m[3][3] -
-            m[2][0]  * m[0][3] * m[3][1] -
+           -m[0][0] * m[2][1] * m[3][3] +
+            m[0][0] * m[2][3] * m[3][1] +
+            m[2][0] * m[0][1] * m[3][3] -
+            m[2][0] * m[0][3] * m[3][1] -
             m[3][0] * m[0][1] * m[2][3] +
             m[3][0] * m[0][3] * m[2][1],
             // 2,2:
-            m[0][0]  * m[1][1] * m[3][3] -
-            m[0][0]  * m[1][3] * m[3][1] -
-            m[1][0]  * m[0][1] * m[3][3] +
-            m[1][0]  * m[0][3] * m[3][1] +
+            m[0][0] * m[1][1] * m[3][3] -
+            m[0][0] * m[1][3] * m[3][1] -
+            m[1][0] * m[0][1] * m[3][3] +
+            m[1][0] * m[0][3] * m[3][1] +
             m[3][0] * m[0][1] * m[1][3] -
             m[3][0] * m[0][3] * m[1][1],
             // 2,3:
@@ -106,26 +107,27 @@ pub fn scalar_inv4x4(m: Matrix4x4) -> Option<Matrix4x4> {
             m[2][0] * m[0][1] * m[1][3] +
             m[2][0] * m[0][3] * m[1][1],
         ],
-        [ // row 3
+        // row 3
+        [
             // 3,0:
-           -m[1][0]  * m[2][1] * m[3][2] +
-            m[1][0]  * m[2][2] * m[3][1] +
-            m[2][0]  * m[1][1] * m[3][2] -
-            m[2][0]  * m[1][2] * m[3][1] -
+           -m[1][0] * m[2][1] * m[3][2] +
+            m[1][0] * m[2][2] * m[3][1] +
+            m[2][0] * m[1][1] * m[3][2] -
+            m[2][0] * m[1][2] * m[3][1] -
             m[3][0] * m[1][1] * m[2][2] +
             m[3][0] * m[1][2] * m[2][1],
             // 3,1:
-            m[0][0]  * m[2][1] * m[3][2] -
-            m[0][0]  * m[2][2] * m[3][1] -
-            m[2][0]  * m[0][1] * m[3][2] +
-            m[2][0]  * m[0][2] * m[3][1] +
+            m[0][0] * m[2][1] * m[3][2] -
+            m[0][0] * m[2][2] * m[3][1] -
+            m[2][0] * m[0][1] * m[3][2] +
+            m[2][0] * m[0][2] * m[3][1] +
             m[3][0] * m[0][1] * m[2][2] -
             m[3][0] * m[0][2] * m[2][1],
             // 3,2:
-           -m[0][0]  * m[1][1] * m[3][2] +
-            m[0][0]  * m[1][2] * m[3][1] +
-            m[1][0]  * m[0][1] * m[3][2] -
-            m[1][0]  * m[0][2] * m[3][1] -
+           -m[0][0] * m[1][1] * m[3][2] +
+            m[0][0] * m[1][2] * m[3][1] +
+            m[1][0] * m[0][1] * m[3][2] -
+            m[1][0] * m[0][2] * m[3][1] -
             m[3][0] * m[0][1] * m[1][2] +
             m[3][0] * m[0][2] * m[1][1],
             // 3,3:
@@ -138,9 +140,10 @@ pub fn scalar_inv4x4(m: Matrix4x4) -> Option<Matrix4x4> {
         ],
     ];
 
-    let det = m[0][0] * inv[0][0] + m[0][1] * inv[1][0] +
-              m[0][2] * inv[2][0] + m[0][3] * inv[3][0];
-    if det == 0. { return None; }
+    let det = m[0][0] * inv[0][0] + m[0][1] * inv[1][0] + m[0][2] * inv[2][0] + m[0][3] * inv[3][0];
+    if det == 0. {
+        return None;
+    }
 
     let det_inv = 1. / det;
 
@@ -163,76 +166,76 @@ pub fn simd_inv4x4(m: Matrix4x4) -> Option<Matrix4x4> {
     // 2 argument shuffle, returns an f32x4
     // the first f32x4 is indexes 0..=3
     // the second f32x4 is indexed 4..=7
-    let tmp1 = f32x4::shuffle::<{[0, 1, 4, 5]}>(m_0, m_1);
-    let row1 = f32x4::shuffle::<{[0, 1, 4, 5]}>(m_2, m_3,);
+    let tmp1 = f32x4::shuffle::<{ [0, 1, 4, 5] }>(m_0, m_1);
+    let row1 = f32x4::shuffle::<{ [0, 1, 4, 5] }>(m_2, m_3);
 
-    let row0 = f32x4::shuffle::<{[0, 2, 4, 6]}>(tmp1, row1);
-    let row1 = f32x4::shuffle::<{[1, 3, 5, 7]}>(row1, tmp1);
+    let row0 = f32x4::shuffle::<{ [0, 2, 4, 6] }>(tmp1, row1);
+    let row1 = f32x4::shuffle::<{ [1, 3, 5, 7] }>(row1, tmp1);
 
-    let tmp1 = f32x4::shuffle::<{[2, 3, 6, 7]}>(m_0, m_1);
-    let row3 = f32x4::shuffle::<{[2, 3, 6, 7]}>(m_2, m_3);
-    let row2 = f32x4::shuffle::<{[0, 2, 4, 6]}>(tmp1, row3);
-    let row3 = f32x4::shuffle::<{[1, 3, 5, 7]}>(row3, tmp1);
+    let tmp1 = f32x4::shuffle::<{ [2, 3, 6, 7] }>(m_0, m_1);
+    let row3 = f32x4::shuffle::<{ [2, 3, 6, 7] }>(m_2, m_3);
+    let row2 = f32x4::shuffle::<{ [0, 2, 4, 6] }>(tmp1, row3);
+    let row3 = f32x4::shuffle::<{ [1, 3, 5, 7] }>(row3, tmp1);
 
     let tmp1 = row2 * row3;
     // there's no syntax for a 1 arg shuffle yet,
     // so we just pass the same f32x4 twice
-    let tmp1 = f32x4::shuffle::<{[1, 0, 3, 2]}>(tmp1, tmp1);
+    let tmp1 = f32x4::shuffle::<{ [1, 0, 3, 2] }>(tmp1, tmp1);
 
     let minor0 = row1 * tmp1;
     let minor1 = row0 * tmp1;
-    let tmp1   = f32x4::shuffle::<{[2, 3, 0, 1]}>(tmp1, tmp1);
+    let tmp1 = f32x4::shuffle::<{ [2, 3, 0, 1] }>(tmp1, tmp1);
     let minor0 = (row1 * tmp1) - minor0;
     let minor1 = (row0 * tmp1) - minor1;
-    let minor1 = f32x4::shuffle::<{[2, 3, 0, 1]}>(minor1, minor1);
+    let minor1 = f32x4::shuffle::<{ [2, 3, 0, 1] }>(minor1, minor1);
 
     let tmp1 = row1 * row2;
-    let tmp1 = f32x4::shuffle::<{[1, 0, 3, 2]}>(tmp1, tmp1);
+    let tmp1 = f32x4::shuffle::<{ [1, 0, 3, 2] }>(tmp1, tmp1);
     let minor0 = (row3 * tmp1) + minor0;
     let minor3 = row0 * tmp1;
-    let tmp1 = f32x4::shuffle::<{[2, 3, 0, 1]}>(tmp1, tmp1);
+    let tmp1 = f32x4::shuffle::<{ [2, 3, 0, 1] }>(tmp1, tmp1);
 
     let minor0 = minor0 - row3 * tmp1;
     let minor3 = row0 * tmp1 - minor3;
-    let minor3 = f32x4::shuffle::<{[2, 3, 0, 1]}>(minor3, minor3);
+    let minor3 = f32x4::shuffle::<{ [2, 3, 0, 1] }>(minor3, minor3);
 
-    let tmp1 = row3 * f32x4::shuffle::<{[2, 3, 0, 1]}>(row1, row1);
-    let tmp1 = f32x4::shuffle::<{[1, 0, 3, 2]}>(tmp1, tmp1);
-    let row2 = f32x4::shuffle::<{[2, 3, 0, 1]}>(row2, row2);
+    let tmp1 = row3 * f32x4::shuffle::<{ [2, 3, 0, 1] }>(row1, row1);
+    let tmp1 = f32x4::shuffle::<{ [1, 0, 3, 2] }>(tmp1, tmp1);
+    let row2 = f32x4::shuffle::<{ [2, 3, 0, 1] }>(row2, row2);
     let minor0 = row2 * tmp1 + minor0;
     let minor2 = row0 * tmp1;
-    let tmp1 = f32x4::shuffle::<{[2, 3, 0, 1]}>(tmp1, tmp1);
+    let tmp1 = f32x4::shuffle::<{ [2, 3, 0, 1] }>(tmp1, tmp1);
     let minor0 = minor0 - row2 * tmp1;
     let minor2 = row0 * tmp1 - minor2;
-    let minor2 = f32x4::shuffle::<{[2, 3, 0, 1]}>(minor2, minor2);
+    let minor2 = f32x4::shuffle::<{ [2, 3, 0, 1] }>(minor2, minor2);
 
     let tmp1 = row0 * row1;
-    let tmp1 = f32x4::shuffle::<{[1, 0, 3, 2]}>(tmp1, tmp1);
+    let tmp1 = f32x4::shuffle::<{ [1, 0, 3, 2] }>(tmp1, tmp1);
     let minor2 = minor2 + row3 * tmp1;
     let minor3 = row2 * tmp1 - minor3;
-    let tmp1 = f32x4::shuffle::<{[2, 3, 0, 1]}>(tmp1, tmp1);
+    let tmp1 = f32x4::shuffle::<{ [2, 3, 0, 1] }>(tmp1, tmp1);
     let minor2 = row3 * tmp1 - minor2;
     let minor3 = minor3 - row2 * tmp1;
 
     let tmp1 = row0 * row3;
-    let tmp1 = f32x4::shuffle::<{[1, 0, 3, 2]}>(tmp1, tmp1);
+    let tmp1 = f32x4::shuffle::<{ [1, 0, 3, 2] }>(tmp1, tmp1);
     let minor1 = minor1 - row2 * tmp1;
     let minor2 = row1 * tmp1 + minor2;
-    let tmp1 = f32x4::shuffle::<{[2, 3, 0, 1]}>(tmp1, tmp1);
+    let tmp1 = f32x4::shuffle::<{ [2, 3, 0, 1] }>(tmp1, tmp1);
     let minor1 = row2 * tmp1 + minor1;
     let minor2 = minor2 - row1 * tmp1;
 
     let tmp1 = row0 * row2;
-    let tmp1 = f32x4::shuffle::<{[1, 0, 3, 2]}>(tmp1, tmp1);
+    let tmp1 = f32x4::shuffle::<{ [1, 0, 3, 2] }>(tmp1, tmp1);
     let minor1 = row3 * tmp1 + minor1;
     let minor3 = minor3 - row1 * tmp1;
-    let tmp1 = f32x4::shuffle::<{[2, 3, 0, 1]}>(tmp1, tmp1);
+    let tmp1 = f32x4::shuffle::<{ [2, 3, 0, 1] }>(tmp1, tmp1);
     let minor1 = minor1 - row3 * tmp1;
     let minor3 = row1 * tmp1 + minor3;
 
     let det = row0 * minor0;
-    let det = f32x4::shuffle::<{[2, 3, 0, 1]}>(det, det) + det;
-    let det = f32x4::shuffle::<{[1, 0, 3, 2]}>(det, det) + det;
+    let det = f32x4::shuffle::<{ [2, 3, 0, 1] }>(det, det) + det;
+    let det = f32x4::shuffle::<{ [1, 0, 3, 2] }>(det, det) + det;
 
     if det.horizontal_sum() == 0. {
         return None;
@@ -256,7 +259,6 @@ pub fn simd_inv4x4(m: Matrix4x4) -> Option<Matrix4x4> {
     Some(Matrix4x4(m))
 }
 
-
 #[cfg(test)]
 #[rustfmt::skip]
 mod tests {
@@ -313,7 +315,6 @@ mod tests {
     }
 }
 
-
 fn main() {
     // Empty main to make cargo happy
 }
diff --git a/crates/core_simd/src/lib.rs b/crates/core_simd/src/lib.rs
index a64904dee304..e48f8062d2c8 100644
--- a/crates/core_simd/src/lib.rs
+++ b/crates/core_simd/src/lib.rs
@@ -1,12 +1,12 @@
 #![no_std]
 #![allow(incomplete_features)]
 #![feature(
-    const_generics, 
+    const_generics,
     platform_intrinsics,
     repr_simd,
     simd_ffi,
     staged_api,
-    stdsimd,
+    stdsimd
 )]
 #![warn(missing_docs)]
 #![unstable(feature = "portable_simd", issue = "86656")]
diff --git a/crates/core_simd/src/vector/float.rs b/crates/core_simd/src/vector/float.rs
index 91087740c450..b6e9b61f71f6 100644
--- a/crates/core_simd/src/vector/float.rs
+++ b/crates/core_simd/src/vector/float.rs
@@ -174,7 +174,7 @@ macro_rules! impl_float_vector {
             }
 
             /// Restrict each lane to a certain interval unless it is NaN.
-            /// 
+            ///
             /// For each lane in `self`, returns the corresponding lane in `max` if the lane is
             /// greater than `max`, and the corresponding lane in `min` if the lane is less
             /// than `min`.  Otherwise returns the lane in `self`.
diff --git a/crates/core_simd/tests/to_bytes.rs b/crates/core_simd/tests/to_bytes.rs
index 11228680dde7..8d662b3238c9 100644
--- a/crates/core_simd/tests/to_bytes.rs
+++ b/crates/core_simd/tests/to_bytes.rs
@@ -6,7 +6,7 @@ use core_simd::SimdU32;
 fn byte_convert() {
     let int = SimdU32::from_array([0xdeadbeef, 0x8badf00d]);
     let bytes = int.to_ne_bytes();
-    assert_eq!(int[0].to_ne_bytes(), bytes[..4]); 
+    assert_eq!(int[0].to_ne_bytes(), bytes[..4]);
     assert_eq!(int[1].to_ne_bytes(), bytes[4..]);
     assert_eq!(SimdU32::from_ne_bytes(bytes), int);
 }
diff --git a/crates/test_helpers/src/lib.rs b/crates/test_helpers/src/lib.rs
index 4f2380b8e5ba..318a7b3005e3 100644
--- a/crates/test_helpers/src/lib.rs
+++ b/crates/test_helpers/src/lib.rs
@@ -304,19 +304,23 @@ pub fn test_ternary_elementwise<
     Vector3: Into<[Scalar3; LANES]> + From<[Scalar3; LANES]> + Copy,
     VectorResult: Into<[ScalarResult; LANES]> + From<[ScalarResult; LANES]> + Copy,
 {
-    test_3(&|x: [Scalar1; LANES], y: [Scalar2; LANES], z: [Scalar3; LANES]| {
-        proptest::prop_assume!(check(x, y, z));
-        let result_1: [ScalarResult; LANES] = fv(x.into(), y.into(), z.into()).into();
-        let result_2: [ScalarResult; LANES] = {
-            let mut result = [ScalarResult::default(); LANES];
-            for ((i1, (i2, i3)), o) in x.iter().zip(y.iter().zip(z.iter())).zip(result.iter_mut()) {
-                *o = fs(*i1, *i2, *i3);
-            }
-            result
-        };
-        crate::prop_assert_biteq!(result_1, result_2);
-        Ok(())
-    });
+    test_3(
+        &|x: [Scalar1; LANES], y: [Scalar2; LANES], z: [Scalar3; LANES]| {
+            proptest::prop_assume!(check(x, y, z));
+            let result_1: [ScalarResult; LANES] = fv(x.into(), y.into(), z.into()).into();
+            let result_2: [ScalarResult; LANES] = {
+                let mut result = [ScalarResult::default(); LANES];
+                for ((i1, (i2, i3)), o) in
+                    x.iter().zip(y.iter().zip(z.iter())).zip(result.iter_mut())
+                {
+                    *o = fs(*i1, *i2, *i3);
+                }
+                result
+            };
+            crate::prop_assert_biteq!(result_1, result_2);
+            Ok(())
+        },
+    );
 }
 
 /// Expand a const-generic test into separate tests for each possible lane count.

From c077bf3c07280d1026fa8a5a1e36c9fbbdaec10b Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Fri, 25 Jun 2021 03:34:10 +0000
Subject: [PATCH 193/249] Rename SimdArray to Vector, remove its generic
 parameter, and remove LanesAtMost32

---
 crates/core_simd/src/array.rs            | 253 -----------------------
 crates/core_simd/src/comparisons.rs      |   6 +-
 crates/core_simd/src/first.rs            | 160 ++++++++++++--
 crates/core_simd/src/fmt.rs              |   2 +-
 crates/core_simd/src/iter.rs             |   8 +-
 crates/core_simd/src/lanes_at_most_32.rs |  54 -----
 crates/core_simd/src/lib.rs              |   6 -
 crates/core_simd/src/masks/bitmask.rs    |  16 +-
 crates/core_simd/src/masks/full_masks.rs |  30 +--
 crates/core_simd/src/masks/mod.rs        |  60 +++---
 crates/core_simd/src/math.rs             |   4 +-
 crates/core_simd/src/ops.rs              |  78 +++----
 crates/core_simd/src/reduction.rs        |   8 +-
 crates/core_simd/src/round.rs            |   6 +-
 crates/core_simd/src/select.rs           |  12 +-
 crates/core_simd/src/to_bytes.rs         |   8 +-
 crates/core_simd/src/vector.rs           |  10 -
 crates/core_simd/src/vector/float.rs     |  14 +-
 crates/core_simd/src/vector/int.rs       |  20 +-
 crates/core_simd/src/vector/mod.rs       | 132 ++++++++++++
 crates/core_simd/src/vector/ptr.rs       |   4 +-
 crates/core_simd/src/vector/uint.rs      |  16 +-
 crates/test_helpers/src/lib.rs           |  48 ++---
 23 files changed, 450 insertions(+), 505 deletions(-)
 delete mode 100644 crates/core_simd/src/array.rs
 delete mode 100644 crates/core_simd/src/lanes_at_most_32.rs
 delete mode 100644 crates/core_simd/src/vector.rs
 create mode 100644 crates/core_simd/src/vector/mod.rs

diff --git a/crates/core_simd/src/array.rs b/crates/core_simd/src/array.rs
deleted file mode 100644
index 25c53097bebe..000000000000
--- a/crates/core_simd/src/array.rs
+++ /dev/null
@@ -1,253 +0,0 @@
-use crate::intrinsics;
-use crate::masks::*;
-use crate::vector::ptr::{SimdConstPtr, SimdMutPtr};
-use crate::vector::*;
-
-/// A representation of a vector as an "array" with indices, implementing
-/// operations applicable to any vector type based solely on "having lanes",
-/// and describing relationships between vector and scalar types.
-pub trait SimdArray<const LANES: usize>: crate::LanesAtMost32
-where
-    SimdUsize<LANES>: crate::LanesAtMost32,
-    SimdIsize<LANES>: crate::LanesAtMost32,
-    MaskSize<LANES>: crate::Mask,
-    Self: Sized,
-{
-    /// The scalar type in every lane of this vector type.
-    type Scalar: Copy + Sized;
-    /// The number of lanes for this vector.
-    const LANES: usize = LANES;
-
-    /// Generates a SIMD vector with the same value in every lane.
-    #[must_use]
-    fn splat(val: Self::Scalar) -> Self;
-
-    /// SIMD gather: construct a SIMD vector by reading from a slice, using potentially discontiguous indices.
-    /// If an index is out of bounds, that lane instead selects the value from the "or" vector.
-    /// ```
-    /// # #![feature(portable_simd)]
-    /// # use core_simd::*;
-    /// let vec: Vec<i32> = vec![10, 11, 12, 13, 14, 15, 16, 17, 18];
-    /// let idxs = SimdUsize::<4>::from_array([9, 3, 0, 5]);
-    /// let alt = SimdI32::from_array([-5, -4, -3, -2]);
-    ///
-    /// let result = SimdI32::<4>::gather_or(&vec, idxs, alt); // Note the lane that is out-of-bounds.
-    /// assert_eq!(result, SimdI32::from_array([-5, 13, 10, 15]));
-    /// ```
-    #[must_use]
-    #[inline]
-    fn gather_or(slice: &[Self::Scalar], idxs: SimdUsize<LANES>, or: Self) -> Self {
-        Self::gather_select(slice, MaskSize::splat(true), idxs, or)
-    }
-
-    /// SIMD gather: construct a SIMD vector by reading from a slice, using potentially discontiguous indices.
-    /// Out-of-bounds indices instead use the default value for that lane (0).
-    /// ```
-    /// # #![feature(portable_simd)]
-    /// # use core_simd::*;
-    /// let vec: Vec<i32> = vec![10, 11, 12, 13, 14, 15, 16, 17, 18];
-    /// let idxs = SimdUsize::<4>::from_array([9, 3, 0, 5]);
-    ///
-    /// let result = SimdI32::<4>::gather_or_default(&vec, idxs); // Note the lane that is out-of-bounds.
-    /// assert_eq!(result, SimdI32::from_array([0, 13, 10, 15]));
-    /// ```
-    #[must_use]
-    #[inline]
-    fn gather_or_default(slice: &[Self::Scalar], idxs: SimdUsize<LANES>) -> Self
-    where
-        Self::Scalar: Default,
-    {
-        Self::gather_or(slice, idxs, Self::splat(Self::Scalar::default()))
-    }
-
-    /// SIMD gather: construct a SIMD vector by reading from a slice, using potentially discontiguous indices.
-    /// Out-of-bounds or masked indices instead select the value from the "or" vector.
-    /// ```
-    /// # #![feature(portable_simd)]
-    /// # use core_simd::*;
-    /// let vec: Vec<i32> = vec![10, 11, 12, 13, 14, 15, 16, 17, 18];
-    /// let idxs = SimdUsize::<4>::from_array([9, 3, 0, 5]);
-    /// let alt = SimdI32::from_array([-5, -4, -3, -2]);
-    /// let mask = MaskSize::from_array([true, true, true, false]); // Note the mask of the last lane.
-    ///
-    /// let result = SimdI32::<4>::gather_select(&vec, mask, idxs, alt); // Note the lane that is out-of-bounds.
-    /// assert_eq!(result, SimdI32::from_array([-5, 13, 10, -2]));
-    /// ```
-    #[must_use]
-    #[inline]
-    fn gather_select(
-        slice: &[Self::Scalar],
-        mask: MaskSize<LANES>,
-        idxs: SimdUsize<LANES>,
-        or: Self,
-    ) -> Self {
-        let mask = (mask & idxs.lanes_lt(SimdUsize::splat(slice.len()))).to_int();
-        let base_ptr = SimdConstPtr::splat(slice.as_ptr());
-        // Ferris forgive me, I have done pointer arithmetic here.
-        let ptrs = base_ptr.wrapping_add(idxs);
-        // SAFETY: The ptrs have been bounds-masked to prevent memory-unsafe reads insha'allah
-        unsafe { intrinsics::simd_gather(or, ptrs, mask) }
-    }
-
-    /// SIMD scatter: write a SIMD vector's values into a slice, using potentially discontiguous indices.
-    /// Out-of-bounds indices are not written.
-    /// `scatter` writes "in order", so if an index receives two writes, only the last is guaranteed.
-    /// ```
-    /// # #![feature(portable_simd)]
-    /// # use core_simd::*;
-    /// let mut vec: Vec<i32> = vec![10, 11, 12, 13, 14, 15, 16, 17, 18];
-    /// let idxs = SimdUsize::<4>::from_array([9, 3, 0, 0]);
-    /// let vals = SimdI32::from_array([-27, 82, -41, 124]);
-    ///
-    /// vals.scatter(&mut vec, idxs); // index 0 receives two writes.
-    /// assert_eq!(vec, vec![124, 11, 12, 82, 14, 15, 16, 17, 18]);
-    /// ```
-    #[inline]
-    fn scatter(self, slice: &mut [Self::Scalar], idxs: SimdUsize<LANES>) {
-        self.scatter_select(slice, MaskSize::splat(true), idxs)
-    }
-
-    /// SIMD scatter: write a SIMD vector's values into a slice, using potentially discontiguous indices.
-    /// Out-of-bounds or masked indices are not written.
-    /// `scatter_select` writes "in order", so if an index receives two writes, only the last is guaranteed.
-    /// ```
-    /// # #![feature(portable_simd)]
-    /// # use core_simd::*;
-    /// let mut vec: Vec<i32> = vec![10, 11, 12, 13, 14, 15, 16, 17, 18];
-    /// let idxs = SimdUsize::<4>::from_array([9, 3, 0, 0]);
-    /// let vals = SimdI32::from_array([-27, 82, -41, 124]);
-    /// let mask = MaskSize::from_array([true, true, true, false]); // Note the mask of the last lane.
-    ///
-    /// vals.scatter_select(&mut vec, mask, idxs); // index 0's second write is masked, thus omitted.
-    /// assert_eq!(vec, vec![-41, 11, 12, 82, 14, 15, 16, 17, 18]);
-    /// ```
-    #[inline]
-    fn scatter_select(
-        self,
-        slice: &mut [Self::Scalar],
-        mask: MaskSize<LANES>,
-        idxs: SimdUsize<LANES>,
-    ) {
-        // We must construct our scatter mask before we derive a pointer!
-        let mask = (mask & idxs.lanes_lt(SimdUsize::splat(slice.len()))).to_int();
-        // SAFETY: This block works with *mut T derived from &mut 'a [T],
-        // which means it is delicate in Rust's borrowing model, circa 2021:
-        // &mut 'a [T] asserts uniqueness, so deriving &'a [T] invalidates live *mut Ts!
-        // Even though this block is largely safe methods, it must be almost exactly this way
-        // to prevent invalidating the raw ptrs while they're live.
-        // Thus, entering this block requires all values to use being already ready:
-        // 0. idxs we want to write to, which are used to construct the mask.
-        // 1. mask, which depends on an initial &'a [T] and the idxs.
-        // 2. actual values to scatter (self).
-        // 3. &mut [T] which will become our base ptr.
-        unsafe {
-            // Now Entering ☢️ *mut T Zone
-            let base_ptr = SimdMutPtr::splat(slice.as_mut_ptr());
-            // Ferris forgive me, I have done pointer arithmetic here.
-            let ptrs = base_ptr.wrapping_add(idxs);
-            // The ptrs have been bounds-masked to prevent memory-unsafe writes insha'allah
-            intrinsics::simd_scatter(self, ptrs, mask)
-            // Cleared ☢️ *mut T Zone
-        }
-    }
-}
-
-macro_rules! impl_simdarray_for {
-    ($simd:ident {type Scalar = $scalar:ident;}) => {
-        impl<const LANES: usize> SimdArray<LANES> for $simd<LANES>
-            where SimdUsize<LANES>: crate::LanesAtMost32,
-            SimdIsize<LANES>: crate::LanesAtMost32,
-            MaskSize<LANES>: crate::Mask,
-            Self: crate::LanesAtMost32,
-        {
-            type Scalar = $scalar;
-
-            #[must_use]
-            #[inline]
-            fn splat(val: Self::Scalar) -> Self {
-                [val; LANES].into()
-            }
-        }
-    };
-
-    ($simd:ident $impl:tt) => {
-        impl<const LANES: usize> SimdArray<LANES> for $simd<LANES>
-            where SimdUsize<LANES>: crate::LanesAtMost32,
-            SimdIsize<LANES>: crate::LanesAtMost32,
-            MaskSize<LANES>: crate::Mask,
-            Self: crate::LanesAtMost32,
-        $impl
-    }
-}
-
-impl_simdarray_for! {
-    SimdUsize {
-        type Scalar = usize;
-    }
-}
-
-impl_simdarray_for! {
-    SimdIsize {
-        type Scalar = isize;
-    }
-}
-
-impl_simdarray_for! {
-    SimdI8 {
-        type Scalar = i8;
-    }
-}
-
-impl_simdarray_for! {
-    SimdI16 {
-        type Scalar = i16;
-    }
-}
-
-impl_simdarray_for! {
-    SimdI32 {
-        type Scalar = i32;
-    }
-}
-
-impl_simdarray_for! {
-    SimdI64 {
-        type Scalar = i64;
-    }
-}
-
-impl_simdarray_for! {
-    SimdU8 {
-        type Scalar = u8;
-    }
-}
-
-impl_simdarray_for! {
-    SimdU16 {
-        type Scalar = u16;
-    }
-}
-
-impl_simdarray_for! {
-    SimdU32 {
-        type Scalar = u32;
-    }
-}
-
-impl_simdarray_for! {
-    SimdU64 {
-        type Scalar = u64;
-    }
-}
-
-impl_simdarray_for! {
-    SimdF32 {
-        type Scalar = f32;
-    }
-}
-
-impl_simdarray_for! {
-    SimdF64 {
-        type Scalar = f64;
-    }
-}
diff --git a/crates/core_simd/src/comparisons.rs b/crates/core_simd/src/comparisons.rs
index e8d11406c097..c3bf07fc4320 100644
--- a/crates/core_simd/src/comparisons.rs
+++ b/crates/core_simd/src/comparisons.rs
@@ -1,12 +1,12 @@
-use crate::LanesAtMost32;
+use crate::Vector;
 
 macro_rules! implement_mask_ops {
     { $($vector:ident => $mask:ident ($inner_ty:ident),)* } => {
         $(
             impl<const LANES: usize> crate::$vector<LANES>
             where
-                crate::$vector<LANES>: LanesAtMost32,
-                crate::$inner_ty<LANES>: LanesAtMost32,
+                crate::$vector<LANES>: Vector,
+                crate::$inner_ty<LANES>: Vector,
                 crate::$mask<LANES>: crate::Mask,
             {
                 /// Test if each lane is equal to the corresponding lane in `other`.
diff --git a/crates/core_simd/src/first.rs b/crates/core_simd/src/first.rs
index 50602829d482..7721b87ecc0b 100644
--- a/crates/core_simd/src/first.rs
+++ b/crates/core_simd/src/first.rs
@@ -1,7 +1,7 @@
 /// Implements common traits on the specified vector `$name`, holding multiple `$lanes` of `$type`.
 macro_rules! impl_vector {
     { $name:ident, $type:ty } => {
-        impl<const LANES: usize> $name<LANES> where Self: crate::LanesAtMost32 {
+        impl<const LANES: usize> $name<LANES> where Self: crate::Vector {
             /// Construct a SIMD vector by setting all lanes to the given value.
             pub const fn splat(value: $type) -> Self {
                 Self([value; LANES])
@@ -44,23 +44,159 @@ macro_rules! impl_vector {
             }
         }
 
-        impl<const LANES: usize> Copy for $name<LANES> where Self: crate::LanesAtMost32 {}
+        impl<const LANES: usize> $name<LANES>
+        where
+            Self: crate::Vector,
+            crate::MaskSize<LANES>: crate::Mask,
+            crate::SimdIsize<LANES>: crate::Vector,
+            crate::SimdUsize<LANES>: crate::Vector,
+        {
+            /// SIMD gather: construct a SIMD vector by reading from a slice, using potentially discontiguous indices.
+            /// If an index is out of bounds, that lane instead selects the value from the "or" vector.
+            /// ```
+            /// # #![feature(portable_simd)]
+            /// # use core_simd::*;
+            /// let vec: Vec<i32> = vec![10, 11, 12, 13, 14, 15, 16, 17, 18];
+            /// let idxs = SimdUsize::<4>::from_array([9, 3, 0, 5]);
+            /// let alt = SimdI32::from_array([-5, -4, -3, -2]);
+            ///
+            /// let result = SimdI32::<4>::gather_or(&vec, idxs, alt); // Note the lane that is out-of-bounds.
+            /// assert_eq!(result, SimdI32::from_array([-5, 13, 10, 15]));
+            /// ```
+            #[must_use]
+            #[inline]
+            pub fn gather_or(slice: &[$type], idxs: crate::SimdUsize<LANES>, or: Self) -> Self {
+                Self::gather_select(slice, crate::MaskSize::splat(true), idxs, or)
+            }
 
-        impl<const LANES: usize> Clone for $name<LANES> where Self: crate::LanesAtMost32 {
+            /// SIMD gather: construct a SIMD vector by reading from a slice, using potentially discontiguous indices.
+            /// Out-of-bounds indices instead use the default value for that lane (0).
+            /// ```
+            /// # #![feature(portable_simd)]
+            /// # use core_simd::*;
+            /// let vec: Vec<i32> = vec![10, 11, 12, 13, 14, 15, 16, 17, 18];
+            /// let idxs = SimdUsize::<4>::from_array([9, 3, 0, 5]);
+            ///
+            /// let result = SimdI32::<4>::gather_or_default(&vec, idxs); // Note the lane that is out-of-bounds.
+            /// assert_eq!(result, SimdI32::from_array([0, 13, 10, 15]));
+            /// ```
+            #[must_use]
+            #[inline]
+            pub fn gather_or_default(slice: &[$type], idxs: crate::SimdUsize<LANES>) -> Self {
+                Self::gather_or(slice, idxs, Self::splat(<$type>::default()))
+            }
+
+            /// SIMD gather: construct a SIMD vector by reading from a slice, using potentially discontiguous indices.
+            /// Out-of-bounds or masked indices instead select the value from the "or" vector.
+            /// ```
+            /// # #![feature(portable_simd)]
+            /// # use core_simd::*;
+            /// let vec: Vec<i32> = vec![10, 11, 12, 13, 14, 15, 16, 17, 18];
+            /// let idxs = SimdUsize::<4>::from_array([9, 3, 0, 5]);
+            /// let alt = SimdI32::from_array([-5, -4, -3, -2]);
+            /// let mask = MaskSize::from_array([true, true, true, false]); // Note the mask of the last lane.
+            ///
+            /// let result = SimdI32::<4>::gather_select(&vec, mask, idxs, alt); // Note the lane that is out-of-bounds.
+            /// assert_eq!(result, SimdI32::from_array([-5, 13, 10, -2]));
+            /// ```
+            #[must_use]
+            #[inline]
+            pub fn gather_select(
+                slice: &[$type],
+                mask: crate::MaskSize<LANES>,
+                idxs: crate::SimdUsize<LANES>,
+                or: Self,
+            ) -> Self
+            {
+                let mask = (mask & idxs.lanes_lt(crate::SimdUsize::splat(slice.len()))).to_int();
+                let base_ptr = crate::vector::ptr::SimdConstPtr::splat(slice.as_ptr());
+                // Ferris forgive me, I have done pointer arithmetic here.
+                let ptrs = base_ptr.wrapping_add(idxs);
+                // SAFETY: The ptrs have been bounds-masked to prevent memory-unsafe reads insha'allah
+                unsafe { crate::intrinsics::simd_gather(or, ptrs, mask) }
+            }
+
+            /// SIMD scatter: write a SIMD vector's values into a slice, using potentially discontiguous indices.
+            /// Out-of-bounds indices are not written.
+            /// `scatter` writes "in order", so if an index receives two writes, only the last is guaranteed.
+            /// ```
+            /// # #![feature(portable_simd)]
+            /// # use core_simd::*;
+            /// let mut vec: Vec<i32> = vec![10, 11, 12, 13, 14, 15, 16, 17, 18];
+            /// let idxs = SimdUsize::<4>::from_array([9, 3, 0, 0]);
+            /// let vals = SimdI32::from_array([-27, 82, -41, 124]);
+            ///
+            /// vals.scatter(&mut vec, idxs); // index 0 receives two writes.
+            /// assert_eq!(vec, vec![124, 11, 12, 82, 14, 15, 16, 17, 18]);
+            /// ```
+            #[inline]
+            pub fn scatter(self, slice: &mut [$type], idxs: crate::SimdUsize<LANES>) {
+                self.scatter_select(slice, crate::MaskSize::splat(true), idxs)
+            }
+
+            /// SIMD scatter: write a SIMD vector's values into a slice, using potentially discontiguous indices.
+            /// Out-of-bounds or masked indices are not written.
+            /// `scatter_select` writes "in order", so if an index receives two writes, only the last is guaranteed.
+            /// ```
+            /// # #![feature(portable_simd)]
+            /// # use core_simd::*;
+            /// let mut vec: Vec<i32> = vec![10, 11, 12, 13, 14, 15, 16, 17, 18];
+            /// let idxs = SimdUsize::<4>::from_array([9, 3, 0, 0]);
+            /// let vals = SimdI32::from_array([-27, 82, -41, 124]);
+            /// let mask = MaskSize::from_array([true, true, true, false]); // Note the mask of the last lane.
+            ///
+            /// vals.scatter_select(&mut vec, mask, idxs); // index 0's second write is masked, thus omitted.
+            /// assert_eq!(vec, vec![-41, 11, 12, 82, 14, 15, 16, 17, 18]);
+            /// ```
+            #[inline]
+            pub fn scatter_select(
+                self,
+                slice: &mut [$type],
+                mask: crate::MaskSize<LANES>,
+                idxs: crate::SimdUsize<LANES>,
+            )
+            {
+                // We must construct our scatter mask before we derive a pointer!
+                let mask = (mask & idxs.lanes_lt(crate::SimdUsize::splat(slice.len()))).to_int();
+                // SAFETY: This block works with *mut T derived from &mut 'a [T],
+                // which means it is delicate in Rust's borrowing model, circa 2021:
+                // &mut 'a [T] asserts uniqueness, so deriving &'a [T] invalidates live *mut Ts!
+                // Even though this block is largely safe methods, it must be almost exactly this way
+                // to prevent invalidating the raw ptrs while they're live.
+                // Thus, entering this block requires all values to use being already ready:
+                // 0. idxs we want to write to, which are used to construct the mask.
+                // 1. mask, which depends on an initial &'a [T] and the idxs.
+                // 2. actual values to scatter (self).
+                // 3. &mut [T] which will become our base ptr.
+                unsafe {
+                    // Now Entering ☢️ *mut T Zone
+                    let base_ptr = crate::vector::ptr::SimdMutPtr::splat(slice.as_mut_ptr());
+                    // Ferris forgive me, I have done pointer arithmetic here.
+                    let ptrs = base_ptr.wrapping_add(idxs);
+                    // The ptrs have been bounds-masked to prevent memory-unsafe writes insha'allah
+                    crate::intrinsics::simd_scatter(self, ptrs, mask)
+                    // Cleared ☢️ *mut T Zone
+                }
+            }
+        }
+
+        impl<const LANES: usize> Copy for $name<LANES> where Self: crate::Vector {}
+
+        impl<const LANES: usize> Clone for $name<LANES> where Self: crate::Vector {
             #[inline]
             fn clone(&self) -> Self {
                 *self
             }
         }
 
-        impl<const LANES: usize> Default for $name<LANES> where Self: crate::LanesAtMost32 {
+        impl<const LANES: usize> Default for $name<LANES> where Self: crate::Vector {
             #[inline]
             fn default() -> Self {
                 Self::splat(<$type>::default())
             }
         }
 
-        impl<const LANES: usize> PartialEq for $name<LANES> where Self: crate::LanesAtMost32 {
+        impl<const LANES: usize> PartialEq for $name<LANES> where Self: crate::Vector {
             #[inline]
             fn eq(&self, other: &Self) -> bool {
                 // TODO use SIMD equality
@@ -68,7 +204,7 @@ macro_rules! impl_vector {
             }
         }
 
-        impl<const LANES: usize> PartialOrd for $name<LANES> where Self: crate::LanesAtMost32 {
+        impl<const LANES: usize> PartialOrd for $name<LANES> where Self: crate::Vector {
             #[inline]
             fn partial_cmp(&self, other: &Self) -> Option<core::cmp::Ordering> {
                 // TODO use SIMD equalitya
@@ -77,14 +213,14 @@ macro_rules! impl_vector {
         }
 
         // array references
-        impl<const LANES: usize> AsRef<[$type; LANES]> for $name<LANES> where Self: crate::LanesAtMost32 {
+        impl<const LANES: usize> AsRef<[$type; LANES]> for $name<LANES> where Self: crate::Vector {
             #[inline]
             fn as_ref(&self) -> &[$type; LANES] {
                 &self.0
             }
         }
 
-        impl<const LANES: usize> AsMut<[$type; LANES]> for $name<LANES> where Self: crate::LanesAtMost32 {
+        impl<const LANES: usize> AsMut<[$type; LANES]> for $name<LANES> where Self: crate::Vector {
             #[inline]
             fn as_mut(&mut self) -> &mut [$type; LANES] {
                 &mut self.0
@@ -92,14 +228,14 @@ macro_rules! impl_vector {
         }
 
         // slice references
-        impl<const LANES: usize> AsRef<[$type]> for $name<LANES> where Self: crate::LanesAtMost32 {
+        impl<const LANES: usize> AsRef<[$type]> for $name<LANES> where Self: crate::Vector {
             #[inline]
             fn as_ref(&self) -> &[$type] {
                 &self.0
             }
         }
 
-        impl<const LANES: usize> AsMut<[$type]> for $name<LANES> where Self: crate::LanesAtMost32 {
+        impl<const LANES: usize> AsMut<[$type]> for $name<LANES> where Self: crate::Vector {
             #[inline]
             fn as_mut(&mut self) -> &mut [$type] {
                 &mut self.0
@@ -107,13 +243,13 @@ macro_rules! impl_vector {
         }
 
         // vector/array conversion
-        impl<const LANES: usize> From<[$type; LANES]> for $name<LANES> where Self: crate::LanesAtMost32 {
+        impl<const LANES: usize> From<[$type; LANES]> for $name<LANES> where Self: crate::Vector {
             fn from(array: [$type; LANES]) -> Self {
                 Self(array)
             }
         }
 
-        impl <const LANES: usize> From<$name<LANES>> for [$type; LANES] where $name<LANES>: crate::LanesAtMost32 {
+        impl <const LANES: usize> From<$name<LANES>> for [$type; LANES] where $name<LANES>: crate::Vector {
             fn from(vector: $name<LANES>) -> Self {
                 vector.to_array()
             }
diff --git a/crates/core_simd/src/fmt.rs b/crates/core_simd/src/fmt.rs
index 1d5010843eb7..554aa91a5f0c 100644
--- a/crates/core_simd/src/fmt.rs
+++ b/crates/core_simd/src/fmt.rs
@@ -35,7 +35,7 @@ macro_rules! impl_fmt_trait {
             $( // repeat trait
                 impl<const LANES: usize> core::fmt::$trait for crate::$type<LANES>
                 where
-                    Self: crate::LanesAtMost32,
+                    Self: crate::Vector,
                 {
                     fn fmt(&self, f: &mut core::fmt::Formatter) -> core::fmt::Result {
                         $format(self.as_ref(), f)
diff --git a/crates/core_simd/src/iter.rs b/crates/core_simd/src/iter.rs
index c1c4c645db6b..460c061be69f 100644
--- a/crates/core_simd/src/iter.rs
+++ b/crates/core_simd/src/iter.rs
@@ -2,7 +2,7 @@ macro_rules! impl_traits {
     { $type:ident } => {
         impl<const LANES: usize> core::iter::Sum<Self> for crate::$type<LANES>
         where
-            Self: crate::LanesAtMost32,
+            Self: crate::Vector,
         {
             fn sum<I: core::iter::Iterator<Item = Self>>(iter: I) -> Self {
                 iter.fold(Default::default(), core::ops::Add::add)
@@ -11,7 +11,7 @@ macro_rules! impl_traits {
 
         impl<const LANES: usize> core::iter::Product<Self> for crate::$type<LANES>
         where
-            Self: crate::LanesAtMost32,
+            Self: crate::Vector,
         {
             fn product<I: core::iter::Iterator<Item = Self>>(iter: I) -> Self {
                 iter.fold(Default::default(), core::ops::Mul::mul)
@@ -20,7 +20,7 @@ macro_rules! impl_traits {
 
         impl<'a, const LANES: usize> core::iter::Sum<&'a Self> for crate::$type<LANES>
         where
-            Self: crate::LanesAtMost32,
+            Self: crate::Vector,
         {
             fn sum<I: core::iter::Iterator<Item = &'a Self>>(iter: I) -> Self {
                 iter.fold(Default::default(), core::ops::Add::add)
@@ -29,7 +29,7 @@ macro_rules! impl_traits {
 
         impl<'a, const LANES: usize> core::iter::Product<&'a Self> for crate::$type<LANES>
         where
-            Self: crate::LanesAtMost32,
+            Self: crate::Vector,
         {
             fn product<I: core::iter::Iterator<Item = &'a Self>>(iter: I) -> Self {
                 iter.fold(Default::default(), core::ops::Mul::mul)
diff --git a/crates/core_simd/src/lanes_at_most_32.rs b/crates/core_simd/src/lanes_at_most_32.rs
deleted file mode 100644
index 2d84b1306ea5..000000000000
--- a/crates/core_simd/src/lanes_at_most_32.rs
+++ /dev/null
@@ -1,54 +0,0 @@
-/// Implemented for vectors that are supported by the implementation.
-pub trait LanesAtMost32: sealed::Sealed {
-    #[doc(hidden)]
-    type BitMask: Into<u64>;
-}
-
-mod sealed {
-    pub trait Sealed {}
-}
-
-macro_rules! impl_for {
-    { $name:ident } => {
-        impl<const LANES: usize> sealed::Sealed for $name<LANES>
-        where
-            $name<LANES>: LanesAtMost32,
-        {}
-
-        impl LanesAtMost32 for $name<1> {
-            type BitMask = u8;
-        }
-        impl LanesAtMost32 for $name<2> {
-            type BitMask = u8;
-        }
-        impl LanesAtMost32 for $name<4> {
-            type BitMask = u8;
-        }
-        impl LanesAtMost32 for $name<8> {
-            type BitMask = u8;
-        }
-        impl LanesAtMost32 for $name<16> {
-            type BitMask = u16;
-        }
-        impl LanesAtMost32 for $name<32> {
-            type BitMask = u32;
-        }
-    }
-}
-
-use crate::*;
-
-impl_for! { SimdU8 }
-impl_for! { SimdU16 }
-impl_for! { SimdU32 }
-impl_for! { SimdU64 }
-impl_for! { SimdUsize }
-
-impl_for! { SimdI8 }
-impl_for! { SimdI16 }
-impl_for! { SimdI32 }
-impl_for! { SimdI64 }
-impl_for! { SimdIsize }
-
-impl_for! { SimdF32 }
-impl_for! { SimdF64 }
diff --git a/crates/core_simd/src/lib.rs b/crates/core_simd/src/lib.rs
index e48f8062d2c8..eb580dcf14d8 100644
--- a/crates/core_simd/src/lib.rs
+++ b/crates/core_simd/src/lib.rs
@@ -35,14 +35,8 @@ mod vendor;
 
 mod math;
 
-mod lanes_at_most_32;
-pub use lanes_at_most_32::LanesAtMost32;
-
 mod masks;
 pub use masks::*;
 
 mod vector;
 pub use vector::*;
-
-mod array;
-pub use array::SimdArray;
diff --git a/crates/core_simd/src/masks/bitmask.rs b/crates/core_simd/src/masks/bitmask.rs
index fab136d2b241..a64750a623f7 100644
--- a/crates/core_simd/src/masks/bitmask.rs
+++ b/crates/core_simd/src/masks/bitmask.rs
@@ -3,11 +3,11 @@ use core::marker::PhantomData;
 
 /// Helper trait for limiting int conversion types
 pub trait ConvertToInt {}
-impl<const LANES: usize> ConvertToInt for crate::SimdI8<LANES> where Self: crate::LanesAtMost32 {}
-impl<const LANES: usize> ConvertToInt for crate::SimdI16<LANES> where Self: crate::LanesAtMost32 {}
-impl<const LANES: usize> ConvertToInt for crate::SimdI32<LANES> where Self: crate::LanesAtMost32 {}
-impl<const LANES: usize> ConvertToInt for crate::SimdI64<LANES> where Self: crate::LanesAtMost32 {}
-impl<const LANES: usize> ConvertToInt for crate::SimdIsize<LANES> where Self: crate::LanesAtMost32 {}
+impl<const LANES: usize> ConvertToInt for crate::SimdI8<LANES> where Self: crate::Vector {}
+impl<const LANES: usize> ConvertToInt for crate::SimdI16<LANES> where Self: crate::Vector {}
+impl<const LANES: usize> ConvertToInt for crate::SimdI32<LANES> where Self: crate::Vector {}
+impl<const LANES: usize> ConvertToInt for crate::SimdI64<LANES> where Self: crate::Vector {}
+impl<const LANES: usize> ConvertToInt for crate::SimdIsize<LANES> where Self: crate::Vector {}
 
 /// A mask where each lane is represented by a single bit.
 #[repr(transparent)]
@@ -80,7 +80,7 @@ impl<T: Mask, const LANES: usize> BitMask<T, LANES> {
     #[inline]
     pub unsafe fn from_int_unchecked<V>(value: V) -> Self
     where
-        V: crate::LanesAtMost32,
+        V: crate::Vector,
     {
         // TODO remove the transmute when rustc is more flexible
         assert_eq!(
@@ -184,8 +184,8 @@ macro_rules! impl_from {
         $(
         impl<const LANES: usize> From<$from<crate::$from<LANES>, LANES>> for $to<crate::$to<LANES>, LANES>
         where
-            crate::$from_inner<LANES>: crate::LanesAtMost32,
-            crate::$to_inner<LANES>: crate::LanesAtMost32,
+            crate::$from_inner<LANES>: crate::Vector,
+            crate::$to_inner<LANES>: crate::Vector,
             crate::$from<LANES>: crate::Mask,
             crate::$to<LANES>: crate::Mask,
         {
diff --git a/crates/core_simd/src/masks/full_masks.rs b/crates/core_simd/src/masks/full_masks.rs
index 7d98333ef607..f083284df8fa 100644
--- a/crates/core_simd/src/masks/full_masks.rs
+++ b/crates/core_simd/src/masks/full_masks.rs
@@ -14,18 +14,18 @@ macro_rules! define_mask {
         #[repr(transparent)]
         pub struct $name<T: Mask, const $lanes: usize>(crate::$type<$lanes2>, PhantomData<T>)
         where
-            crate::$type<LANES>: crate::LanesAtMost32;
+            crate::$type<LANES>: crate::Vector;
 
         impl_full_mask_reductions! { $name, $type }
 
         impl<T: Mask, const LANES: usize> Copy for $name<T, LANES>
         where
-            crate::$type<LANES>: crate::LanesAtMost32,
+            crate::$type<LANES>: crate::Vector,
         {}
 
         impl<T: Mask, const LANES: usize> Clone for $name<T, LANES>
         where
-            crate::$type<LANES>: crate::LanesAtMost32,
+            crate::$type<LANES>: crate::Vector,
         {
             #[inline]
             fn clone(&self) -> Self {
@@ -35,7 +35,7 @@ macro_rules! define_mask {
 
         impl<T: Mask, const LANES: usize> PartialEq for $name<T, LANES>
         where
-            crate::$type<LANES>: crate::LanesAtMost32,
+            crate::$type<LANES>: crate::Vector,
         {
             fn eq(&self, other: &Self) -> bool {
                 self.0 == other.0
@@ -44,7 +44,7 @@ macro_rules! define_mask {
 
         impl<T: Mask, const LANES: usize> PartialOrd for $name<T, LANES>
         where
-            crate::$type<LANES>: crate::LanesAtMost32,
+            crate::$type<LANES>: crate::Vector,
         {
             fn partial_cmp(&self, other: &Self) -> Option<core::cmp::Ordering> {
                 self.0.partial_cmp(&other.0)
@@ -53,12 +53,12 @@ macro_rules! define_mask {
 
         impl<T: Mask, const LANES: usize> Eq for $name<T, LANES>
         where
-            crate::$type<LANES>: crate::LanesAtMost32,
+            crate::$type<LANES>: crate::Vector,
         {}
 
         impl<T: Mask, const LANES: usize> Ord for $name<T, LANES>
         where
-            crate::$type<LANES>: crate::LanesAtMost32,
+            crate::$type<LANES>: crate::Vector,
         {
             fn cmp(&self, other: &Self) -> core::cmp::Ordering {
                 self.0.cmp(&other.0)
@@ -67,7 +67,7 @@ macro_rules! define_mask {
 
         impl<T: Mask, const LANES: usize> $name<T, LANES>
         where
-            crate::$type<LANES>: crate::LanesAtMost32,
+            crate::$type<LANES>: crate::Vector,
         {
             pub fn splat(value: bool) -> Self {
                 Self(
@@ -154,7 +154,7 @@ macro_rules! define_mask {
 
         impl<T: Mask, const LANES: usize> core::convert::From<$name<T, LANES>> for crate::$type<LANES>
         where
-            crate::$type<LANES>: crate::LanesAtMost32,
+            crate::$type<LANES>: crate::Vector,
         {
             fn from(value: $name<T, LANES>) -> Self {
                 value.0
@@ -163,7 +163,7 @@ macro_rules! define_mask {
 
         impl<T: Mask, const LANES: usize> core::ops::BitAnd for $name<T, LANES>
         where
-            crate::$type<LANES>: crate::LanesAtMost32,
+            crate::$type<LANES>: crate::Vector,
         {
             type Output = Self;
             #[inline]
@@ -174,7 +174,7 @@ macro_rules! define_mask {
 
         impl<T: Mask, const LANES: usize> core::ops::BitOr for $name<T, LANES>
         where
-            crate::$type<LANES>: crate::LanesAtMost32,
+            crate::$type<LANES>: crate::Vector,
         {
             type Output = Self;
             #[inline]
@@ -185,7 +185,7 @@ macro_rules! define_mask {
 
         impl<T: Mask, const LANES: usize> core::ops::BitXor for $name<T, LANES>
         where
-            crate::$type<LANES>: crate::LanesAtMost32,
+            crate::$type<LANES>: crate::Vector,
         {
             type Output = Self;
             #[inline]
@@ -196,7 +196,7 @@ macro_rules! define_mask {
 
         impl<T: Mask, const LANES: usize> core::ops::Not for $name<T, LANES>
         where
-            crate::$type<LANES>: crate::LanesAtMost32,
+            crate::$type<LANES>: crate::Vector,
         {
             type Output = Self;
             #[inline]
@@ -242,8 +242,8 @@ macro_rules! impl_from {
         $(
         impl<const LANES: usize, T, U> From<$from<T, LANES>> for $to<U, LANES>
         where
-            crate::$from_inner<LANES>: crate::LanesAtMost32,
-            crate::$to_inner<LANES>: crate::LanesAtMost32,
+            crate::$from_inner<LANES>: crate::Vector,
+            crate::$to_inner<LANES>: crate::Vector,
             T: crate::Mask,
             U: crate::Mask,
         {
diff --git a/crates/core_simd/src/masks/mod.rs b/crates/core_simd/src/masks/mod.rs
index 1d6b2e45224a..43e689d45bc7 100644
--- a/crates/core_simd/src/masks/mod.rs
+++ b/crates/core_simd/src/masks/mod.rs
@@ -12,7 +12,7 @@
 )]
 mod mask_impl;
 
-use crate::{LanesAtMost32, SimdI16, SimdI32, SimdI64, SimdI8, SimdIsize};
+use crate::{SimdI16, SimdI32, SimdI64, SimdI8, SimdIsize, Vector};
 
 mod sealed {
     pub trait Sealed {}
@@ -38,12 +38,12 @@ macro_rules! define_opaque_mask {
         #[allow(non_camel_case_types)]
         pub struct $name<const LANES: usize>($inner_ty)
         where
-            $bits_ty<LANES>: LanesAtMost32,
+            $bits_ty<LANES>: Vector,
             Self: Mask;
 
         impl<const LANES: usize> sealed::Sealed for $name<LANES>
         where
-            $bits_ty<LANES>: LanesAtMost32,
+            $bits_ty<LANES>: Vector,
             Self: Mask,
         {}
         impl Mask for $name<1> {
@@ -75,7 +75,7 @@ macro_rules! define_opaque_mask {
 
         impl<const LANES: usize> $name<LANES>
         where
-            $bits_ty<LANES>: LanesAtMost32,
+            $bits_ty<LANES>: Vector,
             Self: Mask,
         {
             /// Construct a mask by setting all lanes to the given value.
@@ -188,7 +188,7 @@ macro_rules! define_opaque_mask {
         // vector/array conversion
         impl<const LANES: usize> From<[bool; LANES]> for $name<LANES>
         where
-            $bits_ty<LANES>: crate::LanesAtMost32,
+            $bits_ty<LANES>: crate::Vector,
             Self: Mask,
         {
             fn from(array: [bool; LANES]) -> Self {
@@ -198,7 +198,7 @@ macro_rules! define_opaque_mask {
 
         impl <const LANES: usize> From<$name<LANES>> for [bool; LANES]
         where
-            $bits_ty<LANES>: crate::LanesAtMost32,
+            $bits_ty<LANES>: crate::Vector,
             $name<LANES>: Mask,
         {
             fn from(vector: $name<LANES>) -> Self {
@@ -208,13 +208,13 @@ macro_rules! define_opaque_mask {
 
         impl<const LANES: usize> Copy for $name<LANES>
         where
-            $bits_ty<LANES>: LanesAtMost32,
+            $bits_ty<LANES>: Vector,
             Self: Mask,
         {}
 
         impl<const LANES: usize> Clone for $name<LANES>
         where
-            $bits_ty<LANES>: LanesAtMost32,
+            $bits_ty<LANES>: Vector,
             Self: Mask,
         {
             #[inline]
@@ -225,7 +225,7 @@ macro_rules! define_opaque_mask {
 
         impl<const LANES: usize> Default for $name<LANES>
         where
-            $bits_ty<LANES>: LanesAtMost32,
+            $bits_ty<LANES>: Vector,
             Self: Mask,
         {
             #[inline]
@@ -236,7 +236,7 @@ macro_rules! define_opaque_mask {
 
         impl<const LANES: usize> PartialEq for $name<LANES>
         where
-            $bits_ty<LANES>: LanesAtMost32,
+            $bits_ty<LANES>: Vector,
             Self: Mask,
         {
             #[inline]
@@ -247,7 +247,7 @@ macro_rules! define_opaque_mask {
 
         impl<const LANES: usize> PartialOrd for $name<LANES>
         where
-            $bits_ty<LANES>: LanesAtMost32,
+            $bits_ty<LANES>: Vector,
             Self: Mask,
         {
             #[inline]
@@ -258,7 +258,7 @@ macro_rules! define_opaque_mask {
 
         impl<const LANES: usize> core::fmt::Debug for $name<LANES>
         where
-            $bits_ty<LANES>: crate::LanesAtMost32,
+            $bits_ty<LANES>: crate::Vector,
             Self: Mask,
         {
             fn fmt(&self, f: &mut core::fmt::Formatter) -> core::fmt::Result {
@@ -270,7 +270,7 @@ macro_rules! define_opaque_mask {
 
         impl<const LANES: usize> core::ops::BitAnd for $name<LANES>
         where
-            $bits_ty<LANES>: LanesAtMost32,
+            $bits_ty<LANES>: Vector,
             Self: Mask,
         {
             type Output = Self;
@@ -282,7 +282,7 @@ macro_rules! define_opaque_mask {
 
         impl<const LANES: usize> core::ops::BitAnd<bool> for $name<LANES>
         where
-            $bits_ty<LANES>: LanesAtMost32,
+            $bits_ty<LANES>: Vector,
             Self: Mask,
         {
             type Output = Self;
@@ -294,7 +294,7 @@ macro_rules! define_opaque_mask {
 
         impl<const LANES: usize> core::ops::BitAnd<$name<LANES>> for bool
         where
-            $bits_ty<LANES>: LanesAtMost32,
+            $bits_ty<LANES>: Vector,
             $name<LANES>: Mask,
         {
             type Output = $name<LANES>;
@@ -306,7 +306,7 @@ macro_rules! define_opaque_mask {
 
         impl<const LANES: usize> core::ops::BitOr for $name<LANES>
         where
-            $bits_ty<LANES>: LanesAtMost32,
+            $bits_ty<LANES>: Vector,
             Self: Mask,
         {
             type Output = Self;
@@ -318,7 +318,7 @@ macro_rules! define_opaque_mask {
 
         impl<const LANES: usize> core::ops::BitOr<bool> for $name<LANES>
         where
-            $bits_ty<LANES>: LanesAtMost32,
+            $bits_ty<LANES>: Vector,
             Self: Mask,
         {
             type Output = Self;
@@ -330,7 +330,7 @@ macro_rules! define_opaque_mask {
 
         impl<const LANES: usize> core::ops::BitOr<$name<LANES>> for bool
         where
-            $bits_ty<LANES>: LanesAtMost32,
+            $bits_ty<LANES>: Vector,
             $name<LANES>: Mask,
         {
             type Output = $name<LANES>;
@@ -342,7 +342,7 @@ macro_rules! define_opaque_mask {
 
         impl<const LANES: usize> core::ops::BitXor for $name<LANES>
         where
-            $bits_ty<LANES>: LanesAtMost32,
+            $bits_ty<LANES>: Vector,
             Self: Mask,
         {
             type Output = Self;
@@ -354,7 +354,7 @@ macro_rules! define_opaque_mask {
 
         impl<const LANES: usize> core::ops::BitXor<bool> for $name<LANES>
         where
-            $bits_ty<LANES>: LanesAtMost32,
+            $bits_ty<LANES>: Vector,
             Self: Mask,
         {
             type Output = Self;
@@ -366,7 +366,7 @@ macro_rules! define_opaque_mask {
 
         impl<const LANES: usize> core::ops::BitXor<$name<LANES>> for bool
         where
-            $bits_ty<LANES>: LanesAtMost32,
+            $bits_ty<LANES>: Vector,
             $name<LANES>: Mask,
         {
             type Output = $name<LANES>;
@@ -378,7 +378,7 @@ macro_rules! define_opaque_mask {
 
         impl<const LANES: usize> core::ops::Not for $name<LANES>
         where
-            $bits_ty<LANES>: LanesAtMost32,
+            $bits_ty<LANES>: Vector,
             Self: Mask,
         {
             type Output = $name<LANES>;
@@ -390,7 +390,7 @@ macro_rules! define_opaque_mask {
 
         impl<const LANES: usize> core::ops::BitAndAssign for $name<LANES>
         where
-            $bits_ty<LANES>: LanesAtMost32,
+            $bits_ty<LANES>: Vector,
             Self: Mask,
         {
             #[inline]
@@ -401,7 +401,7 @@ macro_rules! define_opaque_mask {
 
         impl<const LANES: usize> core::ops::BitAndAssign<bool> for $name<LANES>
         where
-            $bits_ty<LANES>: LanesAtMost32,
+            $bits_ty<LANES>: Vector,
             Self: Mask,
         {
             #[inline]
@@ -412,7 +412,7 @@ macro_rules! define_opaque_mask {
 
         impl<const LANES: usize> core::ops::BitOrAssign for $name<LANES>
         where
-            $bits_ty<LANES>: LanesAtMost32,
+            $bits_ty<LANES>: Vector,
             Self: Mask,
         {
             #[inline]
@@ -423,7 +423,7 @@ macro_rules! define_opaque_mask {
 
         impl<const LANES: usize> core::ops::BitOrAssign<bool> for $name<LANES>
         where
-            $bits_ty<LANES>: LanesAtMost32,
+            $bits_ty<LANES>: Vector,
             Self: Mask,
         {
             #[inline]
@@ -434,7 +434,7 @@ macro_rules! define_opaque_mask {
 
         impl<const LANES: usize> core::ops::BitXorAssign for $name<LANES>
         where
-            $bits_ty<LANES>: LanesAtMost32,
+            $bits_ty<LANES>: Vector,
             Self: Mask,
         {
             #[inline]
@@ -445,7 +445,7 @@ macro_rules! define_opaque_mask {
 
         impl<const LANES: usize> core::ops::BitXorAssign<bool> for $name<LANES>
         where
-            $bits_ty<LANES>: LanesAtMost32,
+            $bits_ty<LANES>: Vector,
             Self: Mask,
         {
             #[inline]
@@ -555,8 +555,8 @@ macro_rules! impl_from {
         $(
         impl<const LANES: usize> From<$from<LANES>> for $to<LANES>
         where
-            crate::$from_inner<LANES>: crate::LanesAtMost32,
-            crate::$to_inner<LANES>: crate::LanesAtMost32,
+            crate::$from_inner<LANES>: crate::Vector,
+            crate::$to_inner<LANES>: crate::Vector,
             $from<LANES>: Mask,
             Self: Mask,
         {
diff --git a/crates/core_simd/src/math.rs b/crates/core_simd/src/math.rs
index 7290a28362f9..cc0330871136 100644
--- a/crates/core_simd/src/math.rs
+++ b/crates/core_simd/src/math.rs
@@ -1,6 +1,6 @@
 macro_rules! impl_uint_arith {
     ($(($name:ident, $n:ident)),+) => {
-        $( impl<const LANES: usize> $name<LANES> where Self: crate::LanesAtMost32 {
+        $( impl<const LANES: usize> $name<LANES> where Self: crate::Vector {
 
             /// Lanewise saturating add.
             ///
@@ -44,7 +44,7 @@ macro_rules! impl_uint_arith {
 
 macro_rules! impl_int_arith {
     ($(($name:ident, $n:ident)),+) => {
-        $( impl<const LANES: usize> $name<LANES> where Self: crate::LanesAtMost32 {
+        $( impl<const LANES: usize> $name<LANES> where Self: crate::Vector {
 
             /// Lanewise saturating add.
             ///
diff --git a/crates/core_simd/src/ops.rs b/crates/core_simd/src/ops.rs
index c7037d2acbc6..a41782a1464f 100644
--- a/crates/core_simd/src/ops.rs
+++ b/crates/core_simd/src/ops.rs
@@ -1,4 +1,4 @@
-use crate::LanesAtMost32;
+use crate::Vector;
 
 /// Checks if the right-hand side argument of a left- or right-shift would cause overflow.
 fn invalid_shift_rhs<T>(rhs: T) -> bool
@@ -16,7 +16,7 @@ macro_rules! impl_ref_ops {
     {
         impl<const $lanes:ident: usize> core::ops::$trait:ident<$rhs:ty> for $type:ty
         where
-            $($bound:path: LanesAtMost32,)*
+            $($bound:path: Vector,)*
         {
             type Output = $output:ty;
 
@@ -26,7 +26,7 @@ macro_rules! impl_ref_ops {
     } => {
         impl<const $lanes: usize> core::ops::$trait<$rhs> for $type
         where
-            $($bound: LanesAtMost32,)*
+            $($bound: Vector,)*
         {
             type Output = $output;
 
@@ -36,7 +36,7 @@ macro_rules! impl_ref_ops {
 
         impl<const $lanes: usize> core::ops::$trait<&'_ $rhs> for $type
         where
-            $($bound: LanesAtMost32,)*
+            $($bound: Vector,)*
         {
             type Output = <$type as core::ops::$trait<$rhs>>::Output;
 
@@ -48,7 +48,7 @@ macro_rules! impl_ref_ops {
 
         impl<const $lanes: usize> core::ops::$trait<$rhs> for &'_ $type
         where
-            $($bound: LanesAtMost32,)*
+            $($bound: Vector,)*
         {
             type Output = <$type as core::ops::$trait<$rhs>>::Output;
 
@@ -60,7 +60,7 @@ macro_rules! impl_ref_ops {
 
         impl<const $lanes: usize> core::ops::$trait<&'_ $rhs> for &'_ $type
         where
-            $($bound: LanesAtMost32,)*
+            $($bound: Vector,)*
         {
             type Output = <$type as core::ops::$trait<$rhs>>::Output;
 
@@ -75,7 +75,7 @@ macro_rules! impl_ref_ops {
     {
         impl<const $lanes:ident: usize> core::ops::$trait:ident<$rhs:ty> for $type:ty
         where
-            $($bound:path: LanesAtMost32,)*
+            $($bound:path: Vector,)*
         {
             $(#[$attrs:meta])*
             fn $fn:ident(&mut $self_tok:ident, $rhs_arg:ident: $rhs_arg_ty:ty) $body:tt
@@ -83,7 +83,7 @@ macro_rules! impl_ref_ops {
     } => {
         impl<const $lanes: usize> core::ops::$trait<$rhs> for $type
         where
-            $($bound: LanesAtMost32,)*
+            $($bound: Vector,)*
         {
             $(#[$attrs])*
             fn $fn(&mut $self_tok, $rhs_arg: $rhs_arg_ty) $body
@@ -91,7 +91,7 @@ macro_rules! impl_ref_ops {
 
         impl<const $lanes: usize> core::ops::$trait<&'_ $rhs> for $type
         where
-            $($bound: LanesAtMost32,)*
+            $($bound: Vector,)*
         {
             $(#[$attrs])*
             fn $fn(&mut $self_tok, $rhs_arg: &$rhs_arg_ty) {
@@ -104,7 +104,7 @@ macro_rules! impl_ref_ops {
     {
         impl<const $lanes:ident: usize> core::ops::$trait:ident for $type:ty
         where
-            $($bound:path: LanesAtMost32,)*
+            $($bound:path: Vector,)*
         {
             type Output = $output:ty;
             fn $fn:ident($self_tok:ident) -> Self::Output $body:tt
@@ -112,7 +112,7 @@ macro_rules! impl_ref_ops {
     } => {
         impl<const $lanes: usize> core::ops::$trait for $type
         where
-            $($bound: LanesAtMost32,)*
+            $($bound: Vector,)*
         {
             type Output = $output;
             fn $fn($self_tok) -> Self::Output $body
@@ -120,7 +120,7 @@ macro_rules! impl_ref_ops {
 
         impl<const $lanes: usize> core::ops::$trait for &'_ $type
         where
-            $($bound: LanesAtMost32,)*
+            $($bound: Vector,)*
         {
             type Output = <$type as core::ops::$trait>::Output;
             fn $fn($self_tok) -> Self::Output {
@@ -167,7 +167,7 @@ macro_rules! impl_op {
         impl_ref_ops! {
             impl<const LANES: usize> core::ops::Not for crate::$type<LANES>
             where
-                crate::$type<LANES>: LanesAtMost32,
+                crate::$type<LANES>: Vector,
             {
                 type Output = Self;
                 fn not(self) -> Self::Output {
@@ -181,7 +181,7 @@ macro_rules! impl_op {
         impl_ref_ops! {
             impl<const LANES: usize> core::ops::Neg for crate::$type<LANES>
             where
-                crate::$type<LANES>: LanesAtMost32,
+                crate::$type<LANES>: Vector,
             {
                 type Output = Self;
                 fn neg(self) -> Self::Output {
@@ -194,7 +194,7 @@ macro_rules! impl_op {
     { impl Index for $type:ident, $scalar:ty } => {
         impl<I, const LANES: usize> core::ops::Index<I> for crate::$type<LANES>
         where
-            Self: LanesAtMost32,
+            Self: Vector,
             I: core::slice::SliceIndex<[$scalar]>,
         {
             type Output = I::Output;
@@ -206,7 +206,7 @@ macro_rules! impl_op {
 
         impl<I, const LANES: usize> core::ops::IndexMut<I> for crate::$type<LANES>
         where
-            Self: LanesAtMost32,
+            Self: Vector,
             I: core::slice::SliceIndex<[$scalar]>,
         {
             fn index_mut(&mut self, index: I) -> &mut Self::Output {
@@ -221,7 +221,7 @@ macro_rules! impl_op {
         impl_ref_ops! {
             impl<const LANES: usize> core::ops::$trait<Self> for crate::$type<LANES>
             where
-                crate::$type<LANES>: LanesAtMost32,
+                crate::$type<LANES>: Vector,
             {
                 type Output = Self;
 
@@ -237,7 +237,7 @@ macro_rules! impl_op {
         impl_ref_ops! {
             impl<const LANES: usize> core::ops::$trait<$scalar> for crate::$type<LANES>
             where
-                crate::$type<LANES>: LanesAtMost32,
+                crate::$type<LANES>: Vector,
             {
                 type Output = Self;
 
@@ -251,7 +251,7 @@ macro_rules! impl_op {
         impl_ref_ops! {
             impl<const LANES: usize> core::ops::$trait<crate::$type<LANES>> for $scalar
             where
-                crate::$type<LANES>: LanesAtMost32,
+                crate::$type<LANES>: Vector,
             {
                 type Output = crate::$type<LANES>;
 
@@ -265,7 +265,7 @@ macro_rules! impl_op {
         impl_ref_ops! {
             impl<const LANES: usize> core::ops::$assign_trait<Self> for crate::$type<LANES>
             where
-                crate::$type<LANES>: LanesAtMost32,
+                crate::$type<LANES>: Vector,
             {
                 #[inline]
                 fn $assign_trait_fn(&mut self, rhs: Self) {
@@ -279,7 +279,7 @@ macro_rules! impl_op {
         impl_ref_ops! {
             impl<const LANES: usize> core::ops::$assign_trait<$scalar> for crate::$type<LANES>
             where
-                crate::$type<LANES>: LanesAtMost32,
+                crate::$type<LANES>: Vector,
             {
                 #[inline]
                 fn $assign_trait_fn(&mut self, rhs: $scalar) {
@@ -325,7 +325,7 @@ macro_rules! impl_unsigned_int_ops {
                 impl_ref_ops! {
                     impl<const LANES: usize> core::ops::Div<Self> for crate::$vector<LANES>
                     where
-                        crate::$vector<LANES>: LanesAtMost32,
+                        crate::$vector<LANES>: Vector,
                     {
                         type Output = Self;
 
@@ -353,7 +353,7 @@ macro_rules! impl_unsigned_int_ops {
                 impl_ref_ops! {
                     impl<const LANES: usize> core::ops::Div<$scalar> for crate::$vector<LANES>
                     where
-                        crate::$vector<LANES>: LanesAtMost32,
+                        crate::$vector<LANES>: Vector,
                     {
                         type Output = Self;
 
@@ -376,7 +376,7 @@ macro_rules! impl_unsigned_int_ops {
                 impl_ref_ops! {
                     impl<const LANES: usize> core::ops::Div<crate::$vector<LANES>> for $scalar
                     where
-                        crate::$vector<LANES>: LanesAtMost32,
+                        crate::$vector<LANES>: Vector,
                     {
                         type Output = crate::$vector<LANES>;
 
@@ -390,7 +390,7 @@ macro_rules! impl_unsigned_int_ops {
                 impl_ref_ops! {
                     impl<const LANES: usize> core::ops::DivAssign<Self> for crate::$vector<LANES>
                     where
-                        crate::$vector<LANES>: LanesAtMost32,
+                        crate::$vector<LANES>: Vector,
                     {
                         #[inline]
                         fn div_assign(&mut self, rhs: Self) {
@@ -402,7 +402,7 @@ macro_rules! impl_unsigned_int_ops {
                 impl_ref_ops! {
                     impl<const LANES: usize> core::ops::DivAssign<$scalar> for crate::$vector<LANES>
                     where
-                        crate::$vector<LANES>: LanesAtMost32,
+                        crate::$vector<LANES>: Vector,
                     {
                         #[inline]
                         fn div_assign(&mut self, rhs: $scalar) {
@@ -415,7 +415,7 @@ macro_rules! impl_unsigned_int_ops {
                 impl_ref_ops! {
                     impl<const LANES: usize> core::ops::Rem<Self> for crate::$vector<LANES>
                     where
-                        crate::$vector<LANES>: LanesAtMost32,
+                        crate::$vector<LANES>: Vector,
                     {
                         type Output = Self;
 
@@ -443,7 +443,7 @@ macro_rules! impl_unsigned_int_ops {
                 impl_ref_ops! {
                     impl<const LANES: usize> core::ops::Rem<$scalar> for crate::$vector<LANES>
                     where
-                        crate::$vector<LANES>: LanesAtMost32,
+                        crate::$vector<LANES>: Vector,
                     {
                         type Output = Self;
 
@@ -466,7 +466,7 @@ macro_rules! impl_unsigned_int_ops {
                 impl_ref_ops! {
                     impl<const LANES: usize> core::ops::Rem<crate::$vector<LANES>> for $scalar
                     where
-                        crate::$vector<LANES>: LanesAtMost32,
+                        crate::$vector<LANES>: Vector,
                     {
                         type Output = crate::$vector<LANES>;
 
@@ -480,7 +480,7 @@ macro_rules! impl_unsigned_int_ops {
                 impl_ref_ops! {
                     impl<const LANES: usize> core::ops::RemAssign<Self> for crate::$vector<LANES>
                     where
-                        crate::$vector<LANES>: LanesAtMost32,
+                        crate::$vector<LANES>: Vector,
                     {
                         #[inline]
                         fn rem_assign(&mut self, rhs: Self) {
@@ -492,7 +492,7 @@ macro_rules! impl_unsigned_int_ops {
                 impl_ref_ops! {
                     impl<const LANES: usize> core::ops::RemAssign<$scalar> for crate::$vector<LANES>
                     where
-                        crate::$vector<LANES>: LanesAtMost32,
+                        crate::$vector<LANES>: Vector,
                     {
                         #[inline]
                         fn rem_assign(&mut self, rhs: $scalar) {
@@ -505,7 +505,7 @@ macro_rules! impl_unsigned_int_ops {
                 impl_ref_ops! {
                     impl<const LANES: usize> core::ops::Shl<Self> for crate::$vector<LANES>
                     where
-                        crate::$vector<LANES>: LanesAtMost32,
+                        crate::$vector<LANES>: Vector,
                     {
                         type Output = Self;
 
@@ -527,7 +527,7 @@ macro_rules! impl_unsigned_int_ops {
                 impl_ref_ops! {
                     impl<const LANES: usize> core::ops::Shl<$scalar> for crate::$vector<LANES>
                     where
-                        crate::$vector<LANES>: LanesAtMost32,
+                        crate::$vector<LANES>: Vector,
                     {
                         type Output = Self;
 
@@ -546,7 +546,7 @@ macro_rules! impl_unsigned_int_ops {
                 impl_ref_ops! {
                     impl<const LANES: usize> core::ops::ShlAssign<Self> for crate::$vector<LANES>
                     where
-                        crate::$vector<LANES>: LanesAtMost32,
+                        crate::$vector<LANES>: Vector,
                     {
                         #[inline]
                         fn shl_assign(&mut self, rhs: Self) {
@@ -558,7 +558,7 @@ macro_rules! impl_unsigned_int_ops {
                 impl_ref_ops! {
                     impl<const LANES: usize> core::ops::ShlAssign<$scalar> for crate::$vector<LANES>
                     where
-                        crate::$vector<LANES>: LanesAtMost32,
+                        crate::$vector<LANES>: Vector,
                     {
                         #[inline]
                         fn shl_assign(&mut self, rhs: $scalar) {
@@ -570,7 +570,7 @@ macro_rules! impl_unsigned_int_ops {
                 impl_ref_ops! {
                     impl<const LANES: usize> core::ops::Shr<Self> for crate::$vector<LANES>
                     where
-                        crate::$vector<LANES>: LanesAtMost32,
+                        crate::$vector<LANES>: Vector,
                     {
                         type Output = Self;
 
@@ -592,7 +592,7 @@ macro_rules! impl_unsigned_int_ops {
                 impl_ref_ops! {
                     impl<const LANES: usize> core::ops::Shr<$scalar> for crate::$vector<LANES>
                     where
-                        crate::$vector<LANES>: LanesAtMost32,
+                        crate::$vector<LANES>: Vector,
                     {
                         type Output = Self;
 
@@ -611,7 +611,7 @@ macro_rules! impl_unsigned_int_ops {
                 impl_ref_ops! {
                     impl<const LANES: usize> core::ops::ShrAssign<Self> for crate::$vector<LANES>
                     where
-                        crate::$vector<LANES>: LanesAtMost32,
+                        crate::$vector<LANES>: Vector,
                     {
                         #[inline]
                         fn shr_assign(&mut self, rhs: Self) {
@@ -623,7 +623,7 @@ macro_rules! impl_unsigned_int_ops {
                 impl_ref_ops! {
                     impl<const LANES: usize> core::ops::ShrAssign<$scalar> for crate::$vector<LANES>
                     where
-                        crate::$vector<LANES>: LanesAtMost32,
+                        crate::$vector<LANES>: Vector,
                     {
                         #[inline]
                         fn shr_assign(&mut self, rhs: $scalar) {
diff --git a/crates/core_simd/src/reduction.rs b/crates/core_simd/src/reduction.rs
index 8687d1af5167..548d280b22b5 100644
--- a/crates/core_simd/src/reduction.rs
+++ b/crates/core_simd/src/reduction.rs
@@ -2,7 +2,7 @@ macro_rules! impl_integer_reductions {
     { $name:ident, $scalar:ty } => {
         impl<const LANES: usize> crate::$name<LANES>
         where
-            Self: crate::LanesAtMost32
+            Self: crate::Vector
         {
             /// Horizontal wrapping add.  Returns the sum of the lanes of the vector, with wrapping addition.
             #[inline]
@@ -56,7 +56,7 @@ macro_rules! impl_float_reductions {
     { $name:ident, $scalar:ty } => {
         impl<const LANES: usize> crate::$name<LANES>
         where
-            Self: crate::LanesAtMost32
+            Self: crate::Vector
         {
 
             /// Horizontal add.  Returns the sum of the lanes of the vector.
@@ -106,7 +106,7 @@ macro_rules! impl_full_mask_reductions {
     { $name:ident, $bits_ty:ident } => {
         impl<T: crate::Mask, const LANES: usize> $name<T, LANES>
         where
-            crate::$bits_ty<LANES>: crate::LanesAtMost32
+            crate::$bits_ty<LANES>: crate::Vector
         {
             #[inline]
             pub fn any(self) -> bool {
@@ -125,7 +125,7 @@ macro_rules! impl_opaque_mask_reductions {
     { $name:ident, $bits_ty:ident } => {
         impl<const LANES: usize> $name<LANES>
         where
-            crate::$bits_ty<LANES>: crate::LanesAtMost32,
+            crate::$bits_ty<LANES>: crate::Vector,
             $name<LANES>: crate::Mask,
         {
             /// Returns true if any lane is set, or false otherwise.
diff --git a/crates/core_simd/src/round.rs b/crates/core_simd/src/round.rs
index 281851c68ace..5cd7a898eaf4 100644
--- a/crates/core_simd/src/round.rs
+++ b/crates/core_simd/src/round.rs
@@ -5,7 +5,7 @@ macro_rules! implement {
         #[cfg(feature = "std")]
         impl<const LANES: usize> crate::$type<LANES>
         where
-            Self: crate::LanesAtMost32,
+            Self: crate::Vector,
         {
             /// Returns the smallest integer greater than or equal to each lane.
             #[must_use = "method returns a new vector and does not mutate the original value"]
@@ -45,8 +45,8 @@ macro_rules! implement {
 
         impl<const LANES: usize> crate::$type<LANES>
         where
-            Self: crate::LanesAtMost32,
-            crate::$int_type<LANES>: crate::LanesAtMost32,
+            Self: crate::Vector,
+            crate::$int_type<LANES>: crate::Vector,
         {
             /// Rounds toward zero and converts to the same-width integer type, assuming that
             /// the value is finite and fits in that type.
diff --git a/crates/core_simd/src/select.rs b/crates/core_simd/src/select.rs
index dee1d775eb82..1558eb169351 100644
--- a/crates/core_simd/src/select.rs
+++ b/crates/core_simd/src/select.rs
@@ -14,12 +14,12 @@ macro_rules! impl_select {
         $mask:ident ($bits_ty:ident): $($type:ident),*
     } => {
         $(
-        impl<const LANES: usize> Sealed for crate::$type<LANES> where Self: crate::LanesAtMost32 {}
+        impl<const LANES: usize> Sealed for crate::$type<LANES> where Self: crate::Vector {}
         impl<const LANES: usize> Select<crate::$mask<LANES>> for crate::$type<LANES>
         where
             crate::$mask<LANES>: crate::Mask,
-            crate::$bits_ty<LANES>: crate::LanesAtMost32,
-            Self: crate::LanesAtMost32,
+            crate::$bits_ty<LANES>: crate::Vector,
+            Self: crate::Vector,
         {
             #[doc(hidden)]
             #[inline]
@@ -32,12 +32,12 @@ macro_rules! impl_select {
         impl<const LANES: usize> Sealed for crate::$mask<LANES>
         where
             Self: crate::Mask,
-            crate::$bits_ty<LANES>: crate::LanesAtMost32,
+            crate::$bits_ty<LANES>: crate::Vector,
         {}
         impl<const LANES: usize> Select<Self> for crate::$mask<LANES>
         where
             Self: crate::Mask,
-            crate::$bits_ty<LANES>: crate::LanesAtMost32,
+            crate::$bits_ty<LANES>: crate::Vector,
         {
             #[doc(hidden)]
             #[inline]
@@ -49,7 +49,7 @@ macro_rules! impl_select {
         impl<const LANES: usize> crate::$mask<LANES>
         where
             Self: crate::Mask,
-            crate::$bits_ty<LANES>: crate::LanesAtMost32,
+            crate::$bits_ty<LANES>: crate::Vector,
         {
             /// Choose lanes from two vectors.
             ///
diff --git a/crates/core_simd/src/to_bytes.rs b/crates/core_simd/src/to_bytes.rs
index a2d9cc4ef56c..c4f112c9ee7e 100644
--- a/crates/core_simd/src/to_bytes.rs
+++ b/crates/core_simd/src/to_bytes.rs
@@ -18,11 +18,11 @@ pub trait ToBytes: Sealed {
 macro_rules! impl_to_bytes {
     { $name:ident, $($int_width:literal -> $byte_width:literal),* } => {
         $(
-        impl Sealed for crate::$name<$int_width> where Self: crate::LanesAtMost32 {}
+        impl Sealed for crate::$name<$int_width> where Self: crate::Vector {}
         impl ToBytes for crate::$name<$int_width>
         where
-            Self: crate::LanesAtMost32,
-            crate::SimdU8<$byte_width>: crate::LanesAtMost32,
+            Self: crate::Vector,
+            crate::SimdU8<$byte_width>: crate::Vector,
         {
             type Bytes = crate::SimdU8<$byte_width>;
             fn to_bytes_impl(self) -> Self::Bytes {
@@ -36,7 +36,7 @@ macro_rules! impl_to_bytes {
 
         impl<const LANES: usize> crate::$name<LANES>
         where
-            Self: ToBytes + crate::LanesAtMost32,
+            Self: ToBytes + crate::Vector,
         {
             /// Return the memory representation of this integer as a byte array in native byte
             /// order.
diff --git a/crates/core_simd/src/vector.rs b/crates/core_simd/src/vector.rs
deleted file mode 100644
index 95214ea88642..000000000000
--- a/crates/core_simd/src/vector.rs
+++ /dev/null
@@ -1,10 +0,0 @@
-mod float;
-mod int;
-mod uint;
-
-pub use float::*;
-pub use int::*;
-pub use uint::*;
-
-// Vectors of pointers are not for public use at the current time.
-pub(crate) mod ptr;
diff --git a/crates/core_simd/src/vector/float.rs b/crates/core_simd/src/vector/float.rs
index b6e9b61f71f6..4b069a4fab83 100644
--- a/crates/core_simd/src/vector/float.rs
+++ b/crates/core_simd/src/vector/float.rs
@@ -10,8 +10,8 @@ macro_rules! impl_float_vector {
 
         impl<const LANES: usize> $name<LANES>
         where
-            Self: crate::LanesAtMost32,
-            crate::$bits_ty<LANES>: crate::LanesAtMost32,
+            Self: crate::Vector,
+            crate::$bits_ty<LANES>: crate::Vector,
         {
             /// Raw transmutation to an unsigned integer vector type with the
             /// same size and number of lanes.
@@ -78,9 +78,9 @@ macro_rules! impl_float_vector {
 
         impl<const LANES: usize> $name<LANES>
         where
-            Self: crate::LanesAtMost32,
-            crate::$bits_ty<LANES>: crate::LanesAtMost32,
-            crate::$mask_impl_ty<LANES>: crate::LanesAtMost32,
+            Self: crate::Vector,
+            crate::$bits_ty<LANES>: crate::Vector,
+            crate::$mask_impl_ty<LANES>: crate::Vector,
             crate::$mask_ty<LANES>: crate::Mask,
         {
             /// Returns true for each lane if it has a positive sign, including
@@ -197,7 +197,7 @@ macro_rules! impl_float_vector {
 #[repr(simd)]
 pub struct SimdF32<const LANES: usize>([f32; LANES])
 where
-    Self: crate::LanesAtMost32;
+    Self: crate::Vector;
 
 impl_float_vector! { SimdF32, f32, SimdU32, Mask32, SimdI32 }
 
@@ -205,7 +205,7 @@ impl_float_vector! { SimdF32, f32, SimdU32, Mask32, SimdI32 }
 #[repr(simd)]
 pub struct SimdF64<const LANES: usize>([f64; LANES])
 where
-    Self: crate::LanesAtMost32;
+    Self: crate::Vector;
 
 impl_float_vector! { SimdF64, f64, SimdU64, Mask64, SimdI64 }
 
diff --git a/crates/core_simd/src/vector/int.rs b/crates/core_simd/src/vector/int.rs
index 15ad1a7193af..60b3f56d4c4a 100644
--- a/crates/core_simd/src/vector/int.rs
+++ b/crates/core_simd/src/vector/int.rs
@@ -6,9 +6,9 @@ macro_rules! impl_integer_vector {
         impl_vector! { $name, $type }
         impl_integer_reductions! { $name, $type }
 
-        impl<const LANES: usize> Eq for $name<LANES> where Self: crate::LanesAtMost32 {}
+        impl<const LANES: usize> Eq for $name<LANES> where Self: crate::Vector {}
 
-        impl<const LANES: usize> Ord for $name<LANES> where Self: crate::LanesAtMost32 {
+        impl<const LANES: usize> Ord for $name<LANES> where Self: crate::Vector {
             #[inline]
             fn cmp(&self, other: &Self) -> core::cmp::Ordering {
                 // TODO use SIMD cmp
@@ -16,7 +16,7 @@ macro_rules! impl_integer_vector {
             }
         }
 
-        impl<const LANES: usize> core::hash::Hash for $name<LANES> where Self: crate::LanesAtMost32 {
+        impl<const LANES: usize> core::hash::Hash for $name<LANES> where Self: crate::Vector {
             #[inline]
             fn hash<H>(&self, state: &mut H)
             where
@@ -28,8 +28,8 @@ macro_rules! impl_integer_vector {
 
         impl<const LANES: usize> $name<LANES>
         where
-            Self: crate::LanesAtMost32,
-            crate::$mask_impl_ty<LANES>: crate::LanesAtMost32,
+            Self: crate::Vector,
+            crate::$mask_impl_ty<LANES>: crate::Vector,
             crate::$mask_ty<LANES>: crate::Mask,
         {
             /// Returns true for each positive lane and false if it is zero or negative.
@@ -63,7 +63,7 @@ macro_rules! impl_integer_vector {
 #[repr(simd)]
 pub struct SimdIsize<const LANES: usize>([isize; LANES])
 where
-    Self: crate::LanesAtMost32;
+    Self: crate::Vector;
 
 impl_integer_vector! { SimdIsize, isize, MaskSize, SimdIsize }
 
@@ -71,7 +71,7 @@ impl_integer_vector! { SimdIsize, isize, MaskSize, SimdIsize }
 #[repr(simd)]
 pub struct SimdI16<const LANES: usize>([i16; LANES])
 where
-    Self: crate::LanesAtMost32;
+    Self: crate::Vector;
 
 impl_integer_vector! { SimdI16, i16, Mask16, SimdI16 }
 
@@ -79,7 +79,7 @@ impl_integer_vector! { SimdI16, i16, Mask16, SimdI16 }
 #[repr(simd)]
 pub struct SimdI32<const LANES: usize>([i32; LANES])
 where
-    Self: crate::LanesAtMost32;
+    Self: crate::Vector;
 
 impl_integer_vector! { SimdI32, i32, Mask32, SimdI32 }
 
@@ -87,7 +87,7 @@ impl_integer_vector! { SimdI32, i32, Mask32, SimdI32 }
 #[repr(simd)]
 pub struct SimdI64<const LANES: usize>([i64; LANES])
 where
-    Self: crate::LanesAtMost32;
+    Self: crate::Vector;
 
 impl_integer_vector! { SimdI64, i64, Mask64, SimdI64 }
 
@@ -95,7 +95,7 @@ impl_integer_vector! { SimdI64, i64, Mask64, SimdI64 }
 #[repr(simd)]
 pub struct SimdI8<const LANES: usize>([i8; LANES])
 where
-    Self: crate::LanesAtMost32;
+    Self: crate::Vector;
 
 impl_integer_vector! { SimdI8, i8, Mask8, SimdI8 }
 
diff --git a/crates/core_simd/src/vector/mod.rs b/crates/core_simd/src/vector/mod.rs
new file mode 100644
index 000000000000..1af82d9c76ff
--- /dev/null
+++ b/crates/core_simd/src/vector/mod.rs
@@ -0,0 +1,132 @@
+mod float;
+mod int;
+mod uint;
+
+pub use float::*;
+pub use int::*;
+pub use uint::*;
+
+// Vectors of pointers are not for public use at the current time.
+pub(crate) mod ptr;
+
+mod sealed {
+    pub trait Sealed {}
+}
+
+/// A representation of a vector as an "array" with indices, implementing
+/// operations applicable to any vector type based solely on "having lanes",
+/// and describing relationships between vector and scalar types.
+pub trait Vector: sealed::Sealed {
+    /// The scalar type in every lane of this vector type.
+    type Scalar: Copy + Sized;
+
+    /// The number of lanes for this vector.
+    const LANES: usize;
+
+    // Implementation detail until the compiler can support bitmasks of any integer width
+    #[doc(hidden)]
+    type BitMask: Into<u64>;
+
+    /// Generates a SIMD vector with the same value in every lane.
+    #[must_use]
+    fn splat(val: Self::Scalar) -> Self;
+
+}
+
+macro_rules! impl_vector_for {
+    ($simd:ident {type Scalar = $scalar:ident;}) => {
+        impl_vector_for! { $simd<1> { type Scalar = $scalar; type BitMask = u8; } }
+        impl_vector_for! { $simd<2> { type Scalar = $scalar; type BitMask = u8; } }
+        impl_vector_for! { $simd<4> { type Scalar = $scalar; type BitMask = u8; } }
+        impl_vector_for! { $simd<8> { type Scalar = $scalar; type BitMask = u8; } }
+        impl_vector_for! { $simd<16> { type Scalar = $scalar; type BitMask = u16; } }
+        impl_vector_for! { $simd<32> { type Scalar = $scalar; type BitMask = u32; } }
+    };
+    ($simd:ident<$lanes:literal> {type Scalar = $scalar:ident; type BitMask = $bitmask:ident; }) => {
+        impl sealed::Sealed for $simd<$lanes> {}
+
+        impl Vector for $simd<$lanes> {
+            type Scalar = $scalar;
+            const LANES: usize = $lanes;
+
+            type BitMask = $bitmask;
+
+            #[inline]
+            fn splat(val: Self::Scalar) -> Self {
+                [val; $lanes].into()
+            }
+        }
+    };
+}
+
+impl_vector_for! {
+    SimdUsize {
+        type Scalar = usize;
+    }
+}
+
+impl_vector_for! {
+    SimdIsize {
+        type Scalar = isize;
+    }
+}
+
+impl_vector_for! {
+    SimdI8 {
+        type Scalar = i8;
+    }
+}
+
+impl_vector_for! {
+    SimdI16 {
+        type Scalar = i16;
+    }
+}
+
+impl_vector_for! {
+    SimdI32 {
+        type Scalar = i32;
+    }
+}
+
+impl_vector_for! {
+    SimdI64 {
+        type Scalar = i64;
+    }
+}
+
+impl_vector_for! {
+    SimdU8 {
+        type Scalar = u8;
+    }
+}
+
+impl_vector_for! {
+    SimdU16 {
+        type Scalar = u16;
+    }
+}
+
+impl_vector_for! {
+    SimdU32 {
+        type Scalar = u32;
+    }
+}
+
+impl_vector_for! {
+    SimdU64 {
+        type Scalar = u64;
+    }
+}
+
+impl_vector_for! {
+    SimdF32 {
+        type Scalar = f32;
+    }
+}
+
+impl_vector_for! {
+    SimdF64 {
+        type Scalar = f64;
+    }
+}
diff --git a/crates/core_simd/src/vector/ptr.rs b/crates/core_simd/src/vector/ptr.rs
index 30bef038b333..bdd5c24eac85 100644
--- a/crates/core_simd/src/vector/ptr.rs
+++ b/crates/core_simd/src/vector/ptr.rs
@@ -9,7 +9,7 @@ pub(crate) struct SimdConstPtr<T, const LANES: usize>([*const T; LANES]);
 
 impl<T, const LANES: usize> SimdConstPtr<T, LANES>
 where
-    SimdUsize<LANES>: crate::LanesAtMost32,
+    SimdUsize<LANES>: crate::Vector,
     T: Sized,
 {
     #[inline]
@@ -35,7 +35,7 @@ pub(crate) struct SimdMutPtr<T, const LANES: usize>([*mut T; LANES]);
 
 impl<T, const LANES: usize> SimdMutPtr<T, LANES>
 where
-    SimdUsize<LANES>: crate::LanesAtMost32,
+    SimdUsize<LANES>: crate::Vector,
     T: Sized,
 {
     #[inline]
diff --git a/crates/core_simd/src/vector/uint.rs b/crates/core_simd/src/vector/uint.rs
index 0429410ed6d6..1340bec06f36 100644
--- a/crates/core_simd/src/vector/uint.rs
+++ b/crates/core_simd/src/vector/uint.rs
@@ -6,9 +6,9 @@ macro_rules! impl_unsigned_vector {
         impl_vector! { $name, $type }
         impl_integer_reductions! { $name, $type }
 
-        impl<const LANES: usize> Eq for $name<LANES> where Self: crate::LanesAtMost32 {}
+        impl<const LANES: usize> Eq for $name<LANES> where Self: crate::Vector {}
 
-        impl<const LANES: usize> Ord for $name<LANES> where Self: crate::LanesAtMost32 {
+        impl<const LANES: usize> Ord for $name<LANES> where Self: crate::Vector {
             #[inline]
             fn cmp(&self, other: &Self) -> core::cmp::Ordering {
                 // TODO use SIMD cmp
@@ -16,7 +16,7 @@ macro_rules! impl_unsigned_vector {
             }
         }
 
-        impl<const LANES: usize> core::hash::Hash for $name<LANES> where Self: crate::LanesAtMost32 {
+        impl<const LANES: usize> core::hash::Hash for $name<LANES> where Self: crate::Vector {
             #[inline]
             fn hash<H>(&self, state: &mut H)
             where
@@ -32,7 +32,7 @@ macro_rules! impl_unsigned_vector {
 #[repr(simd)]
 pub struct SimdUsize<const LANES: usize>([usize; LANES])
 where
-    Self: crate::LanesAtMost32;
+    Self: crate::Vector;
 
 impl_unsigned_vector! { SimdUsize, usize }
 
@@ -40,7 +40,7 @@ impl_unsigned_vector! { SimdUsize, usize }
 #[repr(simd)]
 pub struct SimdU16<const LANES: usize>([u16; LANES])
 where
-    Self: crate::LanesAtMost32;
+    Self: crate::Vector;
 
 impl_unsigned_vector! { SimdU16, u16 }
 
@@ -48,7 +48,7 @@ impl_unsigned_vector! { SimdU16, u16 }
 #[repr(simd)]
 pub struct SimdU32<const LANES: usize>([u32; LANES])
 where
-    Self: crate::LanesAtMost32;
+    Self: crate::Vector;
 
 impl_unsigned_vector! { SimdU32, u32 }
 
@@ -56,7 +56,7 @@ impl_unsigned_vector! { SimdU32, u32 }
 #[repr(simd)]
 pub struct SimdU64<const LANES: usize>([u64; LANES])
 where
-    Self: crate::LanesAtMost32;
+    Self: crate::Vector;
 
 impl_unsigned_vector! { SimdU64, u64 }
 
@@ -64,7 +64,7 @@ impl_unsigned_vector! { SimdU64, u64 }
 #[repr(simd)]
 pub struct SimdU8<const LANES: usize>([u8; LANES])
 where
-    Self: crate::LanesAtMost32;
+    Self: crate::Vector;
 
 impl_unsigned_vector! { SimdU8, u8 }
 
diff --git a/crates/test_helpers/src/lib.rs b/crates/test_helpers/src/lib.rs
index 318a7b3005e3..5691bf4538ce 100644
--- a/crates/test_helpers/src/lib.rs
+++ b/crates/test_helpers/src/lib.rs
@@ -335,18 +335,18 @@ macro_rules! test_lanes {
 
                 fn implementation<const $lanes: usize>()
                 where
-                    core_simd::SimdU8<$lanes>: core_simd::LanesAtMost32,
-                    core_simd::SimdU16<$lanes>: core_simd::LanesAtMost32,
-                    core_simd::SimdU32<$lanes>: core_simd::LanesAtMost32,
-                    core_simd::SimdU64<$lanes>: core_simd::LanesAtMost32,
-                    core_simd::SimdUsize<$lanes>: core_simd::LanesAtMost32,
-                    core_simd::SimdI8<$lanes>: core_simd::LanesAtMost32,
-                    core_simd::SimdI16<$lanes>: core_simd::LanesAtMost32,
-                    core_simd::SimdI32<$lanes>: core_simd::LanesAtMost32,
-                    core_simd::SimdI64<$lanes>: core_simd::LanesAtMost32,
-                    core_simd::SimdIsize<$lanes>: core_simd::LanesAtMost32,
-                    core_simd::SimdF32<$lanes>: core_simd::LanesAtMost32,
-                    core_simd::SimdF64<$lanes>: core_simd::LanesAtMost32,
+                    core_simd::SimdU8<$lanes>: core_simd::Vector,
+                    core_simd::SimdU16<$lanes>: core_simd::Vector,
+                    core_simd::SimdU32<$lanes>: core_simd::Vector,
+                    core_simd::SimdU64<$lanes>: core_simd::Vector,
+                    core_simd::SimdUsize<$lanes>: core_simd::Vector,
+                    core_simd::SimdI8<$lanes>: core_simd::Vector,
+                    core_simd::SimdI16<$lanes>: core_simd::Vector,
+                    core_simd::SimdI32<$lanes>: core_simd::Vector,
+                    core_simd::SimdI64<$lanes>: core_simd::Vector,
+                    core_simd::SimdIsize<$lanes>: core_simd::Vector,
+                    core_simd::SimdF32<$lanes>: core_simd::Vector,
+                    core_simd::SimdF64<$lanes>: core_simd::Vector,
                     core_simd::Mask8<$lanes>: core_simd::Mask,
                     core_simd::Mask16<$lanes>: core_simd::Mask,
                     core_simd::Mask32<$lanes>: core_simd::Mask,
@@ -409,18 +409,18 @@ macro_rules! test_lanes_panic {
 
                 fn implementation<const $lanes: usize>()
                 where
-                    core_simd::SimdU8<$lanes>: core_simd::LanesAtMost32,
-                    core_simd::SimdU16<$lanes>: core_simd::LanesAtMost32,
-                    core_simd::SimdU32<$lanes>: core_simd::LanesAtMost32,
-                    core_simd::SimdU64<$lanes>: core_simd::LanesAtMost32,
-                    core_simd::SimdUsize<$lanes>: core_simd::LanesAtMost32,
-                    core_simd::SimdI8<$lanes>: core_simd::LanesAtMost32,
-                    core_simd::SimdI16<$lanes>: core_simd::LanesAtMost32,
-                    core_simd::SimdI32<$lanes>: core_simd::LanesAtMost32,
-                    core_simd::SimdI64<$lanes>: core_simd::LanesAtMost32,
-                    core_simd::SimdIsize<$lanes>: core_simd::LanesAtMost32,
-                    core_simd::SimdF32<$lanes>: core_simd::LanesAtMost32,
-                    core_simd::SimdF64<$lanes>: core_simd::LanesAtMost32,
+                    core_simd::SimdU8<$lanes>: core_simd::Vector,
+                    core_simd::SimdU16<$lanes>: core_simd::Vector,
+                    core_simd::SimdU32<$lanes>: core_simd::Vector,
+                    core_simd::SimdU64<$lanes>: core_simd::Vector,
+                    core_simd::SimdUsize<$lanes>: core_simd::Vector,
+                    core_simd::SimdI8<$lanes>: core_simd::Vector,
+                    core_simd::SimdI16<$lanes>: core_simd::Vector,
+                    core_simd::SimdI32<$lanes>: core_simd::Vector,
+                    core_simd::SimdI64<$lanes>: core_simd::Vector,
+                    core_simd::SimdIsize<$lanes>: core_simd::Vector,
+                    core_simd::SimdF32<$lanes>: core_simd::Vector,
+                    core_simd::SimdF64<$lanes>: core_simd::Vector,
                     core_simd::Mask8<$lanes>: core_simd::Mask,
                     core_simd::Mask16<$lanes>: core_simd::Mask,
                     core_simd::Mask32<$lanes>: core_simd::Mask,

From f178dda187c479f463e74f92e3bdd8be13bad9e7 Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Sun, 27 Jun 2021 19:38:30 +0000
Subject: [PATCH 194/249] Add as_slice/as_mut_slice to Vector

---
 crates/core_simd/src/vector/mod.rs | 17 ++++++++++++++++-
 1 file changed, 16 insertions(+), 1 deletion(-)

diff --git a/crates/core_simd/src/vector/mod.rs b/crates/core_simd/src/vector/mod.rs
index 1af82d9c76ff..d1940ab9833d 100644
--- a/crates/core_simd/src/vector/mod.rs
+++ b/crates/core_simd/src/vector/mod.rs
@@ -31,6 +31,11 @@ pub trait Vector: sealed::Sealed {
     #[must_use]
     fn splat(val: Self::Scalar) -> Self;
 
+    /// Returns a slice containing the entire SIMD vector.
+    fn as_slice(&self) -> &[Self::Scalar];
+
+    /// Returns a mutable slice containing the entire SIMD vector.
+    fn as_mut_slice(&mut self) -> &mut [Self::Scalar];
 }
 
 macro_rules! impl_vector_for {
@@ -53,7 +58,17 @@ macro_rules! impl_vector_for {
 
             #[inline]
             fn splat(val: Self::Scalar) -> Self {
-                [val; $lanes].into()
+                Self::splat(val)
+            }
+
+            #[inline]
+            fn as_slice(&self) -> &[Self::Scalar] {
+                self.as_slice()
+            }
+
+            #[inline]
+            fn as_mut_slice(&mut self) -> &mut [Self::Scalar] {
+                self.as_mut_slice()
             }
         }
     };

From fdd7d6e2529fe5a4921f6ec03e42bacf924246aa Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Mon, 28 Jun 2021 00:41:21 +0000
Subject: [PATCH 195/249] Change as_slice to as_array

---
 crates/core_simd/src/first.rs       | 26 +++++---------------------
 crates/core_simd/src/ops.rs         | 20 ++++++++++----------
 crates/core_simd/src/reduction.rs   |  4 ++--
 crates/core_simd/src/vector/int.rs  |  4 ++--
 crates/core_simd/src/vector/mod.rs  | 16 ----------------
 crates/core_simd/src/vector/uint.rs |  4 ++--
 6 files changed, 21 insertions(+), 53 deletions(-)

diff --git a/crates/core_simd/src/first.rs b/crates/core_simd/src/first.rs
index 7721b87ecc0b..4560a0a1467c 100644
--- a/crates/core_simd/src/first.rs
+++ b/crates/core_simd/src/first.rs
@@ -7,13 +7,13 @@ macro_rules! impl_vector {
                 Self([value; LANES])
             }
 
-            /// Returns a slice containing the entire SIMD vector.
-            pub const fn as_slice(&self) -> &[$type] {
+            /// Returns an array reference containing the entire SIMD vector.
+            pub const fn as_array(&self) -> &[$type; LANES] {
                 &self.0
             }
 
-            /// Returns a mutable slice containing the entire SIMD vector.
-            pub fn as_mut_slice(&mut self) -> &mut [$type] {
+            /// Returns a mutable array reference containing the entire SIMD vector.
+            pub fn as_mut_array(&mut self) -> &mut [$type; LANES] {
                 &mut self.0
             }
 
@@ -24,23 +24,7 @@ macro_rules! impl_vector {
 
             /// Converts a SIMD vector to an array.
             pub const fn to_array(self) -> [$type; LANES] {
-                // workaround for rust-lang/rust#80108
-                // TODO fix this
-                #[cfg(target_arch = "wasm32")]
-                {
-                    let mut arr = [self.0[0]; LANES];
-                    let mut i = 0;
-                    while i < LANES {
-                        arr[i] = self.0[i];
-                        i += 1;
-                    }
-                    arr
-                }
-
-                #[cfg(not(target_arch = "wasm32"))]
-                {
-                    self.0
-                }
+                self.0
             }
         }
 
diff --git a/crates/core_simd/src/ops.rs b/crates/core_simd/src/ops.rs
index a41782a1464f..9491bdd1d5a6 100644
--- a/crates/core_simd/src/ops.rs
+++ b/crates/core_simd/src/ops.rs
@@ -331,7 +331,7 @@ macro_rules! impl_unsigned_int_ops {
 
                         #[inline]
                         fn div(self, rhs: Self) -> Self::Output {
-                            if rhs.as_slice()
+                            if rhs.as_array()
                                 .iter()
                                 .any(|x| *x == 0)
                             {
@@ -340,8 +340,8 @@ macro_rules! impl_unsigned_int_ops {
 
                             // Guards for div(MIN, -1),
                             // this check only applies to signed ints
-                            if <$scalar>::MIN != 0 && self.as_slice().iter()
-                                    .zip(rhs.as_slice().iter())
+                            if <$scalar>::MIN != 0 && self.as_array().iter()
+                                    .zip(rhs.as_array().iter())
                                     .any(|(x,y)| *x == <$scalar>::MIN && *y == -1 as _) {
                                 panic!("attempt to divide with overflow");
                             }
@@ -363,7 +363,7 @@ macro_rules! impl_unsigned_int_ops {
                                 panic!("attempt to divide by zero");
                             }
                             if <$scalar>::MIN != 0 &&
-                                self.as_slice().iter().any(|x| *x == <$scalar>::MIN) &&
+                                self.as_array().iter().any(|x| *x == <$scalar>::MIN) &&
                                 rhs == -1 as _ {
                                     panic!("attempt to divide with overflow");
                             }
@@ -421,7 +421,7 @@ macro_rules! impl_unsigned_int_ops {
 
                         #[inline]
                         fn rem(self, rhs: Self) -> Self::Output {
-                            if rhs.as_slice()
+                            if rhs.as_array()
                                 .iter()
                                 .any(|x| *x == 0)
                             {
@@ -430,8 +430,8 @@ macro_rules! impl_unsigned_int_ops {
 
                             // Guards for rem(MIN, -1)
                             // this branch applies the check only to signed ints
-                            if <$scalar>::MIN != 0 && self.as_slice().iter()
-                                    .zip(rhs.as_slice().iter())
+                            if <$scalar>::MIN != 0 && self.as_array().iter()
+                                    .zip(rhs.as_array().iter())
                                     .any(|(x,y)| *x == <$scalar>::MIN && *y == -1 as _) {
                                 panic!("attempt to calculate the remainder with overflow");
                             }
@@ -453,7 +453,7 @@ macro_rules! impl_unsigned_int_ops {
                                 panic!("attempt to calculate the remainder with a divisor of zero");
                             }
                             if <$scalar>::MIN != 0 &&
-                                self.as_slice().iter().any(|x| *x == <$scalar>::MIN) &&
+                                self.as_array().iter().any(|x| *x == <$scalar>::MIN) &&
                                 rhs == -1 as _ {
                                     panic!("attempt to calculate the remainder with overflow");
                             }
@@ -512,7 +512,7 @@ macro_rules! impl_unsigned_int_ops {
                         #[inline]
                         fn shl(self, rhs: Self) -> Self::Output {
                             // TODO there is probably a better way of doing this
-                            if rhs.as_slice()
+                            if rhs.as_array()
                                 .iter()
                                 .copied()
                                 .any(invalid_shift_rhs)
@@ -577,7 +577,7 @@ macro_rules! impl_unsigned_int_ops {
                         #[inline]
                         fn shr(self, rhs: Self) -> Self::Output {
                             // TODO there is probably a better way of doing this
-                            if rhs.as_slice()
+                            if rhs.as_array()
                                 .iter()
                                 .copied()
                                 .any(invalid_shift_rhs)
diff --git a/crates/core_simd/src/reduction.rs b/crates/core_simd/src/reduction.rs
index 548d280b22b5..41cf6fab612a 100644
--- a/crates/core_simd/src/reduction.rs
+++ b/crates/core_simd/src/reduction.rs
@@ -64,7 +64,7 @@ macro_rules! impl_float_reductions {
             pub fn horizontal_sum(self) -> $scalar {
                 // LLVM sum is inaccurate on i586
                 if cfg!(all(target_arch = "x86", not(target_feature = "sse2"))) {
-                    self.as_slice().iter().sum()
+                    self.as_array().iter().sum()
                 } else {
                     unsafe { crate::intrinsics::simd_reduce_add_ordered(self, 0.) }
                 }
@@ -75,7 +75,7 @@ macro_rules! impl_float_reductions {
             pub fn horizontal_product(self) -> $scalar {
                 // LLVM product is inaccurate on i586
                 if cfg!(all(target_arch = "x86", not(target_feature = "sse2"))) {
-                    self.as_slice().iter().product()
+                    self.as_array().iter().product()
                 } else {
                     unsafe { crate::intrinsics::simd_reduce_mul_ordered(self, 1.) }
                 }
diff --git a/crates/core_simd/src/vector/int.rs b/crates/core_simd/src/vector/int.rs
index 60b3f56d4c4a..0ca1ea14e2d5 100644
--- a/crates/core_simd/src/vector/int.rs
+++ b/crates/core_simd/src/vector/int.rs
@@ -12,7 +12,7 @@ macro_rules! impl_integer_vector {
             #[inline]
             fn cmp(&self, other: &Self) -> core::cmp::Ordering {
                 // TODO use SIMD cmp
-                self.to_array().cmp(other.as_ref())
+                self.as_array().cmp(other.as_ref())
             }
         }
 
@@ -22,7 +22,7 @@ macro_rules! impl_integer_vector {
             where
                 H: core::hash::Hasher
             {
-                self.as_slice().hash(state)
+                self.as_array().hash(state)
             }
         }
 
diff --git a/crates/core_simd/src/vector/mod.rs b/crates/core_simd/src/vector/mod.rs
index d1940ab9833d..a7adca80f8bd 100644
--- a/crates/core_simd/src/vector/mod.rs
+++ b/crates/core_simd/src/vector/mod.rs
@@ -30,12 +30,6 @@ pub trait Vector: sealed::Sealed {
     /// Generates a SIMD vector with the same value in every lane.
     #[must_use]
     fn splat(val: Self::Scalar) -> Self;
-
-    /// Returns a slice containing the entire SIMD vector.
-    fn as_slice(&self) -> &[Self::Scalar];
-
-    /// Returns a mutable slice containing the entire SIMD vector.
-    fn as_mut_slice(&mut self) -> &mut [Self::Scalar];
 }
 
 macro_rules! impl_vector_for {
@@ -60,16 +54,6 @@ macro_rules! impl_vector_for {
             fn splat(val: Self::Scalar) -> Self {
                 Self::splat(val)
             }
-
-            #[inline]
-            fn as_slice(&self) -> &[Self::Scalar] {
-                self.as_slice()
-            }
-
-            #[inline]
-            fn as_mut_slice(&mut self) -> &mut [Self::Scalar] {
-                self.as_mut_slice()
-            }
         }
     };
 }
diff --git a/crates/core_simd/src/vector/uint.rs b/crates/core_simd/src/vector/uint.rs
index 1340bec06f36..e7a6a8880121 100644
--- a/crates/core_simd/src/vector/uint.rs
+++ b/crates/core_simd/src/vector/uint.rs
@@ -12,7 +12,7 @@ macro_rules! impl_unsigned_vector {
             #[inline]
             fn cmp(&self, other: &Self) -> core::cmp::Ordering {
                 // TODO use SIMD cmp
-                self.to_array().cmp(other.as_ref())
+                self.as_array().cmp(other.as_ref())
             }
         }
 
@@ -22,7 +22,7 @@ macro_rules! impl_unsigned_vector {
             where
                 H: core::hash::Hasher
             {
-                self.as_slice().hash(state)
+                self.as_array().hash(state)
             }
         }
     }

From 529ffe05d6111afe110226ba6ebc577d1414d452 Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Tue, 29 Jun 2021 21:48:54 +0000
Subject: [PATCH 196/249] Use new module naming

---
 crates/core_simd/src/{masks/mod.rs => masks.rs}   | 4 ++--
 crates/core_simd/src/{vector/mod.rs => vector.rs} | 0
 2 files changed, 2 insertions(+), 2 deletions(-)
 rename crates/core_simd/src/{masks/mod.rs => masks.rs} (99%)
 rename crates/core_simd/src/{vector/mod.rs => vector.rs} (100%)

diff --git a/crates/core_simd/src/masks/mod.rs b/crates/core_simd/src/masks.rs
similarity index 99%
rename from crates/core_simd/src/masks/mod.rs
rename to crates/core_simd/src/masks.rs
index 43e689d45bc7..a4ed3b4a071c 100644
--- a/crates/core_simd/src/masks/mod.rs
+++ b/crates/core_simd/src/masks.rs
@@ -4,11 +4,11 @@
 
 #[cfg_attr(
     not(all(target_arch = "x86_64", target_feature = "avx512f")),
-    path = "full_masks.rs"
+    path = "masks/full_masks.rs"
 )]
 #[cfg_attr(
     all(target_arch = "x86_64", target_feature = "avx512f"),
-    path = "bitmask.rs"
+    path = "masks/bitmask.rs"
 )]
 mod mask_impl;
 
diff --git a/crates/core_simd/src/vector/mod.rs b/crates/core_simd/src/vector.rs
similarity index 100%
rename from crates/core_simd/src/vector/mod.rs
rename to crates/core_simd/src/vector.rs

From f93bef35f35579b60c8b919844a2021f3f035daa Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Tue, 29 Jun 2021 22:08:50 +0000
Subject: [PATCH 197/249] Move vector implementation

---
 crates/core_simd/src/lib.rs                   |   2 -
 crates/core_simd/src/vector.rs                | 101 +-----------------
 .../src/{first.rs => vector/vector_impl.rs}   |  30 ++++++
 3 files changed, 33 insertions(+), 100 deletions(-)
 rename crates/core_simd/src/{first.rs => vector/vector_impl.rs} (90%)

diff --git a/crates/core_simd/src/lib.rs b/crates/core_simd/src/lib.rs
index eb580dcf14d8..2f0f851a92f1 100644
--- a/crates/core_simd/src/lib.rs
+++ b/crates/core_simd/src/lib.rs
@@ -12,8 +12,6 @@
 #![unstable(feature = "portable_simd", issue = "86656")]
 //! Portable SIMD module.
 
-#[macro_use]
-mod first;
 #[macro_use]
 mod permute;
 #[macro_use]
diff --git a/crates/core_simd/src/vector.rs b/crates/core_simd/src/vector.rs
index a7adca80f8bd..2bc7c1cd2b74 100644
--- a/crates/core_simd/src/vector.rs
+++ b/crates/core_simd/src/vector.rs
@@ -1,3 +1,6 @@
+#[macro_use]
+mod vector_impl;
+
 mod float;
 mod int;
 mod uint;
@@ -31,101 +34,3 @@ pub trait Vector: sealed::Sealed {
     #[must_use]
     fn splat(val: Self::Scalar) -> Self;
 }
-
-macro_rules! impl_vector_for {
-    ($simd:ident {type Scalar = $scalar:ident;}) => {
-        impl_vector_for! { $simd<1> { type Scalar = $scalar; type BitMask = u8; } }
-        impl_vector_for! { $simd<2> { type Scalar = $scalar; type BitMask = u8; } }
-        impl_vector_for! { $simd<4> { type Scalar = $scalar; type BitMask = u8; } }
-        impl_vector_for! { $simd<8> { type Scalar = $scalar; type BitMask = u8; } }
-        impl_vector_for! { $simd<16> { type Scalar = $scalar; type BitMask = u16; } }
-        impl_vector_for! { $simd<32> { type Scalar = $scalar; type BitMask = u32; } }
-    };
-    ($simd:ident<$lanes:literal> {type Scalar = $scalar:ident; type BitMask = $bitmask:ident; }) => {
-        impl sealed::Sealed for $simd<$lanes> {}
-
-        impl Vector for $simd<$lanes> {
-            type Scalar = $scalar;
-            const LANES: usize = $lanes;
-
-            type BitMask = $bitmask;
-
-            #[inline]
-            fn splat(val: Self::Scalar) -> Self {
-                Self::splat(val)
-            }
-        }
-    };
-}
-
-impl_vector_for! {
-    SimdUsize {
-        type Scalar = usize;
-    }
-}
-
-impl_vector_for! {
-    SimdIsize {
-        type Scalar = isize;
-    }
-}
-
-impl_vector_for! {
-    SimdI8 {
-        type Scalar = i8;
-    }
-}
-
-impl_vector_for! {
-    SimdI16 {
-        type Scalar = i16;
-    }
-}
-
-impl_vector_for! {
-    SimdI32 {
-        type Scalar = i32;
-    }
-}
-
-impl_vector_for! {
-    SimdI64 {
-        type Scalar = i64;
-    }
-}
-
-impl_vector_for! {
-    SimdU8 {
-        type Scalar = u8;
-    }
-}
-
-impl_vector_for! {
-    SimdU16 {
-        type Scalar = u16;
-    }
-}
-
-impl_vector_for! {
-    SimdU32 {
-        type Scalar = u32;
-    }
-}
-
-impl_vector_for! {
-    SimdU64 {
-        type Scalar = u64;
-    }
-}
-
-impl_vector_for! {
-    SimdF32 {
-        type Scalar = f32;
-    }
-}
-
-impl_vector_for! {
-    SimdF64 {
-        type Scalar = f64;
-    }
-}
diff --git a/crates/core_simd/src/first.rs b/crates/core_simd/src/vector/vector_impl.rs
similarity index 90%
rename from crates/core_simd/src/first.rs
rename to crates/core_simd/src/vector/vector_impl.rs
index 4560a0a1467c..e166a93670b8 100644
--- a/crates/core_simd/src/first.rs
+++ b/crates/core_simd/src/vector/vector_impl.rs
@@ -1,6 +1,36 @@
+macro_rules! impl_vector_trait {
+    ($simd:ident {type Scalar = $scalar:ty;}) => {
+        impl_vector_trait! { $simd<1> { type Scalar = $scalar; type BitMask = u8; } }
+        impl_vector_trait! { $simd<2> { type Scalar = $scalar; type BitMask = u8; } }
+        impl_vector_trait! { $simd<4> { type Scalar = $scalar; type BitMask = u8; } }
+        impl_vector_trait! { $simd<8> { type Scalar = $scalar; type BitMask = u8; } }
+        impl_vector_trait! { $simd<16> { type Scalar = $scalar; type BitMask = u16; } }
+        impl_vector_trait! { $simd<32> { type Scalar = $scalar; type BitMask = u32; } }
+    };
+    ($simd:ident<$lanes:literal> {type Scalar = $scalar:ty; type BitMask = $bitmask:ident; }) => {
+        impl crate::vector::sealed::Sealed for $simd<$lanes> {}
+
+        impl crate::vector::Vector for $simd<$lanes> {
+            type Scalar = $scalar;
+            const LANES: usize = $lanes;
+
+            type BitMask = $bitmask;
+
+            #[inline]
+            fn splat(val: Self::Scalar) -> Self {
+                Self::splat(val)
+            }
+        }
+    };
+}
+
 /// Implements common traits on the specified vector `$name`, holding multiple `$lanes` of `$type`.
 macro_rules! impl_vector {
     { $name:ident, $type:ty } => {
+        impl_vector_trait! {
+            $name { type Scalar = $type; }
+        }
+
         impl<const LANES: usize> $name<LANES> where Self: crate::Vector {
             /// Construct a SIMD vector by setting all lanes to the given value.
             pub const fn splat(value: $type) -> Self {

From 97c25dd7465f4db60c013d7688b809a7da5388a6 Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Sat, 24 Jul 2021 02:54:19 +0000
Subject: [PATCH 198/249] Add lane count marker type

---
 crates/core_simd/src/comparisons.rs        |   6 +-
 crates/core_simd/src/fmt.rs                |   2 +-
 crates/core_simd/src/iter.rs               |  10 +-
 crates/core_simd/src/lane_count.rs         |  43 ++++++
 crates/core_simd/src/lib.rs                |   4 +-
 crates/core_simd/src/masks.rs              | 150 ++++++++------------
 crates/core_simd/src/masks/bitmask.rs      | 151 +++++++++++----------
 crates/core_simd/src/masks/full_masks.rs   | 101 +++++++-------
 crates/core_simd/src/math.rs               |   4 +-
 crates/core_simd/src/ops.rs                |  78 +++++------
 crates/core_simd/src/reduction.rs          |  11 +-
 crates/core_simd/src/round.rs              |   8 +-
 crates/core_simd/src/select.rs             |  16 +--
 crates/core_simd/src/to_bytes.rs           |  12 +-
 crates/core_simd/src/vector.rs             |   4 -
 crates/core_simd/src/vector/float.rs       |  17 +--
 crates/core_simd/src/vector/int.rs         |  25 ++--
 crates/core_simd/src/vector/ptr.rs         |   6 +-
 crates/core_simd/src/vector/uint.rs        |  21 +--
 crates/core_simd/src/vector/vector_impl.rs |  73 ++++------
 crates/test_helpers/src/lib.rs             |  36 +----
 21 files changed, 367 insertions(+), 411 deletions(-)
 create mode 100644 crates/core_simd/src/lane_count.rs

diff --git a/crates/core_simd/src/comparisons.rs b/crates/core_simd/src/comparisons.rs
index c3bf07fc4320..c5e9be9015fe 100644
--- a/crates/core_simd/src/comparisons.rs
+++ b/crates/core_simd/src/comparisons.rs
@@ -1,13 +1,11 @@
-use crate::Vector;
+use crate::{LaneCount, SupportedLaneCount};
 
 macro_rules! implement_mask_ops {
     { $($vector:ident => $mask:ident ($inner_ty:ident),)* } => {
         $(
             impl<const LANES: usize> crate::$vector<LANES>
             where
-                crate::$vector<LANES>: Vector,
-                crate::$inner_ty<LANES>: Vector,
-                crate::$mask<LANES>: crate::Mask,
+                LaneCount<LANES>: SupportedLaneCount,
             {
                 /// Test if each lane is equal to the corresponding lane in `other`.
                 #[inline]
diff --git a/crates/core_simd/src/fmt.rs b/crates/core_simd/src/fmt.rs
index 554aa91a5f0c..78ae5ce3fcea 100644
--- a/crates/core_simd/src/fmt.rs
+++ b/crates/core_simd/src/fmt.rs
@@ -35,7 +35,7 @@ macro_rules! impl_fmt_trait {
             $( // repeat trait
                 impl<const LANES: usize> core::fmt::$trait for crate::$type<LANES>
                 where
-                    Self: crate::Vector,
+                    crate::LaneCount<LANES>: crate::SupportedLaneCount,
                 {
                     fn fmt(&self, f: &mut core::fmt::Formatter) -> core::fmt::Result {
                         $format(self.as_ref(), f)
diff --git a/crates/core_simd/src/iter.rs b/crates/core_simd/src/iter.rs
index 460c061be69f..0020ea5f2016 100644
--- a/crates/core_simd/src/iter.rs
+++ b/crates/core_simd/src/iter.rs
@@ -1,8 +1,10 @@
+use crate::{LaneCount, SupportedLaneCount};
+
 macro_rules! impl_traits {
     { $type:ident } => {
         impl<const LANES: usize> core::iter::Sum<Self> for crate::$type<LANES>
         where
-            Self: crate::Vector,
+            LaneCount<LANES>: SupportedLaneCount,
         {
             fn sum<I: core::iter::Iterator<Item = Self>>(iter: I) -> Self {
                 iter.fold(Default::default(), core::ops::Add::add)
@@ -11,7 +13,7 @@ macro_rules! impl_traits {
 
         impl<const LANES: usize> core::iter::Product<Self> for crate::$type<LANES>
         where
-            Self: crate::Vector,
+            LaneCount<LANES>: SupportedLaneCount,
         {
             fn product<I: core::iter::Iterator<Item = Self>>(iter: I) -> Self {
                 iter.fold(Default::default(), core::ops::Mul::mul)
@@ -20,7 +22,7 @@ macro_rules! impl_traits {
 
         impl<'a, const LANES: usize> core::iter::Sum<&'a Self> for crate::$type<LANES>
         where
-            Self: crate::Vector,
+            LaneCount<LANES>: SupportedLaneCount,
         {
             fn sum<I: core::iter::Iterator<Item = &'a Self>>(iter: I) -> Self {
                 iter.fold(Default::default(), core::ops::Add::add)
@@ -29,7 +31,7 @@ macro_rules! impl_traits {
 
         impl<'a, const LANES: usize> core::iter::Product<&'a Self> for crate::$type<LANES>
         where
-            Self: crate::Vector,
+            LaneCount<LANES>: SupportedLaneCount,
         {
             fn product<I: core::iter::Iterator<Item = &'a Self>>(iter: I) -> Self {
                 iter.fold(Default::default(), core::ops::Mul::mul)
diff --git a/crates/core_simd/src/lane_count.rs b/crates/core_simd/src/lane_count.rs
new file mode 100644
index 000000000000..8fe204dff98e
--- /dev/null
+++ b/crates/core_simd/src/lane_count.rs
@@ -0,0 +1,43 @@
+mod sealed {
+    pub trait Sealed {}
+}
+use sealed::Sealed;
+
+/// A type representing a vector lane count.
+pub struct LaneCount<const LANES: usize>;
+
+/// Helper trait for vector lane counts.
+pub trait SupportedLaneCount: Sealed {
+    /// The bitmask representation of a mask.
+    type BitMask: Copy + Default + AsRef<[u8]> + AsMut<[u8]>;
+
+    #[doc(hidden)]
+    type IntBitMask;
+}
+
+impl<const LANES: usize> Sealed for LaneCount<LANES> {}
+
+impl SupportedLaneCount for LaneCount<1> {
+    type BitMask = [u8; 1];
+    type IntBitMask = u8;
+}
+impl SupportedLaneCount for LaneCount<2> {
+    type BitMask = [u8; 1];
+    type IntBitMask = u8;
+}
+impl SupportedLaneCount for LaneCount<4> {
+    type BitMask = [u8; 1];
+    type IntBitMask = u8;
+}
+impl SupportedLaneCount for LaneCount<8> {
+    type BitMask = [u8; 1];
+    type IntBitMask = u8;
+}
+impl SupportedLaneCount for LaneCount<16> {
+    type BitMask = [u8; 2];
+    type IntBitMask = u16;
+}
+impl SupportedLaneCount for LaneCount<32> {
+    type BitMask = [u8; 4];
+    type IntBitMask = u32;
+}
diff --git a/crates/core_simd/src/lib.rs b/crates/core_simd/src/lib.rs
index 2f0f851a92f1..d8149efe9c7f 100644
--- a/crates/core_simd/src/lib.rs
+++ b/crates/core_simd/src/lib.rs
@@ -27,11 +27,13 @@ mod comparisons;
 mod fmt;
 mod intrinsics;
 mod iter;
+mod math;
 mod ops;
 mod round;
 mod vendor;
 
-mod math;
+mod lane_count;
+pub use lane_count::*;
 
 mod masks;
 pub use masks::*;
diff --git a/crates/core_simd/src/masks.rs b/crates/core_simd/src/masks.rs
index a4ed3b4a071c..d3338a6d366e 100644
--- a/crates/core_simd/src/masks.rs
+++ b/crates/core_simd/src/masks.rs
@@ -12,7 +12,7 @@
 )]
 mod mask_impl;
 
-use crate::{SimdI16, SimdI32, SimdI64, SimdI8, SimdIsize, Vector};
+use crate::{SimdI16, SimdI32, SimdI64, SimdI8, SimdIsize};
 
 mod sealed {
     pub trait Sealed {}
@@ -20,12 +20,12 @@ mod sealed {
 
 /// Helper trait for mask types.
 pub trait Mask: sealed::Sealed {
-    /// The bitmask representation of a mask.
-    type BitMask: Copy + Default + AsRef<[u8]> + AsMut<[u8]>;
+    /// The number of lanes for this mask.
+    const LANES: usize;
 
-    // TODO remove this when rustc intrinsics are more flexible
-    #[doc(hidden)]
-    type IntBitMask;
+    /// Generates a mask with the same value in every lane.
+    #[must_use]
+    fn splat(val: bool) -> Self;
 }
 
 macro_rules! define_opaque_mask {
@@ -38,45 +38,30 @@ macro_rules! define_opaque_mask {
         #[allow(non_camel_case_types)]
         pub struct $name<const LANES: usize>($inner_ty)
         where
-            $bits_ty<LANES>: Vector,
-            Self: Mask;
+            crate::LaneCount<LANES>: crate::SupportedLaneCount;
 
         impl<const LANES: usize> sealed::Sealed for $name<LANES>
         where
-            $bits_ty<LANES>: Vector,
-            Self: Mask,
+            crate::LaneCount<LANES>: crate::SupportedLaneCount,
         {}
-        impl Mask for $name<1> {
-            type BitMask = [u8; 1];
-            type IntBitMask = u8;
-        }
-        impl Mask for $name<2> {
-            type BitMask = [u8; 1];
-            type IntBitMask = u8;
-        }
-        impl Mask for $name<4> {
-            type BitMask = [u8; 1];
-            type IntBitMask = u8;
-        }
-        impl Mask for $name<8> {
-            type BitMask = [u8; 1];
-            type IntBitMask = u8;
-        }
-        impl Mask for $name<16> {
-            type BitMask = [u8; 2];
-            type IntBitMask = u16;
-        }
-        impl Mask for $name<32> {
-            type BitMask = [u8; 4];
-            type IntBitMask = u32;
+
+        impl<const LANES: usize> Mask for $name<LANES>
+        where
+            crate::LaneCount<LANES>: crate::SupportedLaneCount,
+        {
+            const LANES: usize = LANES;
+
+            #[inline]
+            fn splat(value: bool) -> Self {
+                Self::splat(value)
+            }
         }
 
         impl_opaque_mask_reductions! { $name, $bits_ty }
 
         impl<const LANES: usize> $name<LANES>
         where
-            $bits_ty<LANES>: Vector,
-            Self: Mask,
+            crate::LaneCount<LANES>: crate::SupportedLaneCount,
         {
             /// Construct a mask by setting all lanes to the given value.
             pub fn splat(value: bool) -> Self {
@@ -175,21 +160,20 @@ macro_rules! define_opaque_mask {
             }
 
             /// Convert this mask to a bitmask, with one bit set per lane.
-            pub fn to_bitmask(self) -> <Self as Mask>::BitMask {
-                self.0.to_bitmask::<Self>()
+            pub fn to_bitmask(self) -> <crate::LaneCount<LANES> as crate::SupportedLaneCount>::BitMask {
+                self.0.to_bitmask()
             }
 
             /// Convert a bitmask to a mask.
-            pub fn from_bitmask(bitmask: <Self as Mask>::BitMask) -> Self {
-                Self(<$inner_ty>::from_bitmask::<Self>(bitmask))
+            pub fn from_bitmask(bitmask: <crate::LaneCount<LANES> as crate::SupportedLaneCount>::BitMask) -> Self {
+                Self(<$inner_ty>::from_bitmask(bitmask))
             }
         }
 
         // vector/array conversion
         impl<const LANES: usize> From<[bool; LANES]> for $name<LANES>
         where
-            $bits_ty<LANES>: crate::Vector,
-            Self: Mask,
+            crate::LaneCount<LANES>: crate::SupportedLaneCount,
         {
             fn from(array: [bool; LANES]) -> Self {
                 Self::from_array(array)
@@ -198,8 +182,7 @@ macro_rules! define_opaque_mask {
 
         impl <const LANES: usize> From<$name<LANES>> for [bool; LANES]
         where
-            $bits_ty<LANES>: crate::Vector,
-            $name<LANES>: Mask,
+            crate::LaneCount<LANES>: crate::SupportedLaneCount,
         {
             fn from(vector: $name<LANES>) -> Self {
                 vector.to_array()
@@ -208,14 +191,12 @@ macro_rules! define_opaque_mask {
 
         impl<const LANES: usize> Copy for $name<LANES>
         where
-            $bits_ty<LANES>: Vector,
-            Self: Mask,
+            crate::LaneCount<LANES>: crate::SupportedLaneCount,
         {}
 
         impl<const LANES: usize> Clone for $name<LANES>
         where
-            $bits_ty<LANES>: Vector,
-            Self: Mask,
+            crate::LaneCount<LANES>: crate::SupportedLaneCount,
         {
             #[inline]
             fn clone(&self) -> Self {
@@ -225,8 +206,7 @@ macro_rules! define_opaque_mask {
 
         impl<const LANES: usize> Default for $name<LANES>
         where
-            $bits_ty<LANES>: Vector,
-            Self: Mask,
+            crate::LaneCount<LANES>: crate::SupportedLaneCount,
         {
             #[inline]
             fn default() -> Self {
@@ -236,8 +216,7 @@ macro_rules! define_opaque_mask {
 
         impl<const LANES: usize> PartialEq for $name<LANES>
         where
-            $bits_ty<LANES>: Vector,
-            Self: Mask,
+            crate::LaneCount<LANES>: crate::SupportedLaneCount,
         {
             #[inline]
             fn eq(&self, other: &Self) -> bool {
@@ -247,8 +226,7 @@ macro_rules! define_opaque_mask {
 
         impl<const LANES: usize> PartialOrd for $name<LANES>
         where
-            $bits_ty<LANES>: Vector,
-            Self: Mask,
+            crate::LaneCount<LANES>: crate::SupportedLaneCount,
         {
             #[inline]
             fn partial_cmp(&self, other: &Self) -> Option<core::cmp::Ordering> {
@@ -258,8 +236,7 @@ macro_rules! define_opaque_mask {
 
         impl<const LANES: usize> core::fmt::Debug for $name<LANES>
         where
-            $bits_ty<LANES>: crate::Vector,
-            Self: Mask,
+            crate::LaneCount<LANES>: crate::SupportedLaneCount,
         {
             fn fmt(&self, f: &mut core::fmt::Formatter) -> core::fmt::Result {
                 f.debug_list()
@@ -270,8 +247,7 @@ macro_rules! define_opaque_mask {
 
         impl<const LANES: usize> core::ops::BitAnd for $name<LANES>
         where
-            $bits_ty<LANES>: Vector,
-            Self: Mask,
+            crate::LaneCount<LANES>: crate::SupportedLaneCount,
         {
             type Output = Self;
             #[inline]
@@ -282,8 +258,7 @@ macro_rules! define_opaque_mask {
 
         impl<const LANES: usize> core::ops::BitAnd<bool> for $name<LANES>
         where
-            $bits_ty<LANES>: Vector,
-            Self: Mask,
+            crate::LaneCount<LANES>: crate::SupportedLaneCount,
         {
             type Output = Self;
             #[inline]
@@ -294,8 +269,7 @@ macro_rules! define_opaque_mask {
 
         impl<const LANES: usize> core::ops::BitAnd<$name<LANES>> for bool
         where
-            $bits_ty<LANES>: Vector,
-            $name<LANES>: Mask,
+            crate::LaneCount<LANES>: crate::SupportedLaneCount,
         {
             type Output = $name<LANES>;
             #[inline]
@@ -306,8 +280,7 @@ macro_rules! define_opaque_mask {
 
         impl<const LANES: usize> core::ops::BitOr for $name<LANES>
         where
-            $bits_ty<LANES>: Vector,
-            Self: Mask,
+            crate::LaneCount<LANES>: crate::SupportedLaneCount,
         {
             type Output = Self;
             #[inline]
@@ -318,8 +291,7 @@ macro_rules! define_opaque_mask {
 
         impl<const LANES: usize> core::ops::BitOr<bool> for $name<LANES>
         where
-            $bits_ty<LANES>: Vector,
-            Self: Mask,
+            crate::LaneCount<LANES>: crate::SupportedLaneCount,
         {
             type Output = Self;
             #[inline]
@@ -330,8 +302,7 @@ macro_rules! define_opaque_mask {
 
         impl<const LANES: usize> core::ops::BitOr<$name<LANES>> for bool
         where
-            $bits_ty<LANES>: Vector,
-            $name<LANES>: Mask,
+            crate::LaneCount<LANES>: crate::SupportedLaneCount,
         {
             type Output = $name<LANES>;
             #[inline]
@@ -342,8 +313,7 @@ macro_rules! define_opaque_mask {
 
         impl<const LANES: usize> core::ops::BitXor for $name<LANES>
         where
-            $bits_ty<LANES>: Vector,
-            Self: Mask,
+            crate::LaneCount<LANES>: crate::SupportedLaneCount,
         {
             type Output = Self;
             #[inline]
@@ -354,8 +324,7 @@ macro_rules! define_opaque_mask {
 
         impl<const LANES: usize> core::ops::BitXor<bool> for $name<LANES>
         where
-            $bits_ty<LANES>: Vector,
-            Self: Mask,
+            crate::LaneCount<LANES>: crate::SupportedLaneCount,
         {
             type Output = Self;
             #[inline]
@@ -366,8 +335,7 @@ macro_rules! define_opaque_mask {
 
         impl<const LANES: usize> core::ops::BitXor<$name<LANES>> for bool
         where
-            $bits_ty<LANES>: Vector,
-            $name<LANES>: Mask,
+            crate::LaneCount<LANES>: crate::SupportedLaneCount,
         {
             type Output = $name<LANES>;
             #[inline]
@@ -378,8 +346,7 @@ macro_rules! define_opaque_mask {
 
         impl<const LANES: usize> core::ops::Not for $name<LANES>
         where
-            $bits_ty<LANES>: Vector,
-            Self: Mask,
+            crate::LaneCount<LANES>: crate::SupportedLaneCount,
         {
             type Output = $name<LANES>;
             #[inline]
@@ -390,8 +357,7 @@ macro_rules! define_opaque_mask {
 
         impl<const LANES: usize> core::ops::BitAndAssign for $name<LANES>
         where
-            $bits_ty<LANES>: Vector,
-            Self: Mask,
+            crate::LaneCount<LANES>: crate::SupportedLaneCount,
         {
             #[inline]
             fn bitand_assign(&mut self, rhs: Self) {
@@ -401,8 +367,7 @@ macro_rules! define_opaque_mask {
 
         impl<const LANES: usize> core::ops::BitAndAssign<bool> for $name<LANES>
         where
-            $bits_ty<LANES>: Vector,
-            Self: Mask,
+            crate::LaneCount<LANES>: crate::SupportedLaneCount,
         {
             #[inline]
             fn bitand_assign(&mut self, rhs: bool) {
@@ -412,8 +377,7 @@ macro_rules! define_opaque_mask {
 
         impl<const LANES: usize> core::ops::BitOrAssign for $name<LANES>
         where
-            $bits_ty<LANES>: Vector,
-            Self: Mask,
+            crate::LaneCount<LANES>: crate::SupportedLaneCount,
         {
             #[inline]
             fn bitor_assign(&mut self, rhs: Self) {
@@ -423,8 +387,7 @@ macro_rules! define_opaque_mask {
 
         impl<const LANES: usize> core::ops::BitOrAssign<bool> for $name<LANES>
         where
-            $bits_ty<LANES>: Vector,
-            Self: Mask,
+            crate::LaneCount<LANES>: crate::SupportedLaneCount,
         {
             #[inline]
             fn bitor_assign(&mut self, rhs: bool) {
@@ -434,8 +397,7 @@ macro_rules! define_opaque_mask {
 
         impl<const LANES: usize> core::ops::BitXorAssign for $name<LANES>
         where
-            $bits_ty<LANES>: Vector,
-            Self: Mask,
+            crate::LaneCount<LANES>: crate::SupportedLaneCount,
         {
             #[inline]
             fn bitxor_assign(&mut self, rhs: Self) {
@@ -445,8 +407,7 @@ macro_rules! define_opaque_mask {
 
         impl<const LANES: usize> core::ops::BitXorAssign<bool> for $name<LANES>
         where
-            $bits_ty<LANES>: Vector,
-            Self: Mask,
+            crate::LaneCount<LANES>: crate::SupportedLaneCount,
         {
             #[inline]
             fn bitxor_assign(&mut self, rhs: bool) {
@@ -460,7 +421,7 @@ define_opaque_mask! {
     /// Mask for vectors with `LANES` 8-bit elements.
     ///
     /// The layout of this type is unspecified.
-    struct Mask8<const LANES: usize>(mask_impl::Mask8<Self, LANES>);
+    struct Mask8<const LANES: usize>(mask_impl::Mask8<LANES>);
     @bits SimdI8
 }
 
@@ -468,7 +429,7 @@ define_opaque_mask! {
     /// Mask for vectors with `LANES` 16-bit elements.
     ///
     /// The layout of this type is unspecified.
-    struct Mask16<const LANES: usize>(mask_impl::Mask16<Self, LANES>);
+    struct Mask16<const LANES: usize>(mask_impl::Mask16<LANES>);
     @bits SimdI16
 }
 
@@ -476,7 +437,7 @@ define_opaque_mask! {
     /// Mask for vectors with `LANES` 32-bit elements.
     ///
     /// The layout of this type is unspecified.
-    struct Mask32<const LANES: usize>(mask_impl::Mask32<Self, LANES>);
+    struct Mask32<const LANES: usize>(mask_impl::Mask32<LANES>);
     @bits SimdI32
 }
 
@@ -484,7 +445,7 @@ define_opaque_mask! {
     /// Mask for vectors with `LANES` 64-bit elements.
     ///
     /// The layout of this type is unspecified.
-    struct Mask64<const LANES: usize>(mask_impl::Mask64<Self, LANES>);
+    struct Mask64<const LANES: usize>(mask_impl::Mask64<LANES>);
     @bits SimdI64
 }
 
@@ -492,7 +453,7 @@ define_opaque_mask! {
     /// Mask for vectors with `LANES` pointer-width elements.
     ///
     /// The layout of this type is unspecified.
-    struct MaskSize<const LANES: usize>(mask_impl::MaskSize<Self, LANES>);
+    struct MaskSize<const LANES: usize>(mask_impl::MaskSize<LANES>);
     @bits SimdIsize
 }
 
@@ -555,10 +516,7 @@ macro_rules! impl_from {
         $(
         impl<const LANES: usize> From<$from<LANES>> for $to<LANES>
         where
-            crate::$from_inner<LANES>: crate::Vector,
-            crate::$to_inner<LANES>: crate::Vector,
-            $from<LANES>: Mask,
-            Self: Mask,
+            crate::LaneCount<LANES>: crate::SupportedLaneCount,
         {
             fn from(value: $from<LANES>) -> Self {
                 Self(value.0.into())
diff --git a/crates/core_simd/src/masks/bitmask.rs b/crates/core_simd/src/masks/bitmask.rs
index a64750a623f7..b6897728988b 100644
--- a/crates/core_simd/src/masks/bitmask.rs
+++ b/crates/core_simd/src/masks/bitmask.rs
@@ -1,50 +1,81 @@
-use crate::Mask;
-use core::marker::PhantomData;
+use crate::{LaneCount, SupportedLaneCount};
 
 /// Helper trait for limiting int conversion types
 pub trait ConvertToInt {}
-impl<const LANES: usize> ConvertToInt for crate::SimdI8<LANES> where Self: crate::Vector {}
-impl<const LANES: usize> ConvertToInt for crate::SimdI16<LANES> where Self: crate::Vector {}
-impl<const LANES: usize> ConvertToInt for crate::SimdI32<LANES> where Self: crate::Vector {}
-impl<const LANES: usize> ConvertToInt for crate::SimdI64<LANES> where Self: crate::Vector {}
-impl<const LANES: usize> ConvertToInt for crate::SimdIsize<LANES> where Self: crate::Vector {}
+impl<const LANES: usize> ConvertToInt for crate::SimdI8<LANES> where
+    LaneCount<LANES>: SupportedLaneCount
+{
+}
+impl<const LANES: usize> ConvertToInt for crate::SimdI16<LANES> where
+    LaneCount<LANES>: SupportedLaneCount
+{
+}
+impl<const LANES: usize> ConvertToInt for crate::SimdI32<LANES> where
+    LaneCount<LANES>: SupportedLaneCount
+{
+}
+impl<const LANES: usize> ConvertToInt for crate::SimdI64<LANES> where
+    LaneCount<LANES>: SupportedLaneCount
+{
+}
+impl<const LANES: usize> ConvertToInt for crate::SimdIsize<LANES> where
+    LaneCount<LANES>: SupportedLaneCount
+{
+}
 
 /// A mask where each lane is represented by a single bit.
 #[repr(transparent)]
-pub struct BitMask<T: Mask, const LANES: usize>(T::BitMask, PhantomData<[(); LANES]>);
+pub struct BitMask<const LANES: usize>(<LaneCount<LANES> as SupportedLaneCount>::BitMask)
+where
+    LaneCount<LANES>: SupportedLaneCount;
 
-impl<T: Mask, const LANES: usize> Copy for BitMask<T, LANES> {}
+impl<const LANES: usize> Copy for BitMask<LANES> where LaneCount<LANES>: SupportedLaneCount {}
 
-impl<T: Mask, const LANES: usize> Clone for BitMask<T, LANES> {
+impl<const LANES: usize> Clone for BitMask<LANES>
+where
+    LaneCount<LANES>: SupportedLaneCount,
+{
     fn clone(&self) -> Self {
         *self
     }
 }
 
-impl<T: Mask, const LANES: usize> PartialEq for BitMask<T, LANES> {
+impl<const LANES: usize> PartialEq for BitMask<LANES>
+where
+    LaneCount<LANES>: SupportedLaneCount,
+{
     fn eq(&self, other: &Self) -> bool {
         self.0.as_ref() == other.0.as_ref()
     }
 }
 
-impl<T: Mask, const LANES: usize> PartialOrd for BitMask<T, LANES> {
+impl<const LANES: usize> PartialOrd for BitMask<LANES>
+where
+    LaneCount<LANES>: SupportedLaneCount,
+{
     fn partial_cmp(&self, other: &Self) -> Option<core::cmp::Ordering> {
         self.0.as_ref().partial_cmp(other.0.as_ref())
     }
 }
 
-impl<T: Mask, const LANES: usize> Eq for BitMask<T, LANES> {}
+impl<const LANES: usize> Eq for BitMask<LANES> where LaneCount<LANES>: SupportedLaneCount {}
 
-impl<T: Mask, const LANES: usize> Ord for BitMask<T, LANES> {
+impl<const LANES: usize> Ord for BitMask<LANES>
+where
+    LaneCount<LANES>: SupportedLaneCount,
+{
     fn cmp(&self, other: &Self) -> core::cmp::Ordering {
         self.0.as_ref().cmp(other.0.as_ref())
     }
 }
 
-impl<T: Mask, const LANES: usize> BitMask<T, LANES> {
+impl<const LANES: usize> BitMask<LANES>
+where
+    LaneCount<LANES>: SupportedLaneCount,
+{
     #[inline]
     pub fn splat(value: bool) -> Self {
-        let mut mask = T::BitMask::default();
+        let mut mask = <LaneCount<LANES> as SupportedLaneCount>::BitMask::default();
         if value {
             mask.as_mut().fill(u8::MAX)
         } else {
@@ -53,12 +84,12 @@ impl<T: Mask, const LANES: usize> BitMask<T, LANES> {
         if LANES % 8 > 0 {
             *mask.as_mut().last_mut().unwrap() &= u8::MAX >> (8 - LANES % 8);
         }
-        Self(mask, PhantomData)
+        Self(mask)
     }
 
     #[inline]
     pub unsafe fn test_unchecked(&self, lane: usize) -> bool {
-        (self.0.as_ref()[lane / 8] >> lane % 8) & 0x1 > 0
+        (self.0.as_ref()[lane / 8] >> (lane % 8)) & 0x1 > 0
     }
 
     #[inline]
@@ -72,7 +103,8 @@ impl<T: Mask, const LANES: usize> BitMask<T, LANES> {
         V: ConvertToInt + Default + core::ops::Not<Output = V>,
     {
         unsafe {
-            let mask: T::IntBitMask = core::mem::transmute_copy(&self);
+            let mask: <LaneCount<LANES> as SupportedLaneCount>::IntBitMask =
+                core::mem::transmute_copy(&self);
             crate::intrinsics::simd_select_bitmask(mask, !V::default(), V::default())
         }
     }
@@ -84,29 +116,25 @@ impl<T: Mask, const LANES: usize> BitMask<T, LANES> {
     {
         // TODO remove the transmute when rustc is more flexible
         assert_eq!(
-            core::mem::size_of::<T::IntBitMask>(),
-            core::mem::size_of::<T::BitMask>()
+            core::mem::size_of::<<crate::LaneCount::<LANES> as crate::SupportedLaneCount>::BitMask>(
+            ),
+            core::mem::size_of::<
+                <crate::LaneCount::<LANES> as crate::SupportedLaneCount>::IntBitMask,
+            >(),
         );
-        let mask: T::IntBitMask = crate::intrinsics::simd_bitmask(value);
-        Self(core::mem::transmute_copy(&mask), PhantomData)
+        let mask: <LaneCount<LANES> as SupportedLaneCount>::IntBitMask =
+            crate::intrinsics::simd_bitmask(value);
+        Self(core::mem::transmute_copy(&mask))
     }
 
     #[inline]
-    pub fn to_bitmask<U: Mask>(self) -> U::BitMask {
-        assert_eq!(
-            core::mem::size_of::<T::BitMask>(),
-            core::mem::size_of::<U::BitMask>()
-        );
-        unsafe { core::mem::transmute_copy(&self.0) }
+    pub fn to_bitmask(self) -> <LaneCount<LANES> as SupportedLaneCount>::BitMask {
+        self.0
     }
 
     #[inline]
-    pub fn from_bitmask<U: Mask>(bitmask: U::BitMask) -> Self {
-        assert_eq!(
-            core::mem::size_of::<T::BitMask>(),
-            core::mem::size_of::<U::BitMask>()
-        );
-        unsafe { core::mem::transmute_copy(&bitmask) }
+    pub fn from_bitmask(bitmask: <LaneCount<LANES> as SupportedLaneCount>::BitMask) -> Self {
+        Self(bitmask)
     }
 
     #[inline]
@@ -120,9 +148,10 @@ impl<T: Mask, const LANES: usize> BitMask<T, LANES> {
     }
 }
 
-impl<T: Mask, const LANES: usize> core::ops::BitAnd for BitMask<T, LANES>
+impl<const LANES: usize> core::ops::BitAnd for BitMask<LANES>
 where
-    T::BitMask: Default + AsRef<[u8]> + AsMut<[u8]>,
+    LaneCount<LANES>: SupportedLaneCount,
+    <LaneCount<LANES> as SupportedLaneCount>::BitMask: Default + AsRef<[u8]> + AsMut<[u8]>,
 {
     type Output = Self;
     #[inline]
@@ -134,9 +163,10 @@ where
     }
 }
 
-impl<T: Mask, const LANES: usize> core::ops::BitOr for BitMask<T, LANES>
+impl<const LANES: usize> core::ops::BitOr for BitMask<LANES>
 where
-    T::BitMask: Default + AsRef<[u8]> + AsMut<[u8]>,
+    LaneCount<LANES>: SupportedLaneCount,
+    <LaneCount<LANES> as SupportedLaneCount>::BitMask: Default + AsRef<[u8]> + AsMut<[u8]>,
 {
     type Output = Self;
     #[inline]
@@ -148,7 +178,10 @@ where
     }
 }
 
-impl<T: Mask, const LANES: usize> core::ops::BitXor for BitMask<T, LANES> {
+impl<const LANES: usize> core::ops::BitXor for BitMask<LANES>
+where
+    LaneCount<LANES>: SupportedLaneCount,
+{
     type Output = Self;
     #[inline]
     fn bitxor(mut self, rhs: Self) -> Self::Output {
@@ -159,7 +192,10 @@ impl<T: Mask, const LANES: usize> core::ops::BitXor for BitMask<T, LANES> {
     }
 }
 
-impl<T: Mask, const LANES: usize> core::ops::Not for BitMask<T, LANES> {
+impl<const LANES: usize> core::ops::Not for BitMask<LANES>
+where
+    LaneCount<LANES>: SupportedLaneCount,
+{
     type Output = Self;
     #[inline]
     fn not(mut self) -> Self::Output {
@@ -173,31 +209,8 @@ impl<T: Mask, const LANES: usize> core::ops::Not for BitMask<T, LANES> {
     }
 }
 
-pub type Mask8<T, const LANES: usize> = BitMask<T, LANES>;
-pub type Mask16<T, const LANES: usize> = BitMask<T, LANES>;
-pub type Mask32<T, const LANES: usize> = BitMask<T, LANES>;
-pub type Mask64<T, const LANES: usize> = BitMask<T, LANES>;
-pub type MaskSize<T, const LANES: usize> = BitMask<T, LANES>;
-
-macro_rules! impl_from {
-    { $from:ident ($from_inner:ident) => $($to:ident ($to_inner:ident)),* } => {
-        $(
-        impl<const LANES: usize> From<$from<crate::$from<LANES>, LANES>> for $to<crate::$to<LANES>, LANES>
-        where
-            crate::$from_inner<LANES>: crate::Vector,
-            crate::$to_inner<LANES>: crate::Vector,
-            crate::$from<LANES>: crate::Mask,
-            crate::$to<LANES>: crate::Mask,
-        {
-            fn from(value: $from<crate::$from<LANES>, LANES>) -> Self {
-                unsafe { core::mem::transmute_copy(&value) }
-            }
-        }
-        )*
-    }
-}
-impl_from! { Mask8 (SimdI8) => Mask16 (SimdI16), Mask32 (SimdI32), Mask64 (SimdI64), MaskSize (SimdIsize) }
-impl_from! { Mask16 (SimdI16) => Mask32 (SimdI32), Mask64 (SimdI64), MaskSize (SimdIsize), Mask8 (SimdI8) }
-impl_from! { Mask32 (SimdI32) => Mask64 (SimdI64), MaskSize (SimdIsize), Mask8 (SimdI8), Mask16 (SimdI16) }
-impl_from! { Mask64 (SimdI64) => MaskSize (SimdIsize), Mask8 (SimdI8), Mask16 (SimdI16), Mask32 (SimdI32) }
-impl_from! { MaskSize (SimdIsize) => Mask8 (SimdI8), Mask16 (SimdI16), Mask32 (SimdI32), Mask64 (SimdI64) }
+pub type Mask8<const LANES: usize> = BitMask<LANES>;
+pub type Mask16<const LANES: usize> = BitMask<LANES>;
+pub type Mask32<const LANES: usize> = BitMask<LANES>;
+pub type Mask64<const LANES: usize> = BitMask<LANES>;
+pub type MaskSize<const LANES: usize> = BitMask<LANES>;
diff --git a/crates/core_simd/src/masks/full_masks.rs b/crates/core_simd/src/masks/full_masks.rs
index f083284df8fa..af36571134ee 100644
--- a/crates/core_simd/src/masks/full_masks.rs
+++ b/crates/core_simd/src/masks/full_masks.rs
@@ -1,8 +1,5 @@
 //! Masks that take up full SIMD vector registers.
 
-use crate::Mask;
-use core::marker::PhantomData;
-
 macro_rules! define_mask {
     {
         $(#[$attr:meta])*
@@ -12,20 +9,20 @@ macro_rules! define_mask {
     } => {
         $(#[$attr])*
         #[repr(transparent)]
-        pub struct $name<T: Mask, const $lanes: usize>(crate::$type<$lanes2>, PhantomData<T>)
+        pub struct $name<const $lanes: usize>(crate::$type<$lanes>)
         where
-            crate::$type<LANES>: crate::Vector;
+            crate::LaneCount<$lanes>: crate::SupportedLaneCount;
 
         impl_full_mask_reductions! { $name, $type }
 
-        impl<T: Mask, const LANES: usize> Copy for $name<T, LANES>
+        impl<const LANES: usize> Copy for $name<LANES>
         where
-            crate::$type<LANES>: crate::Vector,
+            crate::LaneCount<LANES>: crate::SupportedLaneCount,
         {}
 
-        impl<T: Mask, const LANES: usize> Clone for $name<T, LANES>
+        impl<const LANES: usize> Clone for $name<LANES>
         where
-            crate::$type<LANES>: crate::Vector,
+            crate::LaneCount<LANES>: crate::SupportedLaneCount,
         {
             #[inline]
             fn clone(&self) -> Self {
@@ -33,41 +30,41 @@ macro_rules! define_mask {
             }
         }
 
-        impl<T: Mask, const LANES: usize> PartialEq for $name<T, LANES>
+        impl<const LANES: usize> PartialEq for $name<LANES>
         where
-            crate::$type<LANES>: crate::Vector,
+            crate::LaneCount<LANES>: crate::SupportedLaneCount,
         {
             fn eq(&self, other: &Self) -> bool {
                 self.0 == other.0
             }
         }
 
-        impl<T: Mask, const LANES: usize> PartialOrd for $name<T, LANES>
+        impl<const LANES: usize> PartialOrd for $name<LANES>
         where
-            crate::$type<LANES>: crate::Vector,
+            crate::LaneCount<LANES>: crate::SupportedLaneCount,
         {
             fn partial_cmp(&self, other: &Self) -> Option<core::cmp::Ordering> {
                 self.0.partial_cmp(&other.0)
             }
         }
 
-        impl<T: Mask, const LANES: usize> Eq for $name<T, LANES>
+        impl<const LANES: usize> Eq for $name<LANES>
         where
-            crate::$type<LANES>: crate::Vector,
+            crate::LaneCount<LANES>: crate::SupportedLaneCount,
         {}
 
-        impl<T: Mask, const LANES: usize> Ord for $name<T, LANES>
+        impl<const LANES: usize> Ord for $name<LANES>
         where
-            crate::$type<LANES>: crate::Vector,
+            crate::LaneCount<LANES>: crate::SupportedLaneCount,
         {
             fn cmp(&self, other: &Self) -> core::cmp::Ordering {
                 self.0.cmp(&other.0)
             }
         }
 
-        impl<T: Mask, const LANES: usize> $name<T, LANES>
+        impl<const LANES: usize> $name<LANES>
         where
-            crate::$type<LANES>: crate::Vector,
+            crate::LaneCount<LANES>: crate::SupportedLaneCount,
         {
             pub fn splat(value: bool) -> Self {
                 Self(
@@ -78,7 +75,6 @@ macro_rules! define_mask {
                             0
                         }
                     ),
-                    PhantomData,
                 )
             }
 
@@ -103,16 +99,19 @@ macro_rules! define_mask {
 
             #[inline]
             pub unsafe fn from_int_unchecked(value: crate::$type<LANES>) -> Self {
-                Self(value, PhantomData)
+                Self(value)
             }
 
             #[inline]
-            pub fn to_bitmask<U: crate::Mask>(self) -> U::BitMask {
+            pub fn to_bitmask(self) -> <crate::LaneCount::<LANES> as crate::SupportedLaneCount>::BitMask {
                 unsafe {
-                    // TODO remove the transmute when rustc is more flexible
-                    assert_eq!(core::mem::size_of::<U::IntBitMask>(), core::mem::size_of::<U::BitMask>());
-                    let mask: U::IntBitMask = crate::intrinsics::simd_bitmask(self.0);
-                    let mut bitmask: U::BitMask = core::mem::transmute_copy(&mask);
+                    // TODO remove the transmute when rustc can use arrays of u8 as bitmasks
+                    assert_eq!(
+                        core::mem::size_of::<<crate::LaneCount::<LANES> as crate::SupportedLaneCount>::BitMask>(),
+                        core::mem::size_of::<<crate::LaneCount::<LANES> as crate::SupportedLaneCount>::IntBitMask>(),
+                    );
+                    let bitmask: <crate::LaneCount::<LANES> as crate::SupportedLaneCount>::IntBitMask = crate::intrinsics::simd_bitmask(self.0);
+                    let mut bitmask: <crate::LaneCount::<LANES> as crate::SupportedLaneCount>::BitMask = core::mem::transmute_copy(&bitmask);
 
                     // There is a bug where LLVM appears to implement this operation with the wrong
                     // bit order.
@@ -128,7 +127,7 @@ macro_rules! define_mask {
             }
 
             #[inline]
-            pub fn from_bitmask<U: crate::Mask>(mut bitmask: U::BitMask) -> Self {
+            pub fn from_bitmask(mut bitmask: <crate::LaneCount::<LANES> as crate::SupportedLaneCount>::BitMask) -> Self {
                 unsafe {
                     // There is a bug where LLVM appears to implement this operation with the wrong
                     // bit order.
@@ -139,9 +138,12 @@ macro_rules! define_mask {
                         }
                     }
 
-                    // TODO remove the transmute when rustc is more flexible
-                    assert_eq!(core::mem::size_of::<U::IntBitMask>(), core::mem::size_of::<U::BitMask>());
-                    let bitmask: U::IntBitMask = core::mem::transmute_copy(&bitmask);
+                    // TODO remove the transmute when rustc can use arrays of u8 as bitmasks
+                    assert_eq!(
+                        core::mem::size_of::<<crate::LaneCount::<LANES> as crate::SupportedLaneCount>::BitMask>(),
+                        core::mem::size_of::<<crate::LaneCount::<LANES> as crate::SupportedLaneCount>::IntBitMask>(),
+                    );
+                    let bitmask: <crate::LaneCount::<LANES> as crate::SupportedLaneCount>::IntBitMask = core::mem::transmute_copy(&bitmask);
 
                     Self::from_int_unchecked(crate::intrinsics::simd_select_bitmask(
                         bitmask,
@@ -152,56 +154,56 @@ macro_rules! define_mask {
             }
         }
 
-        impl<T: Mask, const LANES: usize> core::convert::From<$name<T, LANES>> for crate::$type<LANES>
+        impl<const LANES: usize> core::convert::From<$name<LANES>> for crate::$type<LANES>
         where
-            crate::$type<LANES>: crate::Vector,
+            crate::LaneCount<LANES>: crate::SupportedLaneCount,
         {
-            fn from(value: $name<T, LANES>) -> Self {
+            fn from(value: $name<LANES>) -> Self {
                 value.0
             }
         }
 
-        impl<T: Mask, const LANES: usize> core::ops::BitAnd for $name<T, LANES>
+        impl<const LANES: usize> core::ops::BitAnd for $name<LANES>
         where
-            crate::$type<LANES>: crate::Vector,
+            crate::LaneCount<LANES>: crate::SupportedLaneCount,
         {
             type Output = Self;
             #[inline]
             fn bitand(self, rhs: Self) -> Self {
-                Self(self.0 & rhs.0, PhantomData)
+                Self(self.0 & rhs.0)
             }
         }
 
-        impl<T: Mask, const LANES: usize> core::ops::BitOr for $name<T, LANES>
+        impl<const LANES: usize> core::ops::BitOr for $name<LANES>
         where
-            crate::$type<LANES>: crate::Vector,
+            crate::LaneCount<LANES>: crate::SupportedLaneCount,
         {
             type Output = Self;
             #[inline]
             fn bitor(self, rhs: Self) -> Self {
-                Self(self.0 | rhs.0, PhantomData)
+                Self(self.0 | rhs.0)
             }
         }
 
-        impl<T: Mask, const LANES: usize> core::ops::BitXor for $name<T, LANES>
+        impl<const LANES: usize> core::ops::BitXor for $name<LANES>
         where
-            crate::$type<LANES>: crate::Vector,
+            crate::LaneCount<LANES>: crate::SupportedLaneCount,
         {
             type Output = Self;
             #[inline]
             fn bitxor(self, rhs: Self) -> Self::Output {
-                Self(self.0 ^ rhs.0, PhantomData)
+                Self(self.0 ^ rhs.0)
             }
         }
 
-        impl<T: Mask, const LANES: usize> core::ops::Not for $name<T, LANES>
+        impl<const LANES: usize> core::ops::Not for $name<LANES>
         where
-            crate::$type<LANES>: crate::Vector,
+            crate::LaneCount<LANES>: crate::SupportedLaneCount,
         {
             type Output = Self;
             #[inline]
             fn not(self) -> Self::Output {
-                Self(!self.0, PhantomData)
+                Self(!self.0)
             }
         }
     }
@@ -240,14 +242,11 @@ define_mask! {
 macro_rules! impl_from {
     { $from:ident ($from_inner:ident) => $($to:ident ($to_inner:ident)),* } => {
         $(
-        impl<const LANES: usize, T, U> From<$from<T, LANES>> for $to<U, LANES>
+        impl<const LANES: usize> From<$from<LANES>> for $to<LANES>
         where
-            crate::$from_inner<LANES>: crate::Vector,
-            crate::$to_inner<LANES>: crate::Vector,
-            T: crate::Mask,
-            U: crate::Mask,
+            crate::LaneCount<LANES>: crate::SupportedLaneCount,
         {
-            fn from(value: $from<T, LANES>) -> Self {
+            fn from(value: $from<LANES>) -> Self {
                 let mut new = Self::splat(false);
                 for i in 0..LANES {
                     unsafe { new.set_unchecked(i, value.test_unchecked(i)) }
diff --git a/crates/core_simd/src/math.rs b/crates/core_simd/src/math.rs
index cc0330871136..28720eb13e3c 100644
--- a/crates/core_simd/src/math.rs
+++ b/crates/core_simd/src/math.rs
@@ -1,6 +1,6 @@
 macro_rules! impl_uint_arith {
     ($(($name:ident, $n:ident)),+) => {
-        $( impl<const LANES: usize> $name<LANES> where Self: crate::Vector {
+        $( impl<const LANES: usize> $name<LANES> where crate::LaneCount<LANES>: crate::SupportedLaneCount {
 
             /// Lanewise saturating add.
             ///
@@ -44,7 +44,7 @@ macro_rules! impl_uint_arith {
 
 macro_rules! impl_int_arith {
     ($(($name:ident, $n:ident)),+) => {
-        $( impl<const LANES: usize> $name<LANES> where Self: crate::Vector {
+        $( impl<const LANES: usize> $name<LANES> where crate::LaneCount<LANES>: crate::SupportedLaneCount {
 
             /// Lanewise saturating add.
             ///
diff --git a/crates/core_simd/src/ops.rs b/crates/core_simd/src/ops.rs
index 9491bdd1d5a6..c75090aab9c5 100644
--- a/crates/core_simd/src/ops.rs
+++ b/crates/core_simd/src/ops.rs
@@ -1,4 +1,4 @@
-use crate::Vector;
+use crate::{LaneCount, SupportedLaneCount};
 
 /// Checks if the right-hand side argument of a left- or right-shift would cause overflow.
 fn invalid_shift_rhs<T>(rhs: T) -> bool
@@ -16,7 +16,7 @@ macro_rules! impl_ref_ops {
     {
         impl<const $lanes:ident: usize> core::ops::$trait:ident<$rhs:ty> for $type:ty
         where
-            $($bound:path: Vector,)*
+            LaneCount<$lanes2:ident>: SupportedLaneCount,
         {
             type Output = $output:ty;
 
@@ -26,7 +26,7 @@ macro_rules! impl_ref_ops {
     } => {
         impl<const $lanes: usize> core::ops::$trait<$rhs> for $type
         where
-            $($bound: Vector,)*
+            LaneCount<$lanes2>: SupportedLaneCount,
         {
             type Output = $output;
 
@@ -36,7 +36,7 @@ macro_rules! impl_ref_ops {
 
         impl<const $lanes: usize> core::ops::$trait<&'_ $rhs> for $type
         where
-            $($bound: Vector,)*
+            LaneCount<$lanes2>: SupportedLaneCount,
         {
             type Output = <$type as core::ops::$trait<$rhs>>::Output;
 
@@ -48,7 +48,7 @@ macro_rules! impl_ref_ops {
 
         impl<const $lanes: usize> core::ops::$trait<$rhs> for &'_ $type
         where
-            $($bound: Vector,)*
+            LaneCount<$lanes2>: SupportedLaneCount,
         {
             type Output = <$type as core::ops::$trait<$rhs>>::Output;
 
@@ -60,7 +60,7 @@ macro_rules! impl_ref_ops {
 
         impl<const $lanes: usize> core::ops::$trait<&'_ $rhs> for &'_ $type
         where
-            $($bound: Vector,)*
+            LaneCount<$lanes2>: SupportedLaneCount,
         {
             type Output = <$type as core::ops::$trait<$rhs>>::Output;
 
@@ -75,7 +75,7 @@ macro_rules! impl_ref_ops {
     {
         impl<const $lanes:ident: usize> core::ops::$trait:ident<$rhs:ty> for $type:ty
         where
-            $($bound:path: Vector,)*
+            LaneCount<$lanes2:ident>: SupportedLaneCount,
         {
             $(#[$attrs:meta])*
             fn $fn:ident(&mut $self_tok:ident, $rhs_arg:ident: $rhs_arg_ty:ty) $body:tt
@@ -83,7 +83,7 @@ macro_rules! impl_ref_ops {
     } => {
         impl<const $lanes: usize> core::ops::$trait<$rhs> for $type
         where
-            $($bound: Vector,)*
+            LaneCount<$lanes2>: SupportedLaneCount,
         {
             $(#[$attrs])*
             fn $fn(&mut $self_tok, $rhs_arg: $rhs_arg_ty) $body
@@ -91,7 +91,7 @@ macro_rules! impl_ref_ops {
 
         impl<const $lanes: usize> core::ops::$trait<&'_ $rhs> for $type
         where
-            $($bound: Vector,)*
+            LaneCount<$lanes2>: SupportedLaneCount,
         {
             $(#[$attrs])*
             fn $fn(&mut $self_tok, $rhs_arg: &$rhs_arg_ty) {
@@ -104,7 +104,7 @@ macro_rules! impl_ref_ops {
     {
         impl<const $lanes:ident: usize> core::ops::$trait:ident for $type:ty
         where
-            $($bound:path: Vector,)*
+            LaneCount<$lanes2:ident>: SupportedLaneCount,
         {
             type Output = $output:ty;
             fn $fn:ident($self_tok:ident) -> Self::Output $body:tt
@@ -112,7 +112,7 @@ macro_rules! impl_ref_ops {
     } => {
         impl<const $lanes: usize> core::ops::$trait for $type
         where
-            $($bound: Vector,)*
+            LaneCount<$lanes2>: SupportedLaneCount,
         {
             type Output = $output;
             fn $fn($self_tok) -> Self::Output $body
@@ -120,7 +120,7 @@ macro_rules! impl_ref_ops {
 
         impl<const $lanes: usize> core::ops::$trait for &'_ $type
         where
-            $($bound: Vector,)*
+            LaneCount<$lanes2>: SupportedLaneCount,
         {
             type Output = <$type as core::ops::$trait>::Output;
             fn $fn($self_tok) -> Self::Output {
@@ -167,7 +167,7 @@ macro_rules! impl_op {
         impl_ref_ops! {
             impl<const LANES: usize> core::ops::Not for crate::$type<LANES>
             where
-                crate::$type<LANES>: Vector,
+                LaneCount<LANES>: SupportedLaneCount,
             {
                 type Output = Self;
                 fn not(self) -> Self::Output {
@@ -181,7 +181,7 @@ macro_rules! impl_op {
         impl_ref_ops! {
             impl<const LANES: usize> core::ops::Neg for crate::$type<LANES>
             where
-                crate::$type<LANES>: Vector,
+                LaneCount<LANES>: SupportedLaneCount,
             {
                 type Output = Self;
                 fn neg(self) -> Self::Output {
@@ -194,7 +194,7 @@ macro_rules! impl_op {
     { impl Index for $type:ident, $scalar:ty } => {
         impl<I, const LANES: usize> core::ops::Index<I> for crate::$type<LANES>
         where
-            Self: Vector,
+            LaneCount<LANES>: SupportedLaneCount,
             I: core::slice::SliceIndex<[$scalar]>,
         {
             type Output = I::Output;
@@ -206,7 +206,7 @@ macro_rules! impl_op {
 
         impl<I, const LANES: usize> core::ops::IndexMut<I> for crate::$type<LANES>
         where
-            Self: Vector,
+            LaneCount<LANES>: SupportedLaneCount,
             I: core::slice::SliceIndex<[$scalar]>,
         {
             fn index_mut(&mut self, index: I) -> &mut Self::Output {
@@ -221,7 +221,7 @@ macro_rules! impl_op {
         impl_ref_ops! {
             impl<const LANES: usize> core::ops::$trait<Self> for crate::$type<LANES>
             where
-                crate::$type<LANES>: Vector,
+                LaneCount<LANES>: SupportedLaneCount,
             {
                 type Output = Self;
 
@@ -237,7 +237,7 @@ macro_rules! impl_op {
         impl_ref_ops! {
             impl<const LANES: usize> core::ops::$trait<$scalar> for crate::$type<LANES>
             where
-                crate::$type<LANES>: Vector,
+                LaneCount<LANES>: SupportedLaneCount,
             {
                 type Output = Self;
 
@@ -251,7 +251,7 @@ macro_rules! impl_op {
         impl_ref_ops! {
             impl<const LANES: usize> core::ops::$trait<crate::$type<LANES>> for $scalar
             where
-                crate::$type<LANES>: Vector,
+                LaneCount<LANES>: SupportedLaneCount,
             {
                 type Output = crate::$type<LANES>;
 
@@ -265,7 +265,7 @@ macro_rules! impl_op {
         impl_ref_ops! {
             impl<const LANES: usize> core::ops::$assign_trait<Self> for crate::$type<LANES>
             where
-                crate::$type<LANES>: Vector,
+                LaneCount<LANES>: SupportedLaneCount,
             {
                 #[inline]
                 fn $assign_trait_fn(&mut self, rhs: Self) {
@@ -279,7 +279,7 @@ macro_rules! impl_op {
         impl_ref_ops! {
             impl<const LANES: usize> core::ops::$assign_trait<$scalar> for crate::$type<LANES>
             where
-                crate::$type<LANES>: Vector,
+                LaneCount<LANES>: SupportedLaneCount,
             {
                 #[inline]
                 fn $assign_trait_fn(&mut self, rhs: $scalar) {
@@ -325,7 +325,7 @@ macro_rules! impl_unsigned_int_ops {
                 impl_ref_ops! {
                     impl<const LANES: usize> core::ops::Div<Self> for crate::$vector<LANES>
                     where
-                        crate::$vector<LANES>: Vector,
+                        LaneCount<LANES>: SupportedLaneCount,
                     {
                         type Output = Self;
 
@@ -353,7 +353,7 @@ macro_rules! impl_unsigned_int_ops {
                 impl_ref_ops! {
                     impl<const LANES: usize> core::ops::Div<$scalar> for crate::$vector<LANES>
                     where
-                        crate::$vector<LANES>: Vector,
+                        LaneCount<LANES>: SupportedLaneCount,
                     {
                         type Output = Self;
 
@@ -376,7 +376,7 @@ macro_rules! impl_unsigned_int_ops {
                 impl_ref_ops! {
                     impl<const LANES: usize> core::ops::Div<crate::$vector<LANES>> for $scalar
                     where
-                        crate::$vector<LANES>: Vector,
+                        LaneCount<LANES>: SupportedLaneCount,
                     {
                         type Output = crate::$vector<LANES>;
 
@@ -390,7 +390,7 @@ macro_rules! impl_unsigned_int_ops {
                 impl_ref_ops! {
                     impl<const LANES: usize> core::ops::DivAssign<Self> for crate::$vector<LANES>
                     where
-                        crate::$vector<LANES>: Vector,
+                        LaneCount<LANES>: SupportedLaneCount,
                     {
                         #[inline]
                         fn div_assign(&mut self, rhs: Self) {
@@ -402,7 +402,7 @@ macro_rules! impl_unsigned_int_ops {
                 impl_ref_ops! {
                     impl<const LANES: usize> core::ops::DivAssign<$scalar> for crate::$vector<LANES>
                     where
-                        crate::$vector<LANES>: Vector,
+                        LaneCount<LANES>: SupportedLaneCount,
                     {
                         #[inline]
                         fn div_assign(&mut self, rhs: $scalar) {
@@ -415,7 +415,7 @@ macro_rules! impl_unsigned_int_ops {
                 impl_ref_ops! {
                     impl<const LANES: usize> core::ops::Rem<Self> for crate::$vector<LANES>
                     where
-                        crate::$vector<LANES>: Vector,
+                        LaneCount<LANES>: SupportedLaneCount,
                     {
                         type Output = Self;
 
@@ -443,7 +443,7 @@ macro_rules! impl_unsigned_int_ops {
                 impl_ref_ops! {
                     impl<const LANES: usize> core::ops::Rem<$scalar> for crate::$vector<LANES>
                     where
-                        crate::$vector<LANES>: Vector,
+                        LaneCount<LANES>: SupportedLaneCount,
                     {
                         type Output = Self;
 
@@ -466,7 +466,7 @@ macro_rules! impl_unsigned_int_ops {
                 impl_ref_ops! {
                     impl<const LANES: usize> core::ops::Rem<crate::$vector<LANES>> for $scalar
                     where
-                        crate::$vector<LANES>: Vector,
+                        LaneCount<LANES>: SupportedLaneCount,
                     {
                         type Output = crate::$vector<LANES>;
 
@@ -480,7 +480,7 @@ macro_rules! impl_unsigned_int_ops {
                 impl_ref_ops! {
                     impl<const LANES: usize> core::ops::RemAssign<Self> for crate::$vector<LANES>
                     where
-                        crate::$vector<LANES>: Vector,
+                        LaneCount<LANES>: SupportedLaneCount,
                     {
                         #[inline]
                         fn rem_assign(&mut self, rhs: Self) {
@@ -492,7 +492,7 @@ macro_rules! impl_unsigned_int_ops {
                 impl_ref_ops! {
                     impl<const LANES: usize> core::ops::RemAssign<$scalar> for crate::$vector<LANES>
                     where
-                        crate::$vector<LANES>: Vector,
+                        LaneCount<LANES>: SupportedLaneCount,
                     {
                         #[inline]
                         fn rem_assign(&mut self, rhs: $scalar) {
@@ -505,7 +505,7 @@ macro_rules! impl_unsigned_int_ops {
                 impl_ref_ops! {
                     impl<const LANES: usize> core::ops::Shl<Self> for crate::$vector<LANES>
                     where
-                        crate::$vector<LANES>: Vector,
+                        LaneCount<LANES>: SupportedLaneCount,
                     {
                         type Output = Self;
 
@@ -527,7 +527,7 @@ macro_rules! impl_unsigned_int_ops {
                 impl_ref_ops! {
                     impl<const LANES: usize> core::ops::Shl<$scalar> for crate::$vector<LANES>
                     where
-                        crate::$vector<LANES>: Vector,
+                        LaneCount<LANES>: SupportedLaneCount,
                     {
                         type Output = Self;
 
@@ -546,7 +546,7 @@ macro_rules! impl_unsigned_int_ops {
                 impl_ref_ops! {
                     impl<const LANES: usize> core::ops::ShlAssign<Self> for crate::$vector<LANES>
                     where
-                        crate::$vector<LANES>: Vector,
+                        LaneCount<LANES>: SupportedLaneCount,
                     {
                         #[inline]
                         fn shl_assign(&mut self, rhs: Self) {
@@ -558,7 +558,7 @@ macro_rules! impl_unsigned_int_ops {
                 impl_ref_ops! {
                     impl<const LANES: usize> core::ops::ShlAssign<$scalar> for crate::$vector<LANES>
                     where
-                        crate::$vector<LANES>: Vector,
+                        LaneCount<LANES>: SupportedLaneCount,
                     {
                         #[inline]
                         fn shl_assign(&mut self, rhs: $scalar) {
@@ -570,7 +570,7 @@ macro_rules! impl_unsigned_int_ops {
                 impl_ref_ops! {
                     impl<const LANES: usize> core::ops::Shr<Self> for crate::$vector<LANES>
                     where
-                        crate::$vector<LANES>: Vector,
+                        LaneCount<LANES>: SupportedLaneCount,
                     {
                         type Output = Self;
 
@@ -592,7 +592,7 @@ macro_rules! impl_unsigned_int_ops {
                 impl_ref_ops! {
                     impl<const LANES: usize> core::ops::Shr<$scalar> for crate::$vector<LANES>
                     where
-                        crate::$vector<LANES>: Vector,
+                        LaneCount<LANES>: SupportedLaneCount,
                     {
                         type Output = Self;
 
@@ -611,7 +611,7 @@ macro_rules! impl_unsigned_int_ops {
                 impl_ref_ops! {
                     impl<const LANES: usize> core::ops::ShrAssign<Self> for crate::$vector<LANES>
                     where
-                        crate::$vector<LANES>: Vector,
+                        LaneCount<LANES>: SupportedLaneCount,
                     {
                         #[inline]
                         fn shr_assign(&mut self, rhs: Self) {
@@ -623,7 +623,7 @@ macro_rules! impl_unsigned_int_ops {
                 impl_ref_ops! {
                     impl<const LANES: usize> core::ops::ShrAssign<$scalar> for crate::$vector<LANES>
                     where
-                        crate::$vector<LANES>: Vector,
+                        LaneCount<LANES>: SupportedLaneCount,
                     {
                         #[inline]
                         fn shr_assign(&mut self, rhs: $scalar) {
diff --git a/crates/core_simd/src/reduction.rs b/crates/core_simd/src/reduction.rs
index 41cf6fab612a..df227d09e342 100644
--- a/crates/core_simd/src/reduction.rs
+++ b/crates/core_simd/src/reduction.rs
@@ -2,7 +2,7 @@ macro_rules! impl_integer_reductions {
     { $name:ident, $scalar:ty } => {
         impl<const LANES: usize> crate::$name<LANES>
         where
-            Self: crate::Vector
+            crate::LaneCount<LANES>: crate::SupportedLaneCount,
         {
             /// Horizontal wrapping add.  Returns the sum of the lanes of the vector, with wrapping addition.
             #[inline]
@@ -56,7 +56,7 @@ macro_rules! impl_float_reductions {
     { $name:ident, $scalar:ty } => {
         impl<const LANES: usize> crate::$name<LANES>
         where
-            Self: crate::Vector
+            crate::LaneCount<LANES>: crate::SupportedLaneCount,
         {
 
             /// Horizontal add.  Returns the sum of the lanes of the vector.
@@ -104,9 +104,9 @@ macro_rules! impl_float_reductions {
 
 macro_rules! impl_full_mask_reductions {
     { $name:ident, $bits_ty:ident } => {
-        impl<T: crate::Mask, const LANES: usize> $name<T, LANES>
+        impl<const LANES: usize> $name<LANES>
         where
-            crate::$bits_ty<LANES>: crate::Vector
+            crate::LaneCount<LANES>: crate::SupportedLaneCount,
         {
             #[inline]
             pub fn any(self) -> bool {
@@ -125,8 +125,7 @@ macro_rules! impl_opaque_mask_reductions {
     { $name:ident, $bits_ty:ident } => {
         impl<const LANES: usize> $name<LANES>
         where
-            crate::$bits_ty<LANES>: crate::Vector,
-            $name<LANES>: crate::Mask,
+            crate::LaneCount<LANES>: crate::SupportedLaneCount,
         {
             /// Returns true if any lane is set, or false otherwise.
             #[inline]
diff --git a/crates/core_simd/src/round.rs b/crates/core_simd/src/round.rs
index 5cd7a898eaf4..74cae0cf9898 100644
--- a/crates/core_simd/src/round.rs
+++ b/crates/core_simd/src/round.rs
@@ -5,7 +5,7 @@ macro_rules! implement {
         #[cfg(feature = "std")]
         impl<const LANES: usize> crate::$type<LANES>
         where
-            Self: crate::Vector,
+            crate::LaneCount<LANES>: crate::SupportedLaneCount,
         {
             /// Returns the smallest integer greater than or equal to each lane.
             #[must_use = "method returns a new vector and does not mutate the original value"]
@@ -41,13 +41,7 @@ macro_rules! implement {
             pub fn fract(self) -> Self {
                 self - self.trunc()
             }
-        }
 
-        impl<const LANES: usize> crate::$type<LANES>
-        where
-            Self: crate::Vector,
-            crate::$int_type<LANES>: crate::Vector,
-        {
             /// Rounds toward zero and converts to the same-width integer type, assuming that
             /// the value is finite and fits in that type.
             ///
diff --git a/crates/core_simd/src/select.rs b/crates/core_simd/src/select.rs
index 1558eb169351..d70e8a66b95f 100644
--- a/crates/core_simd/src/select.rs
+++ b/crates/core_simd/src/select.rs
@@ -14,12 +14,10 @@ macro_rules! impl_select {
         $mask:ident ($bits_ty:ident): $($type:ident),*
     } => {
         $(
-        impl<const LANES: usize> Sealed for crate::$type<LANES> where Self: crate::Vector {}
+        impl<const LANES: usize> Sealed for crate::$type<LANES> where crate::LaneCount<LANES>: crate::SupportedLaneCount {}
         impl<const LANES: usize> Select<crate::$mask<LANES>> for crate::$type<LANES>
         where
-            crate::$mask<LANES>: crate::Mask,
-            crate::$bits_ty<LANES>: crate::Vector,
-            Self: crate::Vector,
+            crate::LaneCount<LANES>: crate::SupportedLaneCount,
         {
             #[doc(hidden)]
             #[inline]
@@ -31,13 +29,12 @@ macro_rules! impl_select {
 
         impl<const LANES: usize> Sealed for crate::$mask<LANES>
         where
-            Self: crate::Mask,
-            crate::$bits_ty<LANES>: crate::Vector,
+            crate::LaneCount<LANES>: crate::SupportedLaneCount,
         {}
+
         impl<const LANES: usize> Select<Self> for crate::$mask<LANES>
         where
-            Self: crate::Mask,
-            crate::$bits_ty<LANES>: crate::Vector,
+            crate::LaneCount<LANES>: crate::SupportedLaneCount,
         {
             #[doc(hidden)]
             #[inline]
@@ -48,8 +45,7 @@ macro_rules! impl_select {
 
         impl<const LANES: usize> crate::$mask<LANES>
         where
-            Self: crate::Mask,
-            crate::$bits_ty<LANES>: crate::Vector,
+            crate::LaneCount<LANES>: crate::SupportedLaneCount,
         {
             /// Choose lanes from two vectors.
             ///
diff --git a/crates/core_simd/src/to_bytes.rs b/crates/core_simd/src/to_bytes.rs
index c4f112c9ee7e..0823391049fd 100644
--- a/crates/core_simd/src/to_bytes.rs
+++ b/crates/core_simd/src/to_bytes.rs
@@ -18,11 +18,14 @@ pub trait ToBytes: Sealed {
 macro_rules! impl_to_bytes {
     { $name:ident, $($int_width:literal -> $byte_width:literal),* } => {
         $(
-        impl Sealed for crate::$name<$int_width> where Self: crate::Vector {}
+        impl Sealed for crate::$name<$int_width>
+        where
+            crate::LaneCount<$int_width>: crate::SupportedLaneCount,
+        {}
+
         impl ToBytes for crate::$name<$int_width>
         where
-            Self: crate::Vector,
-            crate::SimdU8<$byte_width>: crate::Vector,
+            crate::LaneCount<$int_width>: crate::SupportedLaneCount,
         {
             type Bytes = crate::SimdU8<$byte_width>;
             fn to_bytes_impl(self) -> Self::Bytes {
@@ -36,7 +39,8 @@ macro_rules! impl_to_bytes {
 
         impl<const LANES: usize> crate::$name<LANES>
         where
-            Self: ToBytes + crate::Vector,
+            crate::LaneCount<LANES>: crate::SupportedLaneCount,
+            Self: ToBytes,
         {
             /// Return the memory representation of this integer as a byte array in native byte
             /// order.
diff --git a/crates/core_simd/src/vector.rs b/crates/core_simd/src/vector.rs
index 2bc7c1cd2b74..1f6df533767e 100644
--- a/crates/core_simd/src/vector.rs
+++ b/crates/core_simd/src/vector.rs
@@ -26,10 +26,6 @@ pub trait Vector: sealed::Sealed {
     /// The number of lanes for this vector.
     const LANES: usize;
 
-    // Implementation detail until the compiler can support bitmasks of any integer width
-    #[doc(hidden)]
-    type BitMask: Into<u64>;
-
     /// Generates a SIMD vector with the same value in every lane.
     #[must_use]
     fn splat(val: Self::Scalar) -> Self;
diff --git a/crates/core_simd/src/vector/float.rs b/crates/core_simd/src/vector/float.rs
index 4b069a4fab83..bdeccd037a80 100644
--- a/crates/core_simd/src/vector/float.rs
+++ b/crates/core_simd/src/vector/float.rs
@@ -1,5 +1,7 @@
 #![allow(non_camel_case_types)]
 
+use crate::{LaneCount, SupportedLaneCount};
+
 /// Implements inherent methods for a float vector `$name` containing multiple
 /// `$lanes` of float `$type`, which uses `$bits_ty` as its binary
 /// representation. Called from `define_float_vector!`.
@@ -10,8 +12,7 @@ macro_rules! impl_float_vector {
 
         impl<const LANES: usize> $name<LANES>
         where
-            Self: crate::Vector,
-            crate::$bits_ty<LANES>: crate::Vector,
+            LaneCount<LANES>: SupportedLaneCount,
         {
             /// Raw transmutation to an unsigned integer vector type with the
             /// same size and number of lanes.
@@ -74,15 +75,7 @@ macro_rules! impl_float_vector {
             pub fn to_radians(self) -> Self {
                 self * Self::splat($type::to_radians(1.))
             }
-        }
 
-        impl<const LANES: usize> $name<LANES>
-        where
-            Self: crate::Vector,
-            crate::$bits_ty<LANES>: crate::Vector,
-            crate::$mask_impl_ty<LANES>: crate::Vector,
-            crate::$mask_ty<LANES>: crate::Mask,
-        {
             /// Returns true for each lane if it has a positive sign, including
             /// `+0.0`, `NaN`s with positive sign bit and positive infinity.
             #[inline]
@@ -197,7 +190,7 @@ macro_rules! impl_float_vector {
 #[repr(simd)]
 pub struct SimdF32<const LANES: usize>([f32; LANES])
 where
-    Self: crate::Vector;
+    LaneCount<LANES>: SupportedLaneCount;
 
 impl_float_vector! { SimdF32, f32, SimdU32, Mask32, SimdI32 }
 
@@ -205,7 +198,7 @@ impl_float_vector! { SimdF32, f32, SimdU32, Mask32, SimdI32 }
 #[repr(simd)]
 pub struct SimdF64<const LANES: usize>([f64; LANES])
 where
-    Self: crate::Vector;
+    LaneCount<LANES>: SupportedLaneCount;
 
 impl_float_vector! { SimdF64, f64, SimdU64, Mask64, SimdI64 }
 
diff --git a/crates/core_simd/src/vector/int.rs b/crates/core_simd/src/vector/int.rs
index 0ca1ea14e2d5..73c737762fb6 100644
--- a/crates/core_simd/src/vector/int.rs
+++ b/crates/core_simd/src/vector/int.rs
@@ -1,14 +1,16 @@
 #![allow(non_camel_case_types)]
 
+use crate::{LaneCount, SupportedLaneCount};
+
 /// Implements additional integer traits (Eq, Ord, Hash) on the specified vector `$name`, holding multiple `$lanes` of `$type`.
 macro_rules! impl_integer_vector {
     { $name:ident, $type:ty, $mask_ty:ident, $mask_impl_ty:ident } => {
         impl_vector! { $name, $type }
         impl_integer_reductions! { $name, $type }
 
-        impl<const LANES: usize> Eq for $name<LANES> where Self: crate::Vector {}
+        impl<const LANES: usize> Eq for $name<LANES> where LaneCount<LANES>: SupportedLaneCount {}
 
-        impl<const LANES: usize> Ord for $name<LANES> where Self: crate::Vector {
+        impl<const LANES: usize> Ord for $name<LANES> where LaneCount<LANES>: SupportedLaneCount {
             #[inline]
             fn cmp(&self, other: &Self) -> core::cmp::Ordering {
                 // TODO use SIMD cmp
@@ -16,7 +18,10 @@ macro_rules! impl_integer_vector {
             }
         }
 
-        impl<const LANES: usize> core::hash::Hash for $name<LANES> where Self: crate::Vector {
+        impl<const LANES: usize> core::hash::Hash for $name<LANES>
+        where
+            LaneCount<LANES>: SupportedLaneCount,
+        {
             #[inline]
             fn hash<H>(&self, state: &mut H)
             where
@@ -28,9 +33,7 @@ macro_rules! impl_integer_vector {
 
         impl<const LANES: usize> $name<LANES>
         where
-            Self: crate::Vector,
-            crate::$mask_impl_ty<LANES>: crate::Vector,
-            crate::$mask_ty<LANES>: crate::Mask,
+            LaneCount<LANES>: SupportedLaneCount,
         {
             /// Returns true for each positive lane and false if it is zero or negative.
             #[inline]
@@ -63,7 +66,7 @@ macro_rules! impl_integer_vector {
 #[repr(simd)]
 pub struct SimdIsize<const LANES: usize>([isize; LANES])
 where
-    Self: crate::Vector;
+    LaneCount<LANES>: SupportedLaneCount;
 
 impl_integer_vector! { SimdIsize, isize, MaskSize, SimdIsize }
 
@@ -71,7 +74,7 @@ impl_integer_vector! { SimdIsize, isize, MaskSize, SimdIsize }
 #[repr(simd)]
 pub struct SimdI16<const LANES: usize>([i16; LANES])
 where
-    Self: crate::Vector;
+    LaneCount<LANES>: SupportedLaneCount;
 
 impl_integer_vector! { SimdI16, i16, Mask16, SimdI16 }
 
@@ -79,7 +82,7 @@ impl_integer_vector! { SimdI16, i16, Mask16, SimdI16 }
 #[repr(simd)]
 pub struct SimdI32<const LANES: usize>([i32; LANES])
 where
-    Self: crate::Vector;
+    LaneCount<LANES>: SupportedLaneCount;
 
 impl_integer_vector! { SimdI32, i32, Mask32, SimdI32 }
 
@@ -87,7 +90,7 @@ impl_integer_vector! { SimdI32, i32, Mask32, SimdI32 }
 #[repr(simd)]
 pub struct SimdI64<const LANES: usize>([i64; LANES])
 where
-    Self: crate::Vector;
+    LaneCount<LANES>: SupportedLaneCount;
 
 impl_integer_vector! { SimdI64, i64, Mask64, SimdI64 }
 
@@ -95,7 +98,7 @@ impl_integer_vector! { SimdI64, i64, Mask64, SimdI64 }
 #[repr(simd)]
 pub struct SimdI8<const LANES: usize>([i8; LANES])
 where
-    Self: crate::Vector;
+    LaneCount<LANES>: SupportedLaneCount;
 
 impl_integer_vector! { SimdI8, i8, Mask8, SimdI8 }
 
diff --git a/crates/core_simd/src/vector/ptr.rs b/crates/core_simd/src/vector/ptr.rs
index bdd5c24eac85..9dd1bfd0f365 100644
--- a/crates/core_simd/src/vector/ptr.rs
+++ b/crates/core_simd/src/vector/ptr.rs
@@ -1,5 +1,5 @@
 //! Private implementation details of public gather/scatter APIs.
-use crate::SimdUsize;
+use crate::{LaneCount, SimdUsize, SupportedLaneCount};
 use core::mem;
 
 /// A vector of *const T.
@@ -9,7 +9,7 @@ pub(crate) struct SimdConstPtr<T, const LANES: usize>([*const T; LANES]);
 
 impl<T, const LANES: usize> SimdConstPtr<T, LANES>
 where
-    SimdUsize<LANES>: crate::Vector,
+    LaneCount<LANES>: SupportedLaneCount,
     T: Sized,
 {
     #[inline]
@@ -35,7 +35,7 @@ pub(crate) struct SimdMutPtr<T, const LANES: usize>([*mut T; LANES]);
 
 impl<T, const LANES: usize> SimdMutPtr<T, LANES>
 where
-    SimdUsize<LANES>: crate::Vector,
+    LaneCount<LANES>: SupportedLaneCount,
     T: Sized,
 {
     #[inline]
diff --git a/crates/core_simd/src/vector/uint.rs b/crates/core_simd/src/vector/uint.rs
index e7a6a8880121..b19f694872ae 100644
--- a/crates/core_simd/src/vector/uint.rs
+++ b/crates/core_simd/src/vector/uint.rs
@@ -1,14 +1,16 @@
 #![allow(non_camel_case_types)]
 
+use crate::{LaneCount, SupportedLaneCount};
+
 /// Implements additional integer traits (Eq, Ord, Hash) on the specified vector `$name`, holding multiple `$lanes` of `$type`.
 macro_rules! impl_unsigned_vector {
     { $name:ident, $type:ty } => {
         impl_vector! { $name, $type }
         impl_integer_reductions! { $name, $type }
 
-        impl<const LANES: usize> Eq for $name<LANES> where Self: crate::Vector {}
+        impl<const LANES: usize> Eq for $name<LANES> where LaneCount<LANES>: SupportedLaneCount {}
 
-        impl<const LANES: usize> Ord for $name<LANES> where Self: crate::Vector {
+        impl<const LANES: usize> Ord for $name<LANES> where LaneCount<LANES>: SupportedLaneCount {
             #[inline]
             fn cmp(&self, other: &Self) -> core::cmp::Ordering {
                 // TODO use SIMD cmp
@@ -16,7 +18,10 @@ macro_rules! impl_unsigned_vector {
             }
         }
 
-        impl<const LANES: usize> core::hash::Hash for $name<LANES> where Self: crate::Vector {
+        impl<const LANES: usize> core::hash::Hash for $name<LANES>
+        where
+            LaneCount<LANES>: SupportedLaneCount,
+        {
             #[inline]
             fn hash<H>(&self, state: &mut H)
             where
@@ -32,7 +37,7 @@ macro_rules! impl_unsigned_vector {
 #[repr(simd)]
 pub struct SimdUsize<const LANES: usize>([usize; LANES])
 where
-    Self: crate::Vector;
+    LaneCount<LANES>: SupportedLaneCount;
 
 impl_unsigned_vector! { SimdUsize, usize }
 
@@ -40,7 +45,7 @@ impl_unsigned_vector! { SimdUsize, usize }
 #[repr(simd)]
 pub struct SimdU16<const LANES: usize>([u16; LANES])
 where
-    Self: crate::Vector;
+    LaneCount<LANES>: SupportedLaneCount;
 
 impl_unsigned_vector! { SimdU16, u16 }
 
@@ -48,7 +53,7 @@ impl_unsigned_vector! { SimdU16, u16 }
 #[repr(simd)]
 pub struct SimdU32<const LANES: usize>([u32; LANES])
 where
-    Self: crate::Vector;
+    LaneCount<LANES>: SupportedLaneCount;
 
 impl_unsigned_vector! { SimdU32, u32 }
 
@@ -56,7 +61,7 @@ impl_unsigned_vector! { SimdU32, u32 }
 #[repr(simd)]
 pub struct SimdU64<const LANES: usize>([u64; LANES])
 where
-    Self: crate::Vector;
+    LaneCount<LANES>: SupportedLaneCount;
 
 impl_unsigned_vector! { SimdU64, u64 }
 
@@ -64,7 +69,7 @@ impl_unsigned_vector! { SimdU64, u64 }
 #[repr(simd)]
 pub struct SimdU8<const LANES: usize>([u8; LANES])
 where
-    Self: crate::Vector;
+    LaneCount<LANES>: SupportedLaneCount;
 
 impl_unsigned_vector! { SimdU8, u8 }
 
diff --git a/crates/core_simd/src/vector/vector_impl.rs b/crates/core_simd/src/vector/vector_impl.rs
index e166a93670b8..58ea244adfcb 100644
--- a/crates/core_simd/src/vector/vector_impl.rs
+++ b/crates/core_simd/src/vector/vector_impl.rs
@@ -1,37 +1,28 @@
-macro_rules! impl_vector_trait {
-    ($simd:ident {type Scalar = $scalar:ty;}) => {
-        impl_vector_trait! { $simd<1> { type Scalar = $scalar; type BitMask = u8; } }
-        impl_vector_trait! { $simd<2> { type Scalar = $scalar; type BitMask = u8; } }
-        impl_vector_trait! { $simd<4> { type Scalar = $scalar; type BitMask = u8; } }
-        impl_vector_trait! { $simd<8> { type Scalar = $scalar; type BitMask = u8; } }
-        impl_vector_trait! { $simd<16> { type Scalar = $scalar; type BitMask = u16; } }
-        impl_vector_trait! { $simd<32> { type Scalar = $scalar; type BitMask = u32; } }
-    };
-    ($simd:ident<$lanes:literal> {type Scalar = $scalar:ty; type BitMask = $bitmask:ident; }) => {
-        impl crate::vector::sealed::Sealed for $simd<$lanes> {}
+/// Implements common traits on the specified vector `$name`, holding multiple `$lanes` of `$type`.
+macro_rules! impl_vector {
+    { $name:ident, $type:ty } => {
+        impl<const LANES: usize> crate::vector::sealed::Sealed for $name<LANES>
+        where
+            crate::LaneCount<LANES>: crate::SupportedLaneCount,
+        {}
 
-        impl crate::vector::Vector for $simd<$lanes> {
-            type Scalar = $scalar;
-            const LANES: usize = $lanes;
-
-            type BitMask = $bitmask;
+        impl<const LANES: usize> crate::vector::Vector for $name<LANES>
+        where
+            crate::LaneCount<LANES>: crate::SupportedLaneCount,
+        {
+            type Scalar = $type;
+            const LANES: usize = LANES;
 
             #[inline]
             fn splat(val: Self::Scalar) -> Self {
                 Self::splat(val)
             }
         }
-    };
-}
 
-/// Implements common traits on the specified vector `$name`, holding multiple `$lanes` of `$type`.
-macro_rules! impl_vector {
-    { $name:ident, $type:ty } => {
-        impl_vector_trait! {
-            $name { type Scalar = $type; }
-        }
-
-        impl<const LANES: usize> $name<LANES> where Self: crate::Vector {
+        impl<const LANES: usize> $name<LANES>
+        where
+            crate::LaneCount<LANES>: crate::SupportedLaneCount,
+        {
             /// Construct a SIMD vector by setting all lanes to the given value.
             pub const fn splat(value: $type) -> Self {
                 Self([value; LANES])
@@ -56,15 +47,7 @@ macro_rules! impl_vector {
             pub const fn to_array(self) -> [$type; LANES] {
                 self.0
             }
-        }
 
-        impl<const LANES: usize> $name<LANES>
-        where
-            Self: crate::Vector,
-            crate::MaskSize<LANES>: crate::Mask,
-            crate::SimdIsize<LANES>: crate::Vector,
-            crate::SimdUsize<LANES>: crate::Vector,
-        {
             /// SIMD gather: construct a SIMD vector by reading from a slice, using potentially discontiguous indices.
             /// If an index is out of bounds, that lane instead selects the value from the "or" vector.
             /// ```
@@ -194,23 +177,23 @@ macro_rules! impl_vector {
             }
         }
 
-        impl<const LANES: usize> Copy for $name<LANES> where Self: crate::Vector {}
+        impl<const LANES: usize> Copy for $name<LANES> where crate::LaneCount<LANES>: crate::SupportedLaneCount {}
 
-        impl<const LANES: usize> Clone for $name<LANES> where Self: crate::Vector {
+        impl<const LANES: usize> Clone for $name<LANES> where crate::LaneCount<LANES>: crate::SupportedLaneCount {
             #[inline]
             fn clone(&self) -> Self {
                 *self
             }
         }
 
-        impl<const LANES: usize> Default for $name<LANES> where Self: crate::Vector {
+        impl<const LANES: usize> Default for $name<LANES> where crate::LaneCount<LANES>: crate::SupportedLaneCount {
             #[inline]
             fn default() -> Self {
                 Self::splat(<$type>::default())
             }
         }
 
-        impl<const LANES: usize> PartialEq for $name<LANES> where Self: crate::Vector {
+        impl<const LANES: usize> PartialEq for $name<LANES> where crate::LaneCount<LANES>: crate::SupportedLaneCount {
             #[inline]
             fn eq(&self, other: &Self) -> bool {
                 // TODO use SIMD equality
@@ -218,7 +201,7 @@ macro_rules! impl_vector {
             }
         }
 
-        impl<const LANES: usize> PartialOrd for $name<LANES> where Self: crate::Vector {
+        impl<const LANES: usize> PartialOrd for $name<LANES> where crate::LaneCount<LANES>: crate::SupportedLaneCount {
             #[inline]
             fn partial_cmp(&self, other: &Self) -> Option<core::cmp::Ordering> {
                 // TODO use SIMD equalitya
@@ -227,14 +210,14 @@ macro_rules! impl_vector {
         }
 
         // array references
-        impl<const LANES: usize> AsRef<[$type; LANES]> for $name<LANES> where Self: crate::Vector {
+        impl<const LANES: usize> AsRef<[$type; LANES]> for $name<LANES> where crate::LaneCount<LANES>: crate::SupportedLaneCount {
             #[inline]
             fn as_ref(&self) -> &[$type; LANES] {
                 &self.0
             }
         }
 
-        impl<const LANES: usize> AsMut<[$type; LANES]> for $name<LANES> where Self: crate::Vector {
+        impl<const LANES: usize> AsMut<[$type; LANES]> for $name<LANES> where crate::LaneCount<LANES>: crate::SupportedLaneCount {
             #[inline]
             fn as_mut(&mut self) -> &mut [$type; LANES] {
                 &mut self.0
@@ -242,14 +225,14 @@ macro_rules! impl_vector {
         }
 
         // slice references
-        impl<const LANES: usize> AsRef<[$type]> for $name<LANES> where Self: crate::Vector {
+        impl<const LANES: usize> AsRef<[$type]> for $name<LANES> where crate::LaneCount<LANES>: crate::SupportedLaneCount {
             #[inline]
             fn as_ref(&self) -> &[$type] {
                 &self.0
             }
         }
 
-        impl<const LANES: usize> AsMut<[$type]> for $name<LANES> where Self: crate::Vector {
+        impl<const LANES: usize> AsMut<[$type]> for $name<LANES> where crate::LaneCount<LANES>: crate::SupportedLaneCount {
             #[inline]
             fn as_mut(&mut self) -> &mut [$type] {
                 &mut self.0
@@ -257,13 +240,13 @@ macro_rules! impl_vector {
         }
 
         // vector/array conversion
-        impl<const LANES: usize> From<[$type; LANES]> for $name<LANES> where Self: crate::Vector {
+        impl<const LANES: usize> From<[$type; LANES]> for $name<LANES> where crate::LaneCount<LANES>: crate::SupportedLaneCount {
             fn from(array: [$type; LANES]) -> Self {
                 Self(array)
             }
         }
 
-        impl <const LANES: usize> From<$name<LANES>> for [$type; LANES] where $name<LANES>: crate::Vector {
+        impl <const LANES: usize> From<$name<LANES>> for [$type; LANES] where crate::LaneCount<LANES>: crate::SupportedLaneCount {
             fn from(vector: $name<LANES>) -> Self {
                 vector.to_array()
             }
diff --git a/crates/test_helpers/src/lib.rs b/crates/test_helpers/src/lib.rs
index 5691bf4538ce..5c6478876f30 100644
--- a/crates/test_helpers/src/lib.rs
+++ b/crates/test_helpers/src/lib.rs
@@ -335,23 +335,7 @@ macro_rules! test_lanes {
 
                 fn implementation<const $lanes: usize>()
                 where
-                    core_simd::SimdU8<$lanes>: core_simd::Vector,
-                    core_simd::SimdU16<$lanes>: core_simd::Vector,
-                    core_simd::SimdU32<$lanes>: core_simd::Vector,
-                    core_simd::SimdU64<$lanes>: core_simd::Vector,
-                    core_simd::SimdUsize<$lanes>: core_simd::Vector,
-                    core_simd::SimdI8<$lanes>: core_simd::Vector,
-                    core_simd::SimdI16<$lanes>: core_simd::Vector,
-                    core_simd::SimdI32<$lanes>: core_simd::Vector,
-                    core_simd::SimdI64<$lanes>: core_simd::Vector,
-                    core_simd::SimdIsize<$lanes>: core_simd::Vector,
-                    core_simd::SimdF32<$lanes>: core_simd::Vector,
-                    core_simd::SimdF64<$lanes>: core_simd::Vector,
-                    core_simd::Mask8<$lanes>: core_simd::Mask,
-                    core_simd::Mask16<$lanes>: core_simd::Mask,
-                    core_simd::Mask32<$lanes>: core_simd::Mask,
-                    core_simd::Mask64<$lanes>: core_simd::Mask,
-                    core_simd::MaskSize<$lanes>: core_simd::Mask,
+                    core_simd::LaneCount<$lanes>: core_simd::SupportedLaneCount,
                 $body
 
                 #[cfg(target_arch = "wasm32")]
@@ -409,23 +393,7 @@ macro_rules! test_lanes_panic {
 
                 fn implementation<const $lanes: usize>()
                 where
-                    core_simd::SimdU8<$lanes>: core_simd::Vector,
-                    core_simd::SimdU16<$lanes>: core_simd::Vector,
-                    core_simd::SimdU32<$lanes>: core_simd::Vector,
-                    core_simd::SimdU64<$lanes>: core_simd::Vector,
-                    core_simd::SimdUsize<$lanes>: core_simd::Vector,
-                    core_simd::SimdI8<$lanes>: core_simd::Vector,
-                    core_simd::SimdI16<$lanes>: core_simd::Vector,
-                    core_simd::SimdI32<$lanes>: core_simd::Vector,
-                    core_simd::SimdI64<$lanes>: core_simd::Vector,
-                    core_simd::SimdIsize<$lanes>: core_simd::Vector,
-                    core_simd::SimdF32<$lanes>: core_simd::Vector,
-                    core_simd::SimdF64<$lanes>: core_simd::Vector,
-                    core_simd::Mask8<$lanes>: core_simd::Mask,
-                    core_simd::Mask16<$lanes>: core_simd::Mask,
-                    core_simd::Mask32<$lanes>: core_simd::Mask,
-                    core_simd::Mask64<$lanes>: core_simd::Mask,
-                    core_simd::MaskSize<$lanes>: core_simd::Mask,
+                    core_simd::LaneCount<$lanes>: core_simd::SupportedLaneCount,
                 $body
 
                 #[test]

From 34384b7a68b3a72fc96e5293de7c7486d2cceb92 Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Wed, 28 Jul 2021 04:19:31 +0000
Subject: [PATCH 199/249] Add const_evaluatable_checked feature, change
 to_bitmask to use it, and fix existing std feature

---
 crates/core_simd/Cargo.toml          |   3 +-
 crates/core_simd/examples/nbody.rs   | 301 ++++++++++++++-------------
 crates/core_simd/src/intrinsics.rs   |   1 +
 crates/core_simd/src/lib.rs          |   3 +-
 crates/core_simd/src/round.rs        |   5 +
 crates/core_simd/src/to_bytes.rs     |  73 ++-----
 crates/core_simd/tests/ops_macros.rs |  19 +-
 crates/core_simd/tests/to_bytes.rs   |   4 +-
 8 files changed, 199 insertions(+), 210 deletions(-)

diff --git a/crates/core_simd/Cargo.toml b/crates/core_simd/Cargo.toml
index 6044eabcd140..764b199d5bf9 100644
--- a/crates/core_simd/Cargo.toml
+++ b/crates/core_simd/Cargo.toml
@@ -10,8 +10,9 @@ categories = ["hardware-support", "no-std"]
 license = "MIT OR Apache-2.0"
 
 [features]
-default = ["std"]
+default = ["std", "const_evaluatable_checked"]
 std = []
+const_evaluatable_checked = []
 
 [target.'cfg(target_arch = "wasm32")'.dev-dependencies.wasm-bindgen]
 version = "0.2"
diff --git a/crates/core_simd/examples/nbody.rs b/crates/core_simd/examples/nbody.rs
index 40e4e18b0264..779575985ed9 100644
--- a/crates/core_simd/examples/nbody.rs
+++ b/crates/core_simd/examples/nbody.rs
@@ -1,169 +1,173 @@
-#![feature(portable_simd)]
+#![cfg_attr(feature = "std", feature(portable_simd))]
 
 /// Benchmarks game nbody code
 /// Taken from the `packed_simd` crate
 /// Run this benchmark with `cargo test --example nbody`
-use core_simd::*;
+#[cfg(feature = "std")]
+mod nbody {
+    use core_simd::*;
 
-use std::f64::consts::PI;
-const SOLAR_MASS: f64 = 4.0 * PI * PI;
-const DAYS_PER_YEAR: f64 = 365.24;
+    use std::f64::consts::PI;
+    const SOLAR_MASS: f64 = 4.0 * PI * PI;
+    const DAYS_PER_YEAR: f64 = 365.24;
 
-#[derive(Debug, Clone, Copy)]
-pub struct Body {
-    pub x: f64x4,
-    pub v: f64x4,
-    pub mass: f64,
-}
-
-const N_BODIES: usize = 5;
-const BODIES: [Body; N_BODIES] = [
-    // sun:
-    Body {
-        x: f64x4::from_array([0., 0., 0., 0.]),
-        v: f64x4::from_array([0., 0., 0., 0.]),
-        mass: SOLAR_MASS,
-    },
-    // jupiter:
-    Body {
-        x: f64x4::from_array([
-            4.84143144246472090e+00,
-            -1.16032004402742839e+00,
-            -1.03622044471123109e-01,
-            0.,
-        ]),
-        v: f64x4::from_array([
-            1.66007664274403694e-03 * DAYS_PER_YEAR,
-            7.69901118419740425e-03 * DAYS_PER_YEAR,
-            -6.90460016972063023e-05 * DAYS_PER_YEAR,
-            0.,
-        ]),
-        mass: 9.54791938424326609e-04 * SOLAR_MASS,
-    },
-    // saturn:
-    Body {
-        x: f64x4::from_array([
-            8.34336671824457987e+00,
-            4.12479856412430479e+00,
-            -4.03523417114321381e-01,
-            0.,
-        ]),
-        v: f64x4::from_array([
-            -2.76742510726862411e-03 * DAYS_PER_YEAR,
-            4.99852801234917238e-03 * DAYS_PER_YEAR,
-            2.30417297573763929e-05 * DAYS_PER_YEAR,
-            0.,
-        ]),
-        mass: 2.85885980666130812e-04 * SOLAR_MASS,
-    },
-    // uranus:
-    Body {
-        x: f64x4::from_array([
-            1.28943695621391310e+01,
-            -1.51111514016986312e+01,
-            -2.23307578892655734e-01,
-            0.,
-        ]),
-        v: f64x4::from_array([
-            2.96460137564761618e-03 * DAYS_PER_YEAR,
-            2.37847173959480950e-03 * DAYS_PER_YEAR,
-            -2.96589568540237556e-05 * DAYS_PER_YEAR,
-            0.,
-        ]),
-        mass: 4.36624404335156298e-05 * SOLAR_MASS,
-    },
-    // neptune:
-    Body {
-        x: f64x4::from_array([
-            1.53796971148509165e+01,
-            -2.59193146099879641e+01,
-            1.79258772950371181e-01,
-            0.,
-        ]),
-        v: f64x4::from_array([
-            2.68067772490389322e-03 * DAYS_PER_YEAR,
-            1.62824170038242295e-03 * DAYS_PER_YEAR,
-            -9.51592254519715870e-05 * DAYS_PER_YEAR,
-            0.,
-        ]),
-        mass: 5.15138902046611451e-05 * SOLAR_MASS,
-    },
-];
-
-pub fn offset_momentum(bodies: &mut [Body; N_BODIES]) {
-    let (sun, rest) = bodies.split_at_mut(1);
-    let sun = &mut sun[0];
-    for body in rest {
-        let m_ratio = body.mass / SOLAR_MASS;
-        sun.v -= body.v * m_ratio;
+    #[derive(Debug, Clone, Copy)]
+    struct Body {
+        pub x: f64x4,
+        pub v: f64x4,
+        pub mass: f64,
     }
-}
 
-pub fn energy(bodies: &[Body; N_BODIES]) -> f64 {
-    let mut e = 0.;
-    for i in 0..N_BODIES {
-        let bi = &bodies[i];
-        e += bi.mass * (bi.v * bi.v).horizontal_sum() * 0.5;
-        for bj in bodies.iter().take(N_BODIES).skip(i + 1) {
-            let dx = bi.x - bj.x;
-            e -= bi.mass * bj.mass / (dx * dx).horizontal_sum().sqrt()
+    const N_BODIES: usize = 5;
+    const BODIES: [Body; N_BODIES] = [
+        // sun:
+        Body {
+            x: f64x4::from_array([0., 0., 0., 0.]),
+            v: f64x4::from_array([0., 0., 0., 0.]),
+            mass: SOLAR_MASS,
+        },
+        // jupiter:
+        Body {
+            x: f64x4::from_array([
+                4.84143144246472090e+00,
+                -1.16032004402742839e+00,
+                -1.03622044471123109e-01,
+                0.,
+            ]),
+            v: f64x4::from_array([
+                1.66007664274403694e-03 * DAYS_PER_YEAR,
+                7.69901118419740425e-03 * DAYS_PER_YEAR,
+                -6.90460016972063023e-05 * DAYS_PER_YEAR,
+                0.,
+            ]),
+            mass: 9.54791938424326609e-04 * SOLAR_MASS,
+        },
+        // saturn:
+        Body {
+            x: f64x4::from_array([
+                8.34336671824457987e+00,
+                4.12479856412430479e+00,
+                -4.03523417114321381e-01,
+                0.,
+            ]),
+            v: f64x4::from_array([
+                -2.76742510726862411e-03 * DAYS_PER_YEAR,
+                4.99852801234917238e-03 * DAYS_PER_YEAR,
+                2.30417297573763929e-05 * DAYS_PER_YEAR,
+                0.,
+            ]),
+            mass: 2.85885980666130812e-04 * SOLAR_MASS,
+        },
+        // uranus:
+        Body {
+            x: f64x4::from_array([
+                1.28943695621391310e+01,
+                -1.51111514016986312e+01,
+                -2.23307578892655734e-01,
+                0.,
+            ]),
+            v: f64x4::from_array([
+                2.96460137564761618e-03 * DAYS_PER_YEAR,
+                2.37847173959480950e-03 * DAYS_PER_YEAR,
+                -2.96589568540237556e-05 * DAYS_PER_YEAR,
+                0.,
+            ]),
+            mass: 4.36624404335156298e-05 * SOLAR_MASS,
+        },
+        // neptune:
+        Body {
+            x: f64x4::from_array([
+                1.53796971148509165e+01,
+                -2.59193146099879641e+01,
+                1.79258772950371181e-01,
+                0.,
+            ]),
+            v: f64x4::from_array([
+                2.68067772490389322e-03 * DAYS_PER_YEAR,
+                1.62824170038242295e-03 * DAYS_PER_YEAR,
+                -9.51592254519715870e-05 * DAYS_PER_YEAR,
+                0.,
+            ]),
+            mass: 5.15138902046611451e-05 * SOLAR_MASS,
+        },
+    ];
+
+    fn offset_momentum(bodies: &mut [Body; N_BODIES]) {
+        let (sun, rest) = bodies.split_at_mut(1);
+        let sun = &mut sun[0];
+        for body in rest {
+            let m_ratio = body.mass / SOLAR_MASS;
+            sun.v -= body.v * m_ratio;
         }
     }
-    e
-}
 
-pub fn advance(bodies: &mut [Body; N_BODIES], dt: f64) {
-    const N: usize = N_BODIES * (N_BODIES - 1) / 2;
+    fn energy(bodies: &[Body; N_BODIES]) -> f64 {
+        let mut e = 0.;
+        for i in 0..N_BODIES {
+            let bi = &bodies[i];
+            e += bi.mass * (bi.v * bi.v).horizontal_sum() * 0.5;
+            for bj in bodies.iter().take(N_BODIES).skip(i + 1) {
+                let dx = bi.x - bj.x;
+                e -= bi.mass * bj.mass / (dx * dx).horizontal_sum().sqrt()
+            }
+        }
+        e
+    }
+
+    fn advance(bodies: &mut [Body; N_BODIES], dt: f64) {
+        const N: usize = N_BODIES * (N_BODIES - 1) / 2;
+
+        // compute distance between bodies:
+        let mut r = [f64x4::splat(0.); N];
+        {
+            let mut i = 0;
+            for j in 0..N_BODIES {
+                for k in j + 1..N_BODIES {
+                    r[i] = bodies[j].x - bodies[k].x;
+                    i += 1;
+                }
+            }
+        }
+
+        let mut mag = [0.0; N];
+        for i in (0..N).step_by(2) {
+            let d2s = f64x2::from_array([
+                (r[i] * r[i]).horizontal_sum(),
+                (r[i + 1] * r[i + 1]).horizontal_sum(),
+            ]);
+            let dmags = f64x2::splat(dt) / (d2s * d2s.sqrt());
+            mag[i] = dmags[0];
+            mag[i + 1] = dmags[1];
+        }
 
-    // compute distance between bodies:
-    let mut r = [f64x4::splat(0.); N];
-    {
         let mut i = 0;
         for j in 0..N_BODIES {
             for k in j + 1..N_BODIES {
-                r[i] = bodies[j].x - bodies[k].x;
-                i += 1;
+                let f = r[i] * mag[i];
+                bodies[j].v -= f * bodies[k].mass;
+                bodies[k].v += f * bodies[j].mass;
+                i += 1
             }
         }
-    }
-
-    let mut mag = [0.0; N];
-    for i in (0..N).step_by(2) {
-        let d2s = f64x2::from_array([
-            (r[i] * r[i]).horizontal_sum(),
-            (r[i + 1] * r[i + 1]).horizontal_sum(),
-        ]);
-        let dmags = f64x2::splat(dt) / (d2s * d2s.sqrt());
-        mag[i] = dmags[0];
-        mag[i + 1] = dmags[1];
-    }
-
-    let mut i = 0;
-    for j in 0..N_BODIES {
-        for k in j + 1..N_BODIES {
-            let f = r[i] * mag[i];
-            bodies[j].v -= f * bodies[k].mass;
-            bodies[k].v += f * bodies[j].mass;
-            i += 1
+        for body in bodies {
+            body.x += dt * body.v
         }
     }
-    for body in bodies {
-        body.x += dt * body.v
+
+    pub fn run(n: usize) -> (f64, f64) {
+        let mut bodies = BODIES;
+        offset_momentum(&mut bodies);
+        let energy_before = energy(&bodies);
+        for _ in 0..n {
+            advance(&mut bodies, 0.01);
+        }
+        let energy_after = energy(&bodies);
+
+        (energy_before, energy_after)
     }
 }
 
-pub fn run(n: usize) -> (f64, f64) {
-    let mut bodies = BODIES;
-    offset_momentum(&mut bodies);
-    let energy_before = energy(&bodies);
-    for _ in 0..n {
-        advance(&mut bodies, 0.01);
-    }
-    let energy_after = energy(&bodies);
-
-    (energy_before, energy_after)
-}
-
+#[cfg(feature = "std")]
 #[cfg(test)]
 mod tests {
     // Good enough for demonstration purposes, not going for strictness here.
@@ -173,12 +177,17 @@ mod tests {
     #[test]
     fn test() {
         const OUTPUT: [f64; 2] = [-0.169075164, -0.169087605];
-        let (energy_before, energy_after) = super::run(1000);
+        let (energy_before, energy_after) = super::nbody::run(1000);
         assert!(approx_eq_f64(energy_before, OUTPUT[0]));
         assert!(approx_eq_f64(energy_after, OUTPUT[1]));
     }
 }
 
 fn main() {
-    // empty main to pass CI
+    #[cfg(feature = "std")]
+    {
+        let (energy_before, energy_after) = nbody::run(1000);
+        println!("Energy before: {}", energy_before);
+        println!("Energy after:  {}", energy_after);
+    }
 }
diff --git a/crates/core_simd/src/intrinsics.rs b/crates/core_simd/src/intrinsics.rs
index 944026c080a6..916c0dadf752 100644
--- a/crates/core_simd/src/intrinsics.rs
+++ b/crates/core_simd/src/intrinsics.rs
@@ -47,6 +47,7 @@ extern "platform-intrinsic" {
     pub(crate) fn simd_fabs<T>(x: T) -> T;
 
     /// fsqrt
+    #[cfg(feature = "std")]
     pub(crate) fn simd_fsqrt<T>(x: T) -> T;
 
     /// fma
diff --git a/crates/core_simd/src/lib.rs b/crates/core_simd/src/lib.rs
index d8149efe9c7f..5f88e3c63b5b 100644
--- a/crates/core_simd/src/lib.rs
+++ b/crates/core_simd/src/lib.rs
@@ -1,6 +1,7 @@
 #![no_std]
 #![allow(incomplete_features)]
 #![feature(
+    const_evaluatable_checked,
     const_generics,
     platform_intrinsics,
     repr_simd,
@@ -20,8 +21,8 @@ mod reduction;
 mod select;
 pub use select::Select;
 
+#[cfg(feature = "const_evaluatable_checked")]
 mod to_bytes;
-pub use to_bytes::ToBytes;
 
 mod comparisons;
 mod fmt;
diff --git a/crates/core_simd/src/round.rs b/crates/core_simd/src/round.rs
index 74cae0cf9898..c284ade463fc 100644
--- a/crates/core_simd/src/round.rs
+++ b/crates/core_simd/src/round.rs
@@ -41,7 +41,12 @@ macro_rules! implement {
             pub fn fract(self) -> Self {
                 self - self.trunc()
             }
+        }
 
+        impl<const LANES: usize> crate::$type<LANES>
+        where
+            crate::LaneCount<LANES>: crate::SupportedLaneCount,
+        {
             /// Rounds toward zero and converts to the same-width integer type, assuming that
             /// the value is finite and fits in that type.
             ///
diff --git a/crates/core_simd/src/to_bytes.rs b/crates/core_simd/src/to_bytes.rs
index 0823391049fd..31d7dfebe1a7 100644
--- a/crates/core_simd/src/to_bytes.rs
+++ b/crates/core_simd/src/to_bytes.rs
@@ -1,72 +1,39 @@
-mod sealed {
-    pub trait Sealed {}
-}
-use sealed::Sealed;
-
-/// Supporting trait for byte conversion functions.
-pub trait ToBytes: Sealed {
-    /// The bytes representation of this type.
-    type Bytes;
-
-    #[doc(hidden)]
-    fn to_bytes_impl(self) -> Self::Bytes;
-
-    #[doc(hidden)]
-    fn from_bytes_impl(bytes: Self::Bytes) -> Self;
-}
-
 macro_rules! impl_to_bytes {
-    { $name:ident, $($int_width:literal -> $byte_width:literal),* } => {
-        $(
-        impl Sealed for crate::$name<$int_width>
-        where
-            crate::LaneCount<$int_width>: crate::SupportedLaneCount,
-        {}
-
-        impl ToBytes for crate::$name<$int_width>
-        where
-            crate::LaneCount<$int_width>: crate::SupportedLaneCount,
-        {
-            type Bytes = crate::SimdU8<$byte_width>;
-            fn to_bytes_impl(self) -> Self::Bytes {
-                unsafe { core::mem::transmute(self) }
-            }
-            fn from_bytes_impl(bytes: Self::Bytes) -> Self {
-                unsafe { core::mem::transmute(bytes) }
-            }
-        }
-        )*
-
+    { $name:ident, $size:literal } => {
         impl<const LANES: usize> crate::$name<LANES>
         where
             crate::LaneCount<LANES>: crate::SupportedLaneCount,
-            Self: ToBytes,
+            crate::LaneCount<{{ $size * LANES }}>: crate::SupportedLaneCount,
         {
             /// Return the memory representation of this integer as a byte array in native byte
             /// order.
-            pub fn to_ne_bytes(self) -> <Self as ToBytes>::Bytes { self.to_bytes_impl() }
+            pub fn to_ne_bytes(self) -> crate::SimdU8<{{ $size * LANES }}> {
+                unsafe { core::mem::transmute_copy(&self) }
+            }
 
             /// Create a native endian integer value from its memory representation as a byte array
             /// in native endianness.
-            pub fn from_ne_bytes(bytes: <Self as ToBytes>::Bytes) -> Self { Self::from_bytes_impl(bytes) }
+            pub fn from_ne_bytes(bytes: crate::SimdU8<{{ $size * LANES }}>) -> Self {
+                unsafe { core::mem::transmute_copy(&bytes) }
+            }
         }
     }
 }
 
-impl_to_bytes! { SimdU8, 1 -> 1, 2 -> 2, 4 -> 4, 8 -> 8, 16 -> 16, 32 -> 32 }
-impl_to_bytes! { SimdU16, 1 -> 2, 2 -> 4, 4 -> 8, 8 -> 16, 16 -> 32 }
-impl_to_bytes! { SimdU32, 1 -> 4, 2 -> 8, 4 -> 16, 8 -> 32 }
-impl_to_bytes! { SimdU64, 1 -> 8, 2 -> 16, 4 -> 32 }
+impl_to_bytes! { SimdU8, 1 }
+impl_to_bytes! { SimdU16, 2 }
+impl_to_bytes! { SimdU32, 4 }
+impl_to_bytes! { SimdU64, 8 }
 #[cfg(target_pointer_width = "32")]
-impl_to_bytes! { SimdUsize, 1 -> 4, 2 -> 8, 4 -> 16, 8 -> 32 }
+impl_to_bytes! { SimdUsize, 4 }
 #[cfg(target_pointer_width = "64")]
-impl_to_bytes! { SimdUsize, 1 -> 8, 2 -> 16, 4 -> 32 }
+impl_to_bytes! { SimdUsize, 8 }
 
-impl_to_bytes! { SimdI8, 1 -> 1, 2 -> 2, 4 -> 4, 8 -> 8, 16 -> 16, 32 -> 32 }
-impl_to_bytes! { SimdI16, 1 -> 2, 2 -> 4, 4 -> 8, 8 -> 16, 16 -> 32 }
-impl_to_bytes! { SimdI32, 1 -> 4, 2 -> 8, 4 -> 16, 8 -> 32 }
-impl_to_bytes! { SimdI64, 1 -> 8, 2 -> 16, 4 -> 32 }
+impl_to_bytes! { SimdI8, 1 }
+impl_to_bytes! { SimdI16, 2 }
+impl_to_bytes! { SimdI32, 4 }
+impl_to_bytes! { SimdI64, 8 }
 #[cfg(target_pointer_width = "32")]
-impl_to_bytes! { SimdIsize, 1 -> 4, 2 -> 8, 4 -> 16, 8 -> 32 }
+impl_to_bytes! { SimdIsize, 4 }
 #[cfg(target_pointer_width = "64")]
-impl_to_bytes! { SimdIsize, 1 -> 8, 2 -> 16, 4 -> 32 }
+impl_to_bytes! { SimdIsize, 8 }
diff --git a/crates/core_simd/tests/ops_macros.rs b/crates/core_simd/tests/ops_macros.rs
index cb39e7377054..81553c34aa77 100644
--- a/crates/core_simd/tests/ops_macros.rs
+++ b/crates/core_simd/tests/ops_macros.rs
@@ -443,14 +443,6 @@ macro_rules! impl_float_tests {
                     )
                 }
 
-                fn sqrt<const LANES: usize>() {
-                    test_helpers::test_unary_elementwise(
-                        &Vector::<LANES>::sqrt,
-                        &Scalar::sqrt,
-                        &|_| true,
-                    )
-                }
-
                 fn recip<const LANES: usize>() {
                     test_helpers::test_unary_elementwise(
                         &Vector::<LANES>::recip,
@@ -605,6 +597,17 @@ macro_rules! impl_float_tests {
                     });
                 }
             }
+
+            #[cfg(feature = "std")]
+            test_helpers::test_lanes! {
+                fn sqrt<const LANES: usize>() {
+                    test_helpers::test_unary_elementwise(
+                        &Vector::<LANES>::sqrt,
+                        &Scalar::sqrt,
+                        &|_| true,
+                    )
+                }
+            }
         }
     }
 }
diff --git a/crates/core_simd/tests/to_bytes.rs b/crates/core_simd/tests/to_bytes.rs
index 8d662b3238c9..22c97c95d927 100644
--- a/crates/core_simd/tests/to_bytes.rs
+++ b/crates/core_simd/tests/to_bytes.rs
@@ -1,4 +1,6 @@
-#![feature(portable_simd)]
+#![feature(portable_simd, const_generics, const_evaluatable_checked)]
+#![allow(incomplete_features)]
+#![cfg(feature = "const_evaluatable_checked")]
 
 use core_simd::SimdU32;
 

From 1f69bc459a2d4d2a34009553e57ff93598e6a342 Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Wed, 28 Jul 2021 04:26:55 +0000
Subject: [PATCH 200/249] Add CI for testing cargo features

---
 .github/workflows/ci.yml | 25 +++++++++++++++++++++++++
 1 file changed, 25 insertions(+)

diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
index 9c62a6d40aee..2f15dcc6c161 100644
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -196,3 +196,28 @@ jobs:
       - name: Test (release)
         run: cross test --verbose --target=${{ matrix.target }} --release
 
+  features:
+    name: "Check cargo features (${{ matrix.features }} ${{ matrix.rustflags }})"
+    runs-on: ubuntu-latest
+    strategy:
+      fail-fast: false
+      matrix:
+        rustflags:
+          - ""
+          - "-Ctarget-feature=+avx512" # AVX-512 uses packed bit masks, so enable it to test more code paths
+        features:
+          - ""
+          - "--feature std"
+          - "--feature const_evaluatable_checked"
+          - "--feature std --feature const_evaluatable_checked"
+
+    steps:
+      - uses: actions/checkout@v2
+      - name: Setup Rust
+        run: |
+          rustup update nightly --no-self-update
+          rustup default nightly
+      - name: Check build
+        run: cargo check --all-targets --no-default-features ${{ matrix.features }}
+        env:
+            RUSTFLAGS: ${{ matrix.rustflags }}

From 9ab050796f24b3dcd5b56c303bc48024af027eb4 Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Wed, 28 Jul 2021 04:33:57 +0000
Subject: [PATCH 201/249] Fix feature flag in CI

---
 .github/workflows/ci.yml | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
index 2f15dcc6c161..454bc3154751 100644
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -207,9 +207,9 @@ jobs:
           - "-Ctarget-feature=+avx512" # AVX-512 uses packed bit masks, so enable it to test more code paths
         features:
           - ""
-          - "--feature std"
-          - "--feature const_evaluatable_checked"
-          - "--feature std --feature const_evaluatable_checked"
+          - "--features std"
+          - "--features const_evaluatable_checked"
+          - "--features std --features const_evaluatable_checked"
 
     steps:
       - uses: actions/checkout@v2

From cca91024298b92f5bff5fc7353155aff0eef38e5 Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Thu, 29 Jul 2021 04:55:28 +0000
Subject: [PATCH 202/249] Change bitmasks to use less opaque type

---
 crates/core_simd/src/lane_count.rs       |  7 ++++++-
 crates/core_simd/src/masks.rs            |  4 ++--
 crates/core_simd/src/masks/bitmask.rs    | 10 ++++++----
 crates/core_simd/src/masks/full_masks.rs | 10 +++++-----
 4 files changed, 19 insertions(+), 12 deletions(-)

diff --git a/crates/core_simd/src/lane_count.rs b/crates/core_simd/src/lane_count.rs
index 8fe204dff98e..b017e7d137e3 100644
--- a/crates/core_simd/src/lane_count.rs
+++ b/crates/core_simd/src/lane_count.rs
@@ -6,9 +6,14 @@ use sealed::Sealed;
 /// A type representing a vector lane count.
 pub struct LaneCount<const LANES: usize>;
 
+impl<const LANES: usize> LaneCount<LANES> {
+    /// The number of bytes in a bitmask with this many lanes.
+    pub const BITMASK_LEN: usize = (LANES + 7) / 8;
+}
+
 /// Helper trait for vector lane counts.
 pub trait SupportedLaneCount: Sealed {
-    /// The bitmask representation of a mask.
+    #[doc(hidden)]
     type BitMask: Copy + Default + AsRef<[u8]> + AsMut<[u8]>;
 
     #[doc(hidden)]
diff --git a/crates/core_simd/src/masks.rs b/crates/core_simd/src/masks.rs
index d3338a6d366e..ba7da704f61d 100644
--- a/crates/core_simd/src/masks.rs
+++ b/crates/core_simd/src/masks.rs
@@ -160,12 +160,12 @@ macro_rules! define_opaque_mask {
             }
 
             /// Convert this mask to a bitmask, with one bit set per lane.
-            pub fn to_bitmask(self) -> <crate::LaneCount<LANES> as crate::SupportedLaneCount>::BitMask {
+            pub fn to_bitmask(self) -> [u8; crate::LaneCount::<LANES>::BITMASK_LEN] {
                 self.0.to_bitmask()
             }
 
             /// Convert a bitmask to a mask.
-            pub fn from_bitmask(bitmask: <crate::LaneCount<LANES> as crate::SupportedLaneCount>::BitMask) -> Self {
+            pub fn from_bitmask(bitmask: [u8; crate::LaneCount::<LANES>::BITMASK_LEN]) -> Self {
                 Self(<$inner_ty>::from_bitmask(bitmask))
             }
         }
diff --git a/crates/core_simd/src/masks/bitmask.rs b/crates/core_simd/src/masks/bitmask.rs
index b6897728988b..69edd5235872 100644
--- a/crates/core_simd/src/masks/bitmask.rs
+++ b/crates/core_simd/src/masks/bitmask.rs
@@ -128,13 +128,15 @@ where
     }
 
     #[inline]
-    pub fn to_bitmask(self) -> <LaneCount<LANES> as SupportedLaneCount>::BitMask {
-        self.0
+    pub fn to_bitmask(self) -> [u8; LaneCount::<LANES>::BITMASK_LEN] {
+        // Safety: these are the same type and we are laundering the generic
+        unsafe { core::mem::transmute_copy(&self.0) }
     }
 
     #[inline]
-    pub fn from_bitmask(bitmask: <LaneCount<LANES> as SupportedLaneCount>::BitMask) -> Self {
-        Self(bitmask)
+    pub fn from_bitmask(bitmask: [u8; LaneCount::<LANES>::BITMASK_LEN]) -> Self {
+        // Safety: these are the same type and we are laundering the generic
+        Self(unsafe { core::mem::transmute_copy(&bitmask) })
     }
 
     #[inline]
diff --git a/crates/core_simd/src/masks/full_masks.rs b/crates/core_simd/src/masks/full_masks.rs
index af36571134ee..2923cf1964a0 100644
--- a/crates/core_simd/src/masks/full_masks.rs
+++ b/crates/core_simd/src/masks/full_masks.rs
@@ -103,15 +103,15 @@ macro_rules! define_mask {
             }
 
             #[inline]
-            pub fn to_bitmask(self) -> <crate::LaneCount::<LANES> as crate::SupportedLaneCount>::BitMask {
+            pub fn to_bitmask(self) -> [u8; crate::LaneCount::<LANES>::BITMASK_LEN] {
                 unsafe {
                     // TODO remove the transmute when rustc can use arrays of u8 as bitmasks
                     assert_eq!(
-                        core::mem::size_of::<<crate::LaneCount::<LANES> as crate::SupportedLaneCount>::BitMask>(),
                         core::mem::size_of::<<crate::LaneCount::<LANES> as crate::SupportedLaneCount>::IntBitMask>(),
+                        crate::LaneCount::<LANES>::BITMASK_LEN,
                     );
                     let bitmask: <crate::LaneCount::<LANES> as crate::SupportedLaneCount>::IntBitMask = crate::intrinsics::simd_bitmask(self.0);
-                    let mut bitmask: <crate::LaneCount::<LANES> as crate::SupportedLaneCount>::BitMask = core::mem::transmute_copy(&bitmask);
+                    let mut bitmask: [u8; crate::LaneCount::<LANES>::BITMASK_LEN] = core::mem::transmute_copy(&bitmask);
 
                     // There is a bug where LLVM appears to implement this operation with the wrong
                     // bit order.
@@ -127,7 +127,7 @@ macro_rules! define_mask {
             }
 
             #[inline]
-            pub fn from_bitmask(mut bitmask: <crate::LaneCount::<LANES> as crate::SupportedLaneCount>::BitMask) -> Self {
+            pub fn from_bitmask(mut bitmask: [u8; crate::LaneCount::<LANES>::BITMASK_LEN]) -> Self {
                 unsafe {
                     // There is a bug where LLVM appears to implement this operation with the wrong
                     // bit order.
@@ -140,8 +140,8 @@ macro_rules! define_mask {
 
                     // TODO remove the transmute when rustc can use arrays of u8 as bitmasks
                     assert_eq!(
-                        core::mem::size_of::<<crate::LaneCount::<LANES> as crate::SupportedLaneCount>::BitMask>(),
                         core::mem::size_of::<<crate::LaneCount::<LANES> as crate::SupportedLaneCount>::IntBitMask>(),
+                        crate::LaneCount::<LANES>::BITMASK_LEN,
                     );
                     let bitmask: <crate::LaneCount::<LANES> as crate::SupportedLaneCount>::IntBitMask = core::mem::transmute_copy(&bitmask);
 

From 2acf204b09740f06dfb5efe8d1f45ceeb4ca91df Mon Sep 17 00:00:00 2001
From: Jubilee Young <workingjubilee@gmail.com>
Date: Tue, 27 Jul 2021 19:59:50 -0700
Subject: [PATCH 203/249] Rename to portable-simd and remove other names

Clean up references to the repo's previous name.
Removes the authors field, which is non-obligatory since RFC 3052.
Better to omit than confound: let git log be our witness.
---
 README.md                      | 8 ++++----
 crates/core_simd/Cargo.toml    | 5 ++---
 crates/test_helpers/Cargo.toml | 1 -
 3 files changed, 6 insertions(+), 8 deletions(-)

diff --git a/README.md b/README.md
index ee28297668e9..da536a4d6f2d 100644
--- a/README.md
+++ b/README.md
@@ -1,5 +1,5 @@
-# stdsimd - Rust's standard library portable SIMD API
-[![Build Status](https://travis-ci.com/rust-lang/stdsimd.svg?branch=master)](https://travis-ci.com/rust-lang/stdsimd)
+# The Rust standard library's portable SIMD API
+[![Build Status](https://travis-ci.com/rust-lang/portable-simd.svg?branch=master)](https://travis-ci.com/rust-lang/portable-simd)
 
 Code repository for the [Portable SIMD Project Group](https://github.com/rust-lang/project-portable-simd).
 Please refer to [CONTRIBUTING.md](./CONTRIBUTING.md) for our contributing guidelines.
@@ -31,7 +31,7 @@ name = "hellosimd"
 version = "0.1.0"
 edition = "2018"
 [dependencies]
-core_simd = { git = "https://github.com/rust-lang/stdsimd" }
+core_simd = { git = "https://github.com/rust-lang/portable-simd" }
 ```
 
 and finally write this in `src/main.rs`:
@@ -66,4 +66,4 @@ The `mask` types are "truthy" values, but they use the number of bits in their n
 [simd-guide]: ./beginners-guide.md
 [zulip-project-portable-simd]: https://rust-lang.zulipchat.com/#narrow/stream/257879-project-portable-simd
 [stdarch]: https://github.com/rust-lang/stdarch
-[docs]: https://rust-lang.github.io/stdsimd/core_simd
+[docs]: https://rust-lang.github.io/portable-simd/core_simd
diff --git a/crates/core_simd/Cargo.toml b/crates/core_simd/Cargo.toml
index 6044eabcd140..a6b9f8f353fe 100644
--- a/crates/core_simd/Cargo.toml
+++ b/crates/core_simd/Cargo.toml
@@ -1,10 +1,9 @@
 [package]
 name = "core_simd"
 version = "0.1.0"
-authors = ["Caleb Zulawski <caleb.zulawski@gmail.com>"]
 edition = "2018"
-homepage = "https://github.com/rust-lang/stdsimd"
-repository = "https://github.com/rust-lang/stdsimd"
+homepage = "https://github.com/rust-lang/portable-simd"
+repository = "https://github.com/rust-lang/portable-simd"
 keywords = ["core", "simd", "intrinsics"]
 categories = ["hardware-support", "no-std"]
 license = "MIT OR Apache-2.0"
diff --git a/crates/test_helpers/Cargo.toml b/crates/test_helpers/Cargo.toml
index c9f6397b23b9..e38b223d6c92 100644
--- a/crates/test_helpers/Cargo.toml
+++ b/crates/test_helpers/Cargo.toml
@@ -1,7 +1,6 @@
 [package]
 name = "test_helpers"
 version = "0.1.0"
-authors = ["Caleb Zulawski <caleb.zulawski@gmail.com>"]
 edition = "2018"
 publish = false
 

From 054f25f2b0212ada7caae70a4470e32f503a3eea Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Fri, 6 Aug 2021 02:31:24 +0000
Subject: [PATCH 204/249] Convert all vectors to a single type

---
 crates/core_simd/src/vector.rs       | 76 ++++++++++++++++++++++++++++
 crates/core_simd/src/vector/float.rs | 13 ++---
 crates/core_simd/src/vector/int.rs   | 41 ++++-----------
 crates/core_simd/src/vector/uint.rs  | 41 ++++-----------
 4 files changed, 101 insertions(+), 70 deletions(-)

diff --git a/crates/core_simd/src/vector.rs b/crates/core_simd/src/vector.rs
index 1f6df533767e..ea1a732f2035 100644
--- a/crates/core_simd/src/vector.rs
+++ b/crates/core_simd/src/vector.rs
@@ -12,9 +12,85 @@ pub use uint::*;
 // Vectors of pointers are not for public use at the current time.
 pub(crate) mod ptr;
 
+use crate::{LaneCount, SupportedLaneCount};
+
+/// A SIMD vector of `LANES` elements of type `Element`.
+#[repr(simd)]
+pub struct Simd<Element, const LANES: usize>([Element; LANES])
+where
+    Element: SimdElement,
+    LaneCount<LANES>: SupportedLaneCount;
+
 mod sealed {
     pub trait Sealed {}
 }
+use sealed::Sealed;
+
+/// Marker trait for types that may be used as SIMD vector elements.
+pub unsafe trait SimdElement: Sealed {
+    /// The mask element type corresponding to this element type.
+    type Mask: SimdElement;
+}
+
+impl Sealed for u8 {}
+unsafe impl SimdElement for u8 {
+    type Mask = u8;
+}
+
+impl Sealed for u16 {}
+unsafe impl SimdElement for u16 {
+    type Mask = u16;
+}
+
+impl Sealed for u32 {}
+unsafe impl SimdElement for u32 {
+    type Mask = u32;
+}
+
+impl Sealed for u64 {}
+unsafe impl SimdElement for u64 {
+    type Mask = u64;
+}
+
+impl Sealed for usize {}
+unsafe impl SimdElement for usize {
+    type Mask = usize;
+}
+
+impl Sealed for i8 {}
+unsafe impl SimdElement for i8 {
+    type Mask = i8;
+}
+
+impl Sealed for i16 {}
+unsafe impl SimdElement for i16 {
+    type Mask = i16;
+}
+
+impl Sealed for i32 {}
+unsafe impl SimdElement for i32 {
+    type Mask = i32;
+}
+
+impl Sealed for i64 {}
+unsafe impl SimdElement for i64 {
+    type Mask = i64;
+}
+
+impl Sealed for isize {}
+unsafe impl SimdElement for isize {
+    type Mask = isize;
+}
+
+impl Sealed for f32 {}
+unsafe impl SimdElement for f32 {
+    type Mask = i32;
+}
+
+impl Sealed for f64 {}
+unsafe impl SimdElement for f64 {
+    type Mask = i64;
+}
 
 /// A representation of a vector as an "array" with indices, implementing
 /// operations applicable to any vector type based solely on "having lanes",
diff --git a/crates/core_simd/src/vector/float.rs b/crates/core_simd/src/vector/float.rs
index bdeccd037a80..231fe590ada5 100644
--- a/crates/core_simd/src/vector/float.rs
+++ b/crates/core_simd/src/vector/float.rs
@@ -187,19 +187,12 @@ macro_rules! impl_float_vector {
 }
 
 /// A SIMD vector of containing `LANES` `f32` values.
-#[repr(simd)]
-pub struct SimdF32<const LANES: usize>([f32; LANES])
-where
-    LaneCount<LANES>: SupportedLaneCount;
-
-impl_float_vector! { SimdF32, f32, SimdU32, Mask32, SimdI32 }
+pub type SimdF32<const LANES: usize> = crate::Simd<f32, LANES>;
 
 /// A SIMD vector of containing `LANES` `f64` values.
-#[repr(simd)]
-pub struct SimdF64<const LANES: usize>([f64; LANES])
-where
-    LaneCount<LANES>: SupportedLaneCount;
+pub type SimdF64<const LANES: usize> = crate::Simd<f64, LANES>;
 
+impl_float_vector! { SimdF32, f32, SimdU32, Mask32, SimdI32 }
 impl_float_vector! { SimdF64, f64, SimdU64, Mask64, SimdI64 }
 
 /// Vector of two `f32` values
diff --git a/crates/core_simd/src/vector/int.rs b/crates/core_simd/src/vector/int.rs
index 73c737762fb6..88a17daa2f4b 100644
--- a/crates/core_simd/src/vector/int.rs
+++ b/crates/core_simd/src/vector/int.rs
@@ -62,44 +62,25 @@ macro_rules! impl_integer_vector {
     }
 }
 
-/// A SIMD vector of containing `LANES` `isize` values.
-#[repr(simd)]
-pub struct SimdIsize<const LANES: usize>([isize; LANES])
-where
-    LaneCount<LANES>: SupportedLaneCount;
-
-impl_integer_vector! { SimdIsize, isize, MaskSize, SimdIsize }
+/// A SIMD vector of containing `LANES` `i8` values.
+pub type SimdI8<const LANES: usize> = crate::Simd<i8, LANES>;
 
 /// A SIMD vector of containing `LANES` `i16` values.
-#[repr(simd)]
-pub struct SimdI16<const LANES: usize>([i16; LANES])
-where
-    LaneCount<LANES>: SupportedLaneCount;
-
-impl_integer_vector! { SimdI16, i16, Mask16, SimdI16 }
+pub type SimdI16<const LANES: usize> = crate::Simd<i16, LANES>;
 
 /// A SIMD vector of containing `LANES` `i32` values.
-#[repr(simd)]
-pub struct SimdI32<const LANES: usize>([i32; LANES])
-where
-    LaneCount<LANES>: SupportedLaneCount;
-
-impl_integer_vector! { SimdI32, i32, Mask32, SimdI32 }
+pub type SimdI32<const LANES: usize> = crate::Simd<i32, LANES>;
 
 /// A SIMD vector of containing `LANES` `i64` values.
-#[repr(simd)]
-pub struct SimdI64<const LANES: usize>([i64; LANES])
-where
-    LaneCount<LANES>: SupportedLaneCount;
+pub type SimdI64<const LANES: usize> = crate::Simd<i64, LANES>;
 
+/// A SIMD vector of containing `LANES` `isize` values.
+pub type SimdIsize<const LANES: usize> = crate::Simd<isize, LANES>;
+
+impl_integer_vector! { SimdIsize, isize, MaskSize, SimdIsize }
+impl_integer_vector! { SimdI16, i16, Mask16, SimdI16 }
+impl_integer_vector! { SimdI32, i32, Mask32, SimdI32 }
 impl_integer_vector! { SimdI64, i64, Mask64, SimdI64 }
-
-/// A SIMD vector of containing `LANES` `i8` values.
-#[repr(simd)]
-pub struct SimdI8<const LANES: usize>([i8; LANES])
-where
-    LaneCount<LANES>: SupportedLaneCount;
-
 impl_integer_vector! { SimdI8, i8, Mask8, SimdI8 }
 
 /// Vector of two `isize` values
diff --git a/crates/core_simd/src/vector/uint.rs b/crates/core_simd/src/vector/uint.rs
index b19f694872ae..5bd1a7fd67fd 100644
--- a/crates/core_simd/src/vector/uint.rs
+++ b/crates/core_simd/src/vector/uint.rs
@@ -33,44 +33,25 @@ macro_rules! impl_unsigned_vector {
     }
 }
 
-/// A SIMD vector of containing `LANES` `usize` values.
-#[repr(simd)]
-pub struct SimdUsize<const LANES: usize>([usize; LANES])
-where
-    LaneCount<LANES>: SupportedLaneCount;
-
-impl_unsigned_vector! { SimdUsize, usize }
+/// A SIMD vector of containing `LANES` `u8` values.
+pub type SimdU8<const LANES: usize> = crate::Simd<u8, LANES>;
 
 /// A SIMD vector of containing `LANES` `u16` values.
-#[repr(simd)]
-pub struct SimdU16<const LANES: usize>([u16; LANES])
-where
-    LaneCount<LANES>: SupportedLaneCount;
-
-impl_unsigned_vector! { SimdU16, u16 }
+pub type SimdU16<const LANES: usize> = crate::Simd<u16, LANES>;
 
 /// A SIMD vector of containing `LANES` `u32` values.
-#[repr(simd)]
-pub struct SimdU32<const LANES: usize>([u32; LANES])
-where
-    LaneCount<LANES>: SupportedLaneCount;
-
-impl_unsigned_vector! { SimdU32, u32 }
+pub type SimdU32<const LANES: usize> = crate::Simd<u32, LANES>;
 
 /// A SIMD vector of containing `LANES` `u64` values.
-#[repr(simd)]
-pub struct SimdU64<const LANES: usize>([u64; LANES])
-where
-    LaneCount<LANES>: SupportedLaneCount;
+pub type SimdU64<const LANES: usize> = crate::Simd<u64, LANES>;
 
+/// A SIMD vector of containing `LANES` `usize` values.
+pub type SimdUsize<const LANES: usize> = crate::Simd<usize, LANES>;
+
+impl_unsigned_vector! { SimdUsize, usize }
+impl_unsigned_vector! { SimdU16, u16 }
+impl_unsigned_vector! { SimdU32, u32 }
 impl_unsigned_vector! { SimdU64, u64 }
-
-/// A SIMD vector of containing `LANES` `u8` values.
-#[repr(simd)]
-pub struct SimdU8<const LANES: usize>([u8; LANES])
-where
-    LaneCount<LANES>: SupportedLaneCount;
-
 impl_unsigned_vector! { SimdU8, u8 }
 
 /// Vector of two `usize` values

From dc4dc99649471a06a3fd5dbffabf71eab8ff8f95 Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Fri, 6 Aug 2021 03:45:57 +0000
Subject: [PATCH 205/249] Change to various generic impls

---
 crates/core_simd/src/fmt.rs                | 102 ++-----
 crates/core_simd/src/lib.rs                |   1 +
 crates/core_simd/src/permute.rs            |  21 +-
 crates/core_simd/src/vector.rs             | 338 ++++++++++++++++++++-
 crates/core_simd/src/vector/float.rs       |   1 -
 crates/core_simd/src/vector/int.rs         |  24 --
 crates/core_simd/src/vector/uint.rs        |  26 --
 crates/core_simd/src/vector/vector_impl.rs | 257 ----------------
 8 files changed, 370 insertions(+), 400 deletions(-)
 delete mode 100644 crates/core_simd/src/vector/vector_impl.rs

diff --git a/crates/core_simd/src/fmt.rs b/crates/core_simd/src/fmt.rs
index 78ae5ce3fcea..9ad3a6c100ea 100644
--- a/crates/core_simd/src/fmt.rs
+++ b/crates/core_simd/src/fmt.rs
@@ -1,88 +1,36 @@
-macro_rules! debug_wrapper {
-    { $($trait:ident => $name:ident,)* } => {
+macro_rules! impl_fmt_trait {
+    { $($trait:ident,)* } => {
         $(
-            pub(crate) fn $name<T: core::fmt::$trait>(slice: &[T], f: &mut core::fmt::Formatter) -> core::fmt::Result {
-                #[repr(transparent)]
-                struct Wrapper<'a, T: core::fmt::$trait>(&'a T);
+            impl<Element, const LANES: usize> core::fmt::$trait for crate::Simd<Element, LANES>
+            where
+                crate::LaneCount<LANES>: crate::SupportedLaneCount,
+                Element: crate::SimdElement + core::fmt::$trait,
+            {
+                fn fmt(&self, f: &mut core::fmt::Formatter) -> core::fmt::Result {
+                    #[repr(transparent)]
+                    struct Wrapper<'a, T: core::fmt::$trait>(&'a T);
 
-                impl<T: core::fmt::$trait> core::fmt::Debug for Wrapper<'_, T> {
-                    fn fmt(&self, f: &mut core::fmt::Formatter) -> core::fmt::Result {
-                        self.0.fmt(f)
+                    impl<T: core::fmt::$trait> core::fmt::Debug for Wrapper<'_, T> {
+                        fn fmt(&self, f: &mut core::fmt::Formatter) -> core::fmt::Result {
+                            self.0.fmt(f)
+                        }
                     }
-                }
 
-                f.debug_list()
-                    .entries(slice.iter().map(|x| Wrapper(x)))
-                    .finish()
+                    f.debug_list()
+                        .entries(self.as_array().iter().map(|x| Wrapper(x)))
+                        .finish()
+                }
             }
         )*
     }
 }
 
-debug_wrapper! {
-    Debug => format,
-    Binary => format_binary,
-    LowerExp => format_lower_exp,
-    UpperExp => format_upper_exp,
-    Octal => format_octal,
-    LowerHex => format_lower_hex,
-    UpperHex => format_upper_hex,
-}
-
-macro_rules! impl_fmt_trait {
-    { $($type:ident => $(($trait:ident, $format:ident)),*;)* } => {
-        $( // repeat type
-            $( // repeat trait
-                impl<const LANES: usize> core::fmt::$trait for crate::$type<LANES>
-                where
-                    crate::LaneCount<LANES>: crate::SupportedLaneCount,
-                {
-                    fn fmt(&self, f: &mut core::fmt::Formatter) -> core::fmt::Result {
-                        $format(self.as_ref(), f)
-                    }
-                }
-            )*
-        )*
-    };
-    { integers: $($type:ident,)* } => {
-        impl_fmt_trait! {
-            $($type =>
-              (Debug, format),
-              (Binary, format_binary),
-              (LowerExp, format_lower_exp),
-              (UpperExp, format_upper_exp),
-              (Octal, format_octal),
-              (LowerHex, format_lower_hex),
-              (UpperHex, format_upper_hex);
-            )*
-        }
-    };
-    { floats: $($type:ident,)* } => {
-        impl_fmt_trait! {
-            $($type =>
-              (Debug, format),
-              (LowerExp, format_lower_exp),
-              (UpperExp, format_upper_exp);
-            )*
-        }
-    };
-    { masks: $($type:ident,)* } => {
-        impl_fmt_trait! {
-            $($type =>
-              (Debug, format);
-            )*
-        }
-    }
-}
-
 impl_fmt_trait! {
-    integers:
-        SimdU8, SimdU16, SimdU32, SimdU64,
-        SimdI8, SimdI16, SimdI32, SimdI64,
-        SimdUsize, SimdIsize,
-}
-
-impl_fmt_trait! {
-    floats:
-        SimdF32, SimdF64,
+    Debug,
+    Binary,
+    LowerExp,
+    UpperExp,
+    Octal,
+    LowerHex,
+    UpperHex,
 }
diff --git a/crates/core_simd/src/lib.rs b/crates/core_simd/src/lib.rs
index 5f88e3c63b5b..fc0df1813b94 100644
--- a/crates/core_simd/src/lib.rs
+++ b/crates/core_simd/src/lib.rs
@@ -2,6 +2,7 @@
 #![allow(incomplete_features)]
 #![feature(
     const_evaluatable_checked,
+    const_fn_trait_bound,
     const_generics,
     platform_intrinsics,
     repr_simd,
diff --git a/crates/core_simd/src/permute.rs b/crates/core_simd/src/permute.rs
index 01148a26bad4..4e377d68915b 100644
--- a/crates/core_simd/src/permute.rs
+++ b/crates/core_simd/src/permute.rs
@@ -1,6 +1,9 @@
 macro_rules! impl_shuffle_lane {
-    { $name:ident, $fn:ident, $n:literal } => {
-        impl $name<$n> {
+    { $fn:ident, $n:literal } => {
+        impl<Element> crate::Simd<Element, $n>
+        where
+            Element: crate::SimdElement,
+        {
             /// A const SIMD shuffle that takes 2 SIMD vectors and produces another vector, using
             /// the indices in the const parameter. The first or "self" vector will have its lanes
             /// indexed from 0, and the second vector will have its first lane indexed at $n.
@@ -138,12 +141,8 @@ macro_rules! impl_shuffle_lane {
     }
 }
 
-macro_rules! impl_shuffle_2pow_lanes {
-    { $name:ident } => {
-        impl_shuffle_lane!{ $name, simd_shuffle2, 2 }
-        impl_shuffle_lane!{ $name, simd_shuffle4, 4 }
-        impl_shuffle_lane!{ $name, simd_shuffle8, 8 }
-        impl_shuffle_lane!{ $name, simd_shuffle16, 16 }
-        impl_shuffle_lane!{ $name, simd_shuffle32, 32 }
-    }
-}
+impl_shuffle_lane! { simd_shuffle2, 2 }
+impl_shuffle_lane! { simd_shuffle4, 4 }
+impl_shuffle_lane! { simd_shuffle8, 8 }
+impl_shuffle_lane! { simd_shuffle16, 16 }
+impl_shuffle_lane! { simd_shuffle32, 32 }
diff --git a/crates/core_simd/src/vector.rs b/crates/core_simd/src/vector.rs
index ea1a732f2035..2e3855bff3cf 100644
--- a/crates/core_simd/src/vector.rs
+++ b/crates/core_simd/src/vector.rs
@@ -1,6 +1,3 @@
-#[macro_use]
-mod vector_impl;
-
 mod float;
 mod int;
 mod uint;
@@ -21,13 +18,325 @@ where
     Element: SimdElement,
     LaneCount<LANES>: SupportedLaneCount;
 
+impl<Element, const LANES: usize> Simd<Element, LANES>
+where
+    LaneCount<LANES>: SupportedLaneCount,
+    Element: SimdElement,
+{
+    /// Construct a SIMD vector by setting all lanes to the given value.
+    pub const fn splat(value: Element) -> Self {
+        Self([value; LANES])
+    }
+
+    /// Returns an array reference containing the entire SIMD vector.
+    pub const fn as_array(&self) -> &[Element; LANES] {
+        &self.0
+    }
+
+    /// Returns a mutable array reference containing the entire SIMD vector.
+    pub fn as_mut_array(&mut self) -> &mut [Element; LANES] {
+        &mut self.0
+    }
+
+    /// Converts an array to a SIMD vector.
+    pub const fn from_array(array: [Element; LANES]) -> Self {
+        Self(array)
+    }
+
+    /// Converts a SIMD vector to an array.
+    pub const fn to_array(self) -> [Element; LANES] {
+        self.0
+    }
+
+    /// SIMD gather: construct a SIMD vector by reading from a slice, using potentially discontiguous indices.
+    /// If an index is out of bounds, that lane instead selects the value from the "or" vector.
+    /// ```
+    /// # #![feature(portable_simd)]
+    /// # use core_simd::*;
+    /// let vec: Vec<i32> = vec![10, 11, 12, 13, 14, 15, 16, 17, 18];
+    /// let idxs = SimdUsize::<4>::from_array([9, 3, 0, 5]);
+    /// let alt = SimdI32::from_array([-5, -4, -3, -2]);
+    ///
+    /// let result = SimdI32::<4>::gather_or(&vec, idxs, alt); // Note the lane that is out-of-bounds.
+    /// assert_eq!(result, SimdI32::from_array([-5, 13, 10, 15]));
+    /// ```
+    #[must_use]
+    #[inline]
+    pub fn gather_or(slice: &[Element], idxs: crate::SimdUsize<LANES>, or: Self) -> Self {
+        Self::gather_select(slice, crate::MaskSize::splat(true), idxs, or)
+    }
+
+    /// SIMD gather: construct a SIMD vector by reading from a slice, using potentially discontiguous indices.
+    /// Out-of-bounds indices instead use the default value for that lane (0).
+    /// ```
+    /// # #![feature(portable_simd)]
+    /// # use core_simd::*;
+    /// let vec: Vec<i32> = vec![10, 11, 12, 13, 14, 15, 16, 17, 18];
+    /// let idxs = SimdUsize::<4>::from_array([9, 3, 0, 5]);
+    ///
+    /// let result = SimdI32::<4>::gather_or_default(&vec, idxs); // Note the lane that is out-of-bounds.
+    /// assert_eq!(result, SimdI32::from_array([0, 13, 10, 15]));
+    /// ```
+    #[must_use]
+    #[inline]
+    pub fn gather_or_default(slice: &[Element], idxs: crate::SimdUsize<LANES>) -> Self
+    where
+        Element: Default,
+    {
+        Self::gather_or(slice, idxs, Self::splat(Element::default()))
+    }
+
+    /// SIMD gather: construct a SIMD vector by reading from a slice, using potentially discontiguous indices.
+    /// Out-of-bounds or masked indices instead select the value from the "or" vector.
+    /// ```
+    /// # #![feature(portable_simd)]
+    /// # use core_simd::*;
+    /// let vec: Vec<i32> = vec![10, 11, 12, 13, 14, 15, 16, 17, 18];
+    /// let idxs = SimdUsize::<4>::from_array([9, 3, 0, 5]);
+    /// let alt = SimdI32::from_array([-5, -4, -3, -2]);
+    /// let mask = MaskSize::from_array([true, true, true, false]); // Note the mask of the last lane.
+    ///
+    /// let result = SimdI32::<4>::gather_select(&vec, mask, idxs, alt); // Note the lane that is out-of-bounds.
+    /// assert_eq!(result, SimdI32::from_array([-5, 13, 10, -2]));
+    /// ```
+    #[must_use]
+    #[inline]
+    pub fn gather_select(
+        slice: &[Element],
+        mask: crate::MaskSize<LANES>,
+        idxs: crate::SimdUsize<LANES>,
+        or: Self,
+    ) -> Self {
+        let mask = (mask & idxs.lanes_lt(crate::SimdUsize::splat(slice.len()))).to_int();
+        let base_ptr = crate::vector::ptr::SimdConstPtr::splat(slice.as_ptr());
+        // Ferris forgive me, I have done pointer arithmetic here.
+        let ptrs = base_ptr.wrapping_add(idxs);
+        // SAFETY: The ptrs have been bounds-masked to prevent memory-unsafe reads insha'allah
+        unsafe { crate::intrinsics::simd_gather(or, ptrs, mask) }
+    }
+
+    /// SIMD scatter: write a SIMD vector's values into a slice, using potentially discontiguous indices.
+    /// Out-of-bounds indices are not written.
+    /// `scatter` writes "in order", so if an index receives two writes, only the last is guaranteed.
+    /// ```
+    /// # #![feature(portable_simd)]
+    /// # use core_simd::*;
+    /// let mut vec: Vec<i32> = vec![10, 11, 12, 13, 14, 15, 16, 17, 18];
+    /// let idxs = SimdUsize::<4>::from_array([9, 3, 0, 0]);
+    /// let vals = SimdI32::from_array([-27, 82, -41, 124]);
+    ///
+    /// vals.scatter(&mut vec, idxs); // index 0 receives two writes.
+    /// assert_eq!(vec, vec![124, 11, 12, 82, 14, 15, 16, 17, 18]);
+    /// ```
+    #[inline]
+    pub fn scatter(self, slice: &mut [Element], idxs: crate::SimdUsize<LANES>) {
+        self.scatter_select(slice, crate::MaskSize::splat(true), idxs)
+    }
+
+    /// SIMD scatter: write a SIMD vector's values into a slice, using potentially discontiguous indices.
+    /// Out-of-bounds or masked indices are not written.
+    /// `scatter_select` writes "in order", so if an index receives two writes, only the last is guaranteed.
+    /// ```
+    /// # #![feature(portable_simd)]
+    /// # use core_simd::*;
+    /// let mut vec: Vec<i32> = vec![10, 11, 12, 13, 14, 15, 16, 17, 18];
+    /// let idxs = SimdUsize::<4>::from_array([9, 3, 0, 0]);
+    /// let vals = SimdI32::from_array([-27, 82, -41, 124]);
+    /// let mask = MaskSize::from_array([true, true, true, false]); // Note the mask of the last lane.
+    ///
+    /// vals.scatter_select(&mut vec, mask, idxs); // index 0's second write is masked, thus omitted.
+    /// assert_eq!(vec, vec![-41, 11, 12, 82, 14, 15, 16, 17, 18]);
+    /// ```
+    #[inline]
+    pub fn scatter_select(
+        self,
+        slice: &mut [Element],
+        mask: crate::MaskSize<LANES>,
+        idxs: crate::SimdUsize<LANES>,
+    ) {
+        // We must construct our scatter mask before we derive a pointer!
+        let mask = (mask & idxs.lanes_lt(crate::SimdUsize::splat(slice.len()))).to_int();
+        // SAFETY: This block works with *mut T derived from &mut 'a [T],
+        // which means it is delicate in Rust's borrowing model, circa 2021:
+        // &mut 'a [T] asserts uniqueness, so deriving &'a [T] invalidates live *mut Ts!
+        // Even though this block is largely safe methods, it must be almost exactly this way
+        // to prevent invalidating the raw ptrs while they're live.
+        // Thus, entering this block requires all values to use being already ready:
+        // 0. idxs we want to write to, which are used to construct the mask.
+        // 1. mask, which depends on an initial &'a [T] and the idxs.
+        // 2. actual values to scatter (self).
+        // 3. &mut [T] which will become our base ptr.
+        unsafe {
+            // Now Entering ☢️ *mut T Zone
+            let base_ptr = crate::vector::ptr::SimdMutPtr::splat(slice.as_mut_ptr());
+            // Ferris forgive me, I have done pointer arithmetic here.
+            let ptrs = base_ptr.wrapping_add(idxs);
+            // The ptrs have been bounds-masked to prevent memory-unsafe writes insha'allah
+            crate::intrinsics::simd_scatter(self, ptrs, mask)
+            // Cleared ☢️ *mut T Zone
+        }
+    }
+}
+
+impl<Element, const LANES: usize> Copy for Simd<Element, LANES>
+where
+    Element: SimdElement,
+    LaneCount<LANES>: SupportedLaneCount,
+{
+}
+
+impl<Element, const LANES: usize> Clone for Simd<Element, LANES>
+where
+    Element: SimdElement,
+    LaneCount<LANES>: SupportedLaneCount,
+{
+    fn clone(&self) -> Self {
+        *self
+    }
+}
+
+impl<Element, const LANES: usize> Default for Simd<Element, LANES>
+where
+    LaneCount<LANES>: SupportedLaneCount,
+    Element: SimdElement + Default,
+{
+    #[inline]
+    fn default() -> Self {
+        Self::splat(Element::default())
+    }
+}
+
+impl<Element, const LANES: usize> PartialEq for Simd<Element, LANES>
+where
+    LaneCount<LANES>: SupportedLaneCount,
+    Element: SimdElement + PartialEq,
+{
+    #[inline]
+    fn eq(&self, other: &Self) -> bool {
+        // TODO use SIMD equality
+        self.to_array() == other.to_array()
+    }
+}
+
+impl<Element, const LANES: usize> PartialOrd for Simd<Element, LANES>
+where
+    LaneCount<LANES>: SupportedLaneCount,
+    Element: SimdElement + PartialOrd,
+{
+    #[inline]
+    fn partial_cmp(&self, other: &Self) -> Option<core::cmp::Ordering> {
+        // TODO use SIMD equality
+        self.to_array().partial_cmp(other.as_ref())
+    }
+}
+
+impl<Element, const LANES: usize> Eq for Simd<Element, LANES>
+where
+    LaneCount<LANES>: SupportedLaneCount,
+    Element: SimdElement + Eq,
+{
+}
+
+impl<Element, const LANES: usize> Ord for Simd<Element, LANES>
+where
+    LaneCount<LANES>: SupportedLaneCount,
+    Element: SimdElement + Ord,
+{
+    #[inline]
+    fn cmp(&self, other: &Self) -> core::cmp::Ordering {
+        // TODO use SIMD equality
+        self.to_array().cmp(other.as_ref())
+    }
+}
+
+impl<Element, const LANES: usize> core::hash::Hash for Simd<Element, LANES>
+where
+    LaneCount<LANES>: SupportedLaneCount,
+    Element: SimdElement + core::hash::Hash,
+{
+    #[inline]
+    fn hash<H>(&self, state: &mut H)
+    where
+        H: core::hash::Hasher,
+    {
+        self.as_array().hash(state)
+    }
+}
+
+// array references
+impl<Element, const LANES: usize> AsRef<[Element; LANES]> for Simd<Element, LANES>
+where
+    LaneCount<LANES>: SupportedLaneCount,
+    Element: SimdElement,
+{
+    #[inline]
+    fn as_ref(&self) -> &[Element; LANES] {
+        &self.0
+    }
+}
+
+impl<Element, const LANES: usize> AsMut<[Element; LANES]> for Simd<Element, LANES>
+where
+    LaneCount<LANES>: SupportedLaneCount,
+    Element: SimdElement,
+{
+    #[inline]
+    fn as_mut(&mut self) -> &mut [Element; LANES] {
+        &mut self.0
+    }
+}
+
+// slice references
+impl<Element, const LANES: usize> AsRef<[Element]> for Simd<Element, LANES>
+where
+    LaneCount<LANES>: SupportedLaneCount,
+    Element: SimdElement,
+{
+    #[inline]
+    fn as_ref(&self) -> &[Element] {
+        &self.0
+    }
+}
+
+impl<Element, const LANES: usize> AsMut<[Element]> for Simd<Element, LANES>
+where
+    LaneCount<LANES>: SupportedLaneCount,
+    Element: SimdElement,
+{
+    #[inline]
+    fn as_mut(&mut self) -> &mut [Element] {
+        &mut self.0
+    }
+}
+
+// vector/array conversion
+impl<Element, const LANES: usize> From<[Element; LANES]> for Simd<Element, LANES>
+where
+    LaneCount<LANES>: SupportedLaneCount,
+    Element: SimdElement,
+{
+    fn from(array: [Element; LANES]) -> Self {
+        Self(array)
+    }
+}
+
+impl<Element, const LANES: usize> From<Simd<Element, LANES>> for [Element; LANES]
+where
+    LaneCount<LANES>: SupportedLaneCount,
+    Element: SimdElement,
+{
+    fn from(vector: Simd<Element, LANES>) -> Self {
+        vector.to_array()
+    }
+}
+
 mod sealed {
     pub trait Sealed {}
 }
 use sealed::Sealed;
 
 /// Marker trait for types that may be used as SIMD vector elements.
-pub unsafe trait SimdElement: Sealed {
+pub unsafe trait SimdElement: Sealed + Copy {
     /// The mask element type corresponding to this element type.
     type Mask: SimdElement;
 }
@@ -106,3 +415,24 @@ pub trait Vector: sealed::Sealed {
     #[must_use]
     fn splat(val: Self::Scalar) -> Self;
 }
+
+impl<Element, const LANES: usize> Sealed for Simd<Element, LANES>
+where
+    LaneCount<LANES>: SupportedLaneCount,
+    Element: SimdElement,
+{
+}
+
+impl<Element, const LANES: usize> Vector for Simd<Element, LANES>
+where
+    LaneCount<LANES>: SupportedLaneCount,
+    Element: SimdElement,
+{
+    type Scalar = Element;
+    const LANES: usize = LANES;
+
+    #[inline]
+    fn splat(val: Self::Scalar) -> Self {
+        Self::splat(val)
+    }
+}
diff --git a/crates/core_simd/src/vector/float.rs b/crates/core_simd/src/vector/float.rs
index 231fe590ada5..40959d66872c 100644
--- a/crates/core_simd/src/vector/float.rs
+++ b/crates/core_simd/src/vector/float.rs
@@ -7,7 +7,6 @@ use crate::{LaneCount, SupportedLaneCount};
 /// representation. Called from `define_float_vector!`.
 macro_rules! impl_float_vector {
     { $name:ident, $type:ident, $bits_ty:ident, $mask_ty:ident, $mask_impl_ty:ident } => {
-        impl_vector! { $name, $type }
         impl_float_reductions! { $name, $type }
 
         impl<const LANES: usize> $name<LANES>
diff --git a/crates/core_simd/src/vector/int.rs b/crates/core_simd/src/vector/int.rs
index 88a17daa2f4b..74c4a0f2fb63 100644
--- a/crates/core_simd/src/vector/int.rs
+++ b/crates/core_simd/src/vector/int.rs
@@ -5,32 +5,8 @@ use crate::{LaneCount, SupportedLaneCount};
 /// Implements additional integer traits (Eq, Ord, Hash) on the specified vector `$name`, holding multiple `$lanes` of `$type`.
 macro_rules! impl_integer_vector {
     { $name:ident, $type:ty, $mask_ty:ident, $mask_impl_ty:ident } => {
-        impl_vector! { $name, $type }
         impl_integer_reductions! { $name, $type }
 
-        impl<const LANES: usize> Eq for $name<LANES> where LaneCount<LANES>: SupportedLaneCount {}
-
-        impl<const LANES: usize> Ord for $name<LANES> where LaneCount<LANES>: SupportedLaneCount {
-            #[inline]
-            fn cmp(&self, other: &Self) -> core::cmp::Ordering {
-                // TODO use SIMD cmp
-                self.as_array().cmp(other.as_ref())
-            }
-        }
-
-        impl<const LANES: usize> core::hash::Hash for $name<LANES>
-        where
-            LaneCount<LANES>: SupportedLaneCount,
-        {
-            #[inline]
-            fn hash<H>(&self, state: &mut H)
-            where
-                H: core::hash::Hasher
-            {
-                self.as_array().hash(state)
-            }
-        }
-
         impl<const LANES: usize> $name<LANES>
         where
             LaneCount<LANES>: SupportedLaneCount,
diff --git a/crates/core_simd/src/vector/uint.rs b/crates/core_simd/src/vector/uint.rs
index 5bd1a7fd67fd..6dfcbe795939 100644
--- a/crates/core_simd/src/vector/uint.rs
+++ b/crates/core_simd/src/vector/uint.rs
@@ -1,35 +1,9 @@
 #![allow(non_camel_case_types)]
 
-use crate::{LaneCount, SupportedLaneCount};
-
 /// Implements additional integer traits (Eq, Ord, Hash) on the specified vector `$name`, holding multiple `$lanes` of `$type`.
 macro_rules! impl_unsigned_vector {
     { $name:ident, $type:ty } => {
-        impl_vector! { $name, $type }
         impl_integer_reductions! { $name, $type }
-
-        impl<const LANES: usize> Eq for $name<LANES> where LaneCount<LANES>: SupportedLaneCount {}
-
-        impl<const LANES: usize> Ord for $name<LANES> where LaneCount<LANES>: SupportedLaneCount {
-            #[inline]
-            fn cmp(&self, other: &Self) -> core::cmp::Ordering {
-                // TODO use SIMD cmp
-                self.as_array().cmp(other.as_ref())
-            }
-        }
-
-        impl<const LANES: usize> core::hash::Hash for $name<LANES>
-        where
-            LaneCount<LANES>: SupportedLaneCount,
-        {
-            #[inline]
-            fn hash<H>(&self, state: &mut H)
-            where
-                H: core::hash::Hasher
-            {
-                self.as_array().hash(state)
-            }
-        }
     }
 }
 
diff --git a/crates/core_simd/src/vector/vector_impl.rs b/crates/core_simd/src/vector/vector_impl.rs
deleted file mode 100644
index 58ea244adfcb..000000000000
--- a/crates/core_simd/src/vector/vector_impl.rs
+++ /dev/null
@@ -1,257 +0,0 @@
-/// Implements common traits on the specified vector `$name`, holding multiple `$lanes` of `$type`.
-macro_rules! impl_vector {
-    { $name:ident, $type:ty } => {
-        impl<const LANES: usize> crate::vector::sealed::Sealed for $name<LANES>
-        where
-            crate::LaneCount<LANES>: crate::SupportedLaneCount,
-        {}
-
-        impl<const LANES: usize> crate::vector::Vector for $name<LANES>
-        where
-            crate::LaneCount<LANES>: crate::SupportedLaneCount,
-        {
-            type Scalar = $type;
-            const LANES: usize = LANES;
-
-            #[inline]
-            fn splat(val: Self::Scalar) -> Self {
-                Self::splat(val)
-            }
-        }
-
-        impl<const LANES: usize> $name<LANES>
-        where
-            crate::LaneCount<LANES>: crate::SupportedLaneCount,
-        {
-            /// Construct a SIMD vector by setting all lanes to the given value.
-            pub const fn splat(value: $type) -> Self {
-                Self([value; LANES])
-            }
-
-            /// Returns an array reference containing the entire SIMD vector.
-            pub const fn as_array(&self) -> &[$type; LANES] {
-                &self.0
-            }
-
-            /// Returns a mutable array reference containing the entire SIMD vector.
-            pub fn as_mut_array(&mut self) -> &mut [$type; LANES] {
-                &mut self.0
-            }
-
-            /// Converts an array to a SIMD vector.
-            pub const fn from_array(array: [$type; LANES]) -> Self {
-                Self(array)
-            }
-
-            /// Converts a SIMD vector to an array.
-            pub const fn to_array(self) -> [$type; LANES] {
-                self.0
-            }
-
-            /// SIMD gather: construct a SIMD vector by reading from a slice, using potentially discontiguous indices.
-            /// If an index is out of bounds, that lane instead selects the value from the "or" vector.
-            /// ```
-            /// # #![feature(portable_simd)]
-            /// # use core_simd::*;
-            /// let vec: Vec<i32> = vec![10, 11, 12, 13, 14, 15, 16, 17, 18];
-            /// let idxs = SimdUsize::<4>::from_array([9, 3, 0, 5]);
-            /// let alt = SimdI32::from_array([-5, -4, -3, -2]);
-            ///
-            /// let result = SimdI32::<4>::gather_or(&vec, idxs, alt); // Note the lane that is out-of-bounds.
-            /// assert_eq!(result, SimdI32::from_array([-5, 13, 10, 15]));
-            /// ```
-            #[must_use]
-            #[inline]
-            pub fn gather_or(slice: &[$type], idxs: crate::SimdUsize<LANES>, or: Self) -> Self {
-                Self::gather_select(slice, crate::MaskSize::splat(true), idxs, or)
-            }
-
-            /// SIMD gather: construct a SIMD vector by reading from a slice, using potentially discontiguous indices.
-            /// Out-of-bounds indices instead use the default value for that lane (0).
-            /// ```
-            /// # #![feature(portable_simd)]
-            /// # use core_simd::*;
-            /// let vec: Vec<i32> = vec![10, 11, 12, 13, 14, 15, 16, 17, 18];
-            /// let idxs = SimdUsize::<4>::from_array([9, 3, 0, 5]);
-            ///
-            /// let result = SimdI32::<4>::gather_or_default(&vec, idxs); // Note the lane that is out-of-bounds.
-            /// assert_eq!(result, SimdI32::from_array([0, 13, 10, 15]));
-            /// ```
-            #[must_use]
-            #[inline]
-            pub fn gather_or_default(slice: &[$type], idxs: crate::SimdUsize<LANES>) -> Self {
-                Self::gather_or(slice, idxs, Self::splat(<$type>::default()))
-            }
-
-            /// SIMD gather: construct a SIMD vector by reading from a slice, using potentially discontiguous indices.
-            /// Out-of-bounds or masked indices instead select the value from the "or" vector.
-            /// ```
-            /// # #![feature(portable_simd)]
-            /// # use core_simd::*;
-            /// let vec: Vec<i32> = vec![10, 11, 12, 13, 14, 15, 16, 17, 18];
-            /// let idxs = SimdUsize::<4>::from_array([9, 3, 0, 5]);
-            /// let alt = SimdI32::from_array([-5, -4, -3, -2]);
-            /// let mask = MaskSize::from_array([true, true, true, false]); // Note the mask of the last lane.
-            ///
-            /// let result = SimdI32::<4>::gather_select(&vec, mask, idxs, alt); // Note the lane that is out-of-bounds.
-            /// assert_eq!(result, SimdI32::from_array([-5, 13, 10, -2]));
-            /// ```
-            #[must_use]
-            #[inline]
-            pub fn gather_select(
-                slice: &[$type],
-                mask: crate::MaskSize<LANES>,
-                idxs: crate::SimdUsize<LANES>,
-                or: Self,
-            ) -> Self
-            {
-                let mask = (mask & idxs.lanes_lt(crate::SimdUsize::splat(slice.len()))).to_int();
-                let base_ptr = crate::vector::ptr::SimdConstPtr::splat(slice.as_ptr());
-                // Ferris forgive me, I have done pointer arithmetic here.
-                let ptrs = base_ptr.wrapping_add(idxs);
-                // SAFETY: The ptrs have been bounds-masked to prevent memory-unsafe reads insha'allah
-                unsafe { crate::intrinsics::simd_gather(or, ptrs, mask) }
-            }
-
-            /// SIMD scatter: write a SIMD vector's values into a slice, using potentially discontiguous indices.
-            /// Out-of-bounds indices are not written.
-            /// `scatter` writes "in order", so if an index receives two writes, only the last is guaranteed.
-            /// ```
-            /// # #![feature(portable_simd)]
-            /// # use core_simd::*;
-            /// let mut vec: Vec<i32> = vec![10, 11, 12, 13, 14, 15, 16, 17, 18];
-            /// let idxs = SimdUsize::<4>::from_array([9, 3, 0, 0]);
-            /// let vals = SimdI32::from_array([-27, 82, -41, 124]);
-            ///
-            /// vals.scatter(&mut vec, idxs); // index 0 receives two writes.
-            /// assert_eq!(vec, vec![124, 11, 12, 82, 14, 15, 16, 17, 18]);
-            /// ```
-            #[inline]
-            pub fn scatter(self, slice: &mut [$type], idxs: crate::SimdUsize<LANES>) {
-                self.scatter_select(slice, crate::MaskSize::splat(true), idxs)
-            }
-
-            /// SIMD scatter: write a SIMD vector's values into a slice, using potentially discontiguous indices.
-            /// Out-of-bounds or masked indices are not written.
-            /// `scatter_select` writes "in order", so if an index receives two writes, only the last is guaranteed.
-            /// ```
-            /// # #![feature(portable_simd)]
-            /// # use core_simd::*;
-            /// let mut vec: Vec<i32> = vec![10, 11, 12, 13, 14, 15, 16, 17, 18];
-            /// let idxs = SimdUsize::<4>::from_array([9, 3, 0, 0]);
-            /// let vals = SimdI32::from_array([-27, 82, -41, 124]);
-            /// let mask = MaskSize::from_array([true, true, true, false]); // Note the mask of the last lane.
-            ///
-            /// vals.scatter_select(&mut vec, mask, idxs); // index 0's second write is masked, thus omitted.
-            /// assert_eq!(vec, vec![-41, 11, 12, 82, 14, 15, 16, 17, 18]);
-            /// ```
-            #[inline]
-            pub fn scatter_select(
-                self,
-                slice: &mut [$type],
-                mask: crate::MaskSize<LANES>,
-                idxs: crate::SimdUsize<LANES>,
-            )
-            {
-                // We must construct our scatter mask before we derive a pointer!
-                let mask = (mask & idxs.lanes_lt(crate::SimdUsize::splat(slice.len()))).to_int();
-                // SAFETY: This block works with *mut T derived from &mut 'a [T],
-                // which means it is delicate in Rust's borrowing model, circa 2021:
-                // &mut 'a [T] asserts uniqueness, so deriving &'a [T] invalidates live *mut Ts!
-                // Even though this block is largely safe methods, it must be almost exactly this way
-                // to prevent invalidating the raw ptrs while they're live.
-                // Thus, entering this block requires all values to use being already ready:
-                // 0. idxs we want to write to, which are used to construct the mask.
-                // 1. mask, which depends on an initial &'a [T] and the idxs.
-                // 2. actual values to scatter (self).
-                // 3. &mut [T] which will become our base ptr.
-                unsafe {
-                    // Now Entering ☢️ *mut T Zone
-                    let base_ptr = crate::vector::ptr::SimdMutPtr::splat(slice.as_mut_ptr());
-                    // Ferris forgive me, I have done pointer arithmetic here.
-                    let ptrs = base_ptr.wrapping_add(idxs);
-                    // The ptrs have been bounds-masked to prevent memory-unsafe writes insha'allah
-                    crate::intrinsics::simd_scatter(self, ptrs, mask)
-                    // Cleared ☢️ *mut T Zone
-                }
-            }
-        }
-
-        impl<const LANES: usize> Copy for $name<LANES> where crate::LaneCount<LANES>: crate::SupportedLaneCount {}
-
-        impl<const LANES: usize> Clone for $name<LANES> where crate::LaneCount<LANES>: crate::SupportedLaneCount {
-            #[inline]
-            fn clone(&self) -> Self {
-                *self
-            }
-        }
-
-        impl<const LANES: usize> Default for $name<LANES> where crate::LaneCount<LANES>: crate::SupportedLaneCount {
-            #[inline]
-            fn default() -> Self {
-                Self::splat(<$type>::default())
-            }
-        }
-
-        impl<const LANES: usize> PartialEq for $name<LANES> where crate::LaneCount<LANES>: crate::SupportedLaneCount {
-            #[inline]
-            fn eq(&self, other: &Self) -> bool {
-                // TODO use SIMD equality
-                self.to_array() == other.to_array()
-            }
-        }
-
-        impl<const LANES: usize> PartialOrd for $name<LANES> where crate::LaneCount<LANES>: crate::SupportedLaneCount {
-            #[inline]
-            fn partial_cmp(&self, other: &Self) -> Option<core::cmp::Ordering> {
-                // TODO use SIMD equalitya
-                self.to_array().partial_cmp(other.as_ref())
-            }
-        }
-
-        // array references
-        impl<const LANES: usize> AsRef<[$type; LANES]> for $name<LANES> where crate::LaneCount<LANES>: crate::SupportedLaneCount {
-            #[inline]
-            fn as_ref(&self) -> &[$type; LANES] {
-                &self.0
-            }
-        }
-
-        impl<const LANES: usize> AsMut<[$type; LANES]> for $name<LANES> where crate::LaneCount<LANES>: crate::SupportedLaneCount {
-            #[inline]
-            fn as_mut(&mut self) -> &mut [$type; LANES] {
-                &mut self.0
-            }
-        }
-
-        // slice references
-        impl<const LANES: usize> AsRef<[$type]> for $name<LANES> where crate::LaneCount<LANES>: crate::SupportedLaneCount {
-            #[inline]
-            fn as_ref(&self) -> &[$type] {
-                &self.0
-            }
-        }
-
-        impl<const LANES: usize> AsMut<[$type]> for $name<LANES> where crate::LaneCount<LANES>: crate::SupportedLaneCount {
-            #[inline]
-            fn as_mut(&mut self) -> &mut [$type] {
-                &mut self.0
-            }
-        }
-
-        // vector/array conversion
-        impl<const LANES: usize> From<[$type; LANES]> for $name<LANES> where crate::LaneCount<LANES>: crate::SupportedLaneCount {
-            fn from(array: [$type; LANES]) -> Self {
-                Self(array)
-            }
-        }
-
-        impl <const LANES: usize> From<$name<LANES>> for [$type; LANES] where crate::LaneCount<LANES>: crate::SupportedLaneCount {
-            fn from(vector: $name<LANES>) -> Self {
-                vector.to_array()
-            }
-        }
-
-        impl_shuffle_2pow_lanes!{ $name }
-    }
-}

From 8cc38ae292b1dbc27713cd8e267fa9b4d32c12b9 Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Sat, 7 Aug 2021 01:16:23 +0000
Subject: [PATCH 206/249] Remove Vector trait

---
 crates/core_simd/src/vector.rs | 36 ----------------------------------
 1 file changed, 36 deletions(-)

diff --git a/crates/core_simd/src/vector.rs b/crates/core_simd/src/vector.rs
index 2e3855bff3cf..9fb6d17d40af 100644
--- a/crates/core_simd/src/vector.rs
+++ b/crates/core_simd/src/vector.rs
@@ -400,39 +400,3 @@ impl Sealed for f64 {}
 unsafe impl SimdElement for f64 {
     type Mask = i64;
 }
-
-/// A representation of a vector as an "array" with indices, implementing
-/// operations applicable to any vector type based solely on "having lanes",
-/// and describing relationships between vector and scalar types.
-pub trait Vector: sealed::Sealed {
-    /// The scalar type in every lane of this vector type.
-    type Scalar: Copy + Sized;
-
-    /// The number of lanes for this vector.
-    const LANES: usize;
-
-    /// Generates a SIMD vector with the same value in every lane.
-    #[must_use]
-    fn splat(val: Self::Scalar) -> Self;
-}
-
-impl<Element, const LANES: usize> Sealed for Simd<Element, LANES>
-where
-    LaneCount<LANES>: SupportedLaneCount,
-    Element: SimdElement,
-{
-}
-
-impl<Element, const LANES: usize> Vector for Simd<Element, LANES>
-where
-    LaneCount<LANES>: SupportedLaneCount,
-    Element: SimdElement,
-{
-    type Scalar = Element;
-    const LANES: usize = LANES;
-
-    #[inline]
-    fn splat(val: Self::Scalar) -> Self {
-        Self::splat(val)
-    }
-}

From ddc67e3bf22fe1e79604e5fa0ee2836c87e20b20 Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Sat, 7 Aug 2021 01:17:27 +0000
Subject: [PATCH 207/249] Remove Mask trait

---
 crates/core_simd/src/masks.rs | 31 -------------------------------
 1 file changed, 31 deletions(-)

diff --git a/crates/core_simd/src/masks.rs b/crates/core_simd/src/masks.rs
index ba7da704f61d..3bb444062ab8 100644
--- a/crates/core_simd/src/masks.rs
+++ b/crates/core_simd/src/masks.rs
@@ -14,20 +14,6 @@ mod mask_impl;
 
 use crate::{SimdI16, SimdI32, SimdI64, SimdI8, SimdIsize};
 
-mod sealed {
-    pub trait Sealed {}
-}
-
-/// Helper trait for mask types.
-pub trait Mask: sealed::Sealed {
-    /// The number of lanes for this mask.
-    const LANES: usize;
-
-    /// Generates a mask with the same value in every lane.
-    #[must_use]
-    fn splat(val: bool) -> Self;
-}
-
 macro_rules! define_opaque_mask {
     {
         $(#[$attr:meta])*
@@ -40,23 +26,6 @@ macro_rules! define_opaque_mask {
         where
             crate::LaneCount<LANES>: crate::SupportedLaneCount;
 
-        impl<const LANES: usize> sealed::Sealed for $name<LANES>
-        where
-            crate::LaneCount<LANES>: crate::SupportedLaneCount,
-        {}
-
-        impl<const LANES: usize> Mask for $name<LANES>
-        where
-            crate::LaneCount<LANES>: crate::SupportedLaneCount,
-        {
-            const LANES: usize = LANES;
-
-            #[inline]
-            fn splat(value: bool) -> Self {
-                Self::splat(value)
-            }
-        }
-
         impl_opaque_mask_reductions! { $name, $bits_ty }
 
         impl<const LANES: usize> $name<LANES>

From de13b20b2706607c9c130e16d5fb2b83241a94c1 Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Sat, 7 Aug 2021 04:30:24 +0000
Subject: [PATCH 208/249] Convert all masks to a single type

---
 crates/core_simd/src/masks.rs            | 837 ++++++++++++-----------
 crates/core_simd/src/masks/bitmask.rs    | 124 ++--
 crates/core_simd/src/masks/full_masks.rs | 447 ++++++------
 crates/core_simd/src/ops.rs              |  52 +-
 crates/core_simd/src/reduction.rs        |  28 +-
 5 files changed, 730 insertions(+), 758 deletions(-)

diff --git a/crates/core_simd/src/masks.rs b/crates/core_simd/src/masks.rs
index 3bb444062ab8..d4a0cff4e238 100644
--- a/crates/core_simd/src/masks.rs
+++ b/crates/core_simd/src/masks.rs
@@ -12,420 +12,461 @@
 )]
 mod mask_impl;
 
-use crate::{SimdI16, SimdI32, SimdI64, SimdI8, SimdIsize};
+use crate::{LaneCount, Simd, SimdElement, SupportedLaneCount};
 
-macro_rules! define_opaque_mask {
-    {
-        $(#[$attr:meta])*
-        struct $name:ident<const $lanes:ident: usize>($inner_ty:ty);
-        @bits $bits_ty:ident
-    } => {
-        $(#[$attr])*
-        #[allow(non_camel_case_types)]
-        pub struct $name<const LANES: usize>($inner_ty)
-        where
-            crate::LaneCount<LANES>: crate::SupportedLaneCount;
+/// Marker trait for types that may be used as SIMD mask elements.
+pub unsafe trait MaskElement: SimdElement {
+    #[doc(hidden)]
+    fn valid<const LANES: usize>(values: Simd<Self, LANES>) -> bool
+    where
+        LaneCount<LANES>: SupportedLaneCount;
 
-        impl_opaque_mask_reductions! { $name, $bits_ty }
+    #[doc(hidden)]
+    fn eq(self, other: Self) -> bool;
 
-        impl<const LANES: usize> $name<LANES>
-        where
-            crate::LaneCount<LANES>: crate::SupportedLaneCount,
-        {
-            /// Construct a mask by setting all lanes to the given value.
-            pub fn splat(value: bool) -> Self {
-                Self(<$inner_ty>::splat(value))
-            }
+    #[doc(hidden)]
+    const TRUE: Self;
 
-            /// Converts an array to a SIMD vector.
-            pub fn from_array(array: [bool; LANES]) -> Self {
-                let mut vector = Self::splat(false);
-                let mut i = 0;
-                while i < $lanes {
-                    vector.set(i, array[i]);
-                    i += 1;
-                }
-                vector
-            }
-
-            /// Converts a SIMD vector to an array.
-            pub fn to_array(self) -> [bool; LANES] {
-                let mut array = [false; LANES];
-                let mut i = 0;
-                while i < $lanes {
-                    array[i] = self.test(i);
-                    i += 1;
-                }
-                array
-            }
-
-            /// Converts a vector of integers to a mask, where 0 represents `false` and -1
-            /// represents `true`.
-            ///
-            /// # Safety
-            /// All lanes must be either 0 or -1.
-            #[inline]
-            pub unsafe fn from_int_unchecked(value: $bits_ty<LANES>) -> Self {
-                Self(<$inner_ty>::from_int_unchecked(value))
-            }
-
-            /// Converts a vector of integers to a mask, where 0 represents `false` and -1
-            /// represents `true`.
-            ///
-            /// # Panics
-            /// Panics if any lane is not 0 or -1.
-            #[inline]
-            pub fn from_int(value: $bits_ty<LANES>) -> Self {
-                assert!(
-                    (value.lanes_eq($bits_ty::splat(0)) | value.lanes_eq($bits_ty::splat(-1))).all(),
-                    "all values must be either 0 or -1",
-                );
-                unsafe { Self::from_int_unchecked(value) }
-            }
-
-            /// Converts the mask to a vector of integers, where 0 represents `false` and -1
-            /// represents `true`.
-            #[inline]
-            pub fn to_int(self) -> $bits_ty<LANES> {
-                self.0.to_int()
-            }
-
-            /// Tests the value of the specified lane.
-            ///
-            /// # Safety
-            /// `lane` must be less than `LANES`.
-            #[inline]
-            pub unsafe fn test_unchecked(&self, lane: usize) -> bool {
-                self.0.test_unchecked(lane)
-            }
-
-            /// Tests the value of the specified lane.
-            ///
-            /// # Panics
-            /// Panics if `lane` is greater than or equal to the number of lanes in the vector.
-            #[inline]
-            pub fn test(&self, lane: usize) -> bool {
-                assert!(lane < LANES, "lane index out of range");
-                unsafe { self.test_unchecked(lane) }
-            }
-
-            /// Sets the value of the specified lane.
-            ///
-            /// # Safety
-            /// `lane` must be less than `LANES`.
-            #[inline]
-            pub unsafe fn set_unchecked(&mut self, lane: usize, value: bool) {
-                self.0.set_unchecked(lane, value);
-            }
-
-            /// Sets the value of the specified lane.
-            ///
-            /// # Panics
-            /// Panics if `lane` is greater than or equal to the number of lanes in the vector.
-            #[inline]
-            pub fn set(&mut self, lane: usize, value: bool) {
-                assert!(lane < LANES, "lane index out of range");
-                unsafe { self.set_unchecked(lane, value); }
-            }
-
-            /// Convert this mask to a bitmask, with one bit set per lane.
-            pub fn to_bitmask(self) -> [u8; crate::LaneCount::<LANES>::BITMASK_LEN] {
-                self.0.to_bitmask()
-            }
-
-            /// Convert a bitmask to a mask.
-            pub fn from_bitmask(bitmask: [u8; crate::LaneCount::<LANES>::BITMASK_LEN]) -> Self {
-                Self(<$inner_ty>::from_bitmask(bitmask))
-            }
-        }
-
-        // vector/array conversion
-        impl<const LANES: usize> From<[bool; LANES]> for $name<LANES>
-        where
-            crate::LaneCount<LANES>: crate::SupportedLaneCount,
-        {
-            fn from(array: [bool; LANES]) -> Self {
-                Self::from_array(array)
-            }
-        }
-
-        impl <const LANES: usize> From<$name<LANES>> for [bool; LANES]
-        where
-            crate::LaneCount<LANES>: crate::SupportedLaneCount,
-        {
-            fn from(vector: $name<LANES>) -> Self {
-                vector.to_array()
-            }
-        }
-
-        impl<const LANES: usize> Copy for $name<LANES>
-        where
-            crate::LaneCount<LANES>: crate::SupportedLaneCount,
-        {}
-
-        impl<const LANES: usize> Clone for $name<LANES>
-        where
-            crate::LaneCount<LANES>: crate::SupportedLaneCount,
-        {
-            #[inline]
-            fn clone(&self) -> Self {
-                *self
-            }
-        }
-
-        impl<const LANES: usize> Default for $name<LANES>
-        where
-            crate::LaneCount<LANES>: crate::SupportedLaneCount,
-        {
-            #[inline]
-            fn default() -> Self {
-                Self::splat(false)
-            }
-        }
-
-        impl<const LANES: usize> PartialEq for $name<LANES>
-        where
-            crate::LaneCount<LANES>: crate::SupportedLaneCount,
-        {
-            #[inline]
-            fn eq(&self, other: &Self) -> bool {
-                self.0 == other.0
-            }
-        }
-
-        impl<const LANES: usize> PartialOrd for $name<LANES>
-        where
-            crate::LaneCount<LANES>: crate::SupportedLaneCount,
-        {
-            #[inline]
-            fn partial_cmp(&self, other: &Self) -> Option<core::cmp::Ordering> {
-                self.0.partial_cmp(&other.0)
-            }
-        }
-
-        impl<const LANES: usize> core::fmt::Debug for $name<LANES>
-        where
-            crate::LaneCount<LANES>: crate::SupportedLaneCount,
-        {
-            fn fmt(&self, f: &mut core::fmt::Formatter) -> core::fmt::Result {
-                f.debug_list()
-                    .entries((0..LANES).map(|lane| self.test(lane)))
-                    .finish()
-            }
-        }
-
-        impl<const LANES: usize> core::ops::BitAnd for $name<LANES>
-        where
-            crate::LaneCount<LANES>: crate::SupportedLaneCount,
-        {
-            type Output = Self;
-            #[inline]
-            fn bitand(self, rhs: Self) -> Self {
-                Self(self.0 & rhs.0)
-            }
-        }
-
-        impl<const LANES: usize> core::ops::BitAnd<bool> for $name<LANES>
-        where
-            crate::LaneCount<LANES>: crate::SupportedLaneCount,
-        {
-            type Output = Self;
-            #[inline]
-            fn bitand(self, rhs: bool) -> Self {
-                self & Self::splat(rhs)
-            }
-        }
-
-        impl<const LANES: usize> core::ops::BitAnd<$name<LANES>> for bool
-        where
-            crate::LaneCount<LANES>: crate::SupportedLaneCount,
-        {
-            type Output = $name<LANES>;
-            #[inline]
-            fn bitand(self, rhs: $name<LANES>) -> $name<LANES> {
-                $name::<LANES>::splat(self) & rhs
-            }
-        }
-
-        impl<const LANES: usize> core::ops::BitOr for $name<LANES>
-        where
-            crate::LaneCount<LANES>: crate::SupportedLaneCount,
-        {
-            type Output = Self;
-            #[inline]
-            fn bitor(self, rhs: Self) -> Self {
-                Self(self.0 | rhs.0)
-            }
-        }
-
-        impl<const LANES: usize> core::ops::BitOr<bool> for $name<LANES>
-        where
-            crate::LaneCount<LANES>: crate::SupportedLaneCount,
-        {
-            type Output = Self;
-            #[inline]
-            fn bitor(self, rhs: bool) -> Self {
-                self | Self::splat(rhs)
-            }
-        }
-
-        impl<const LANES: usize> core::ops::BitOr<$name<LANES>> for bool
-        where
-            crate::LaneCount<LANES>: crate::SupportedLaneCount,
-        {
-            type Output = $name<LANES>;
-            #[inline]
-            fn bitor(self, rhs: $name<LANES>) -> $name<LANES> {
-                $name::<LANES>::splat(self) | rhs
-            }
-        }
-
-        impl<const LANES: usize> core::ops::BitXor for $name<LANES>
-        where
-            crate::LaneCount<LANES>: crate::SupportedLaneCount,
-        {
-            type Output = Self;
-            #[inline]
-            fn bitxor(self, rhs: Self) -> Self::Output {
-                Self(self.0 ^ rhs.0)
-            }
-        }
-
-        impl<const LANES: usize> core::ops::BitXor<bool> for $name<LANES>
-        where
-            crate::LaneCount<LANES>: crate::SupportedLaneCount,
-        {
-            type Output = Self;
-            #[inline]
-            fn bitxor(self, rhs: bool) -> Self::Output {
-                self ^ Self::splat(rhs)
-            }
-        }
-
-        impl<const LANES: usize> core::ops::BitXor<$name<LANES>> for bool
-        where
-            crate::LaneCount<LANES>: crate::SupportedLaneCount,
-        {
-            type Output = $name<LANES>;
-            #[inline]
-            fn bitxor(self, rhs: $name<LANES>) -> Self::Output {
-                $name::<LANES>::splat(self) ^ rhs
-            }
-        }
-
-        impl<const LANES: usize> core::ops::Not for $name<LANES>
-        where
-            crate::LaneCount<LANES>: crate::SupportedLaneCount,
-        {
-            type Output = $name<LANES>;
-            #[inline]
-            fn not(self) -> Self::Output {
-                Self(!self.0)
-            }
-        }
-
-        impl<const LANES: usize> core::ops::BitAndAssign for $name<LANES>
-        where
-            crate::LaneCount<LANES>: crate::SupportedLaneCount,
-        {
-            #[inline]
-            fn bitand_assign(&mut self, rhs: Self) {
-                self.0 = self.0 & rhs.0;
-            }
-        }
-
-        impl<const LANES: usize> core::ops::BitAndAssign<bool> for $name<LANES>
-        where
-            crate::LaneCount<LANES>: crate::SupportedLaneCount,
-        {
-            #[inline]
-            fn bitand_assign(&mut self, rhs: bool) {
-                *self &= Self::splat(rhs);
-            }
-        }
-
-        impl<const LANES: usize> core::ops::BitOrAssign for $name<LANES>
-        where
-            crate::LaneCount<LANES>: crate::SupportedLaneCount,
-        {
-            #[inline]
-            fn bitor_assign(&mut self, rhs: Self) {
-                self.0 = self.0 | rhs.0;
-            }
-        }
-
-        impl<const LANES: usize> core::ops::BitOrAssign<bool> for $name<LANES>
-        where
-            crate::LaneCount<LANES>: crate::SupportedLaneCount,
-        {
-            #[inline]
-            fn bitor_assign(&mut self, rhs: bool) {
-                *self |= Self::splat(rhs);
-            }
-        }
-
-        impl<const LANES: usize> core::ops::BitXorAssign for $name<LANES>
-        where
-            crate::LaneCount<LANES>: crate::SupportedLaneCount,
-        {
-            #[inline]
-            fn bitxor_assign(&mut self, rhs: Self) {
-                self.0 = self.0 ^ rhs.0;
-            }
-        }
-
-        impl<const LANES: usize> core::ops::BitXorAssign<bool> for $name<LANES>
-        where
-            crate::LaneCount<LANES>: crate::SupportedLaneCount,
-        {
-            #[inline]
-            fn bitxor_assign(&mut self, rhs: bool) {
-                *self ^= Self::splat(rhs);
-            }
-        }
-    };
+    #[doc(hidden)]
+    const FALSE: Self;
 }
 
-define_opaque_mask! {
-    /// Mask for vectors with `LANES` 8-bit elements.
+macro_rules! impl_element {
+    { $ty:ty } => {
+        unsafe impl MaskElement for $ty {
+            fn valid<const LANES: usize>(value: Simd<Self, LANES>) -> bool
+            where
+                LaneCount<LANES>: SupportedLaneCount,
+            {
+                (value.lanes_eq(Simd::splat(0)) | value.lanes_eq(Simd::splat(-1))).all()
+            }
+
+            fn eq(self, other: Self) -> bool { self == other }
+
+            const TRUE: Self = -1;
+            const FALSE: Self = 0;
+        }
+    }
+}
+
+impl_element! { i8 }
+impl_element! { i16 }
+impl_element! { i32 }
+impl_element! { i64 }
+impl_element! { isize }
+
+/// A SIMD vector mask for `LANES` elements of width specified by `Element`.
+///
+/// The layout of this type is unspecified.
+#[repr(transparent)]
+pub struct Mask<Element, const LANES: usize>(mask_impl::Mask<Element, LANES>)
+where
+    Element: MaskElement,
+    LaneCount<LANES>: SupportedLaneCount;
+
+impl<Element, const LANES: usize> Copy for Mask<Element, LANES>
+where
+    Element: MaskElement,
+    LaneCount<LANES>: SupportedLaneCount,
+{
+}
+
+impl<Element, const LANES: usize> Clone for Mask<Element, LANES>
+where
+    Element: MaskElement,
+    LaneCount<LANES>: SupportedLaneCount,
+{
+    fn clone(&self) -> Self {
+        *self
+    }
+}
+
+impl<Element, const LANES: usize> Mask<Element, LANES>
+where
+    Element: MaskElement,
+    LaneCount<LANES>: SupportedLaneCount,
+{
+    /// Construct a mask by setting all lanes to the given value.
+    pub fn splat(value: bool) -> Self {
+        Self(mask_impl::Mask::splat(value))
+    }
+
+    /// Converts an array to a SIMD vector.
+    pub fn from_array(array: [bool; LANES]) -> Self {
+        let mut vector = Self::splat(false);
+        for (i, v) in array.iter().enumerate() {
+            vector.set(i, *v);
+        }
+        vector
+    }
+
+    /// Converts a SIMD vector to an array.
+    pub fn to_array(self) -> [bool; LANES] {
+        let mut array = [false; LANES];
+        for (i, v) in array.iter_mut().enumerate() {
+            *v = self.test(i);
+        }
+        array
+    }
+
+    /// Converts a vector of integers to a mask, where 0 represents `false` and -1
+    /// represents `true`.
     ///
-    /// The layout of this type is unspecified.
-    struct Mask8<const LANES: usize>(mask_impl::Mask8<LANES>);
-    @bits SimdI8
+    /// # Safety
+    /// All lanes must be either 0 or -1.
+    #[inline]
+    pub unsafe fn from_int_unchecked(value: Simd<Element, LANES>) -> Self {
+        Self(mask_impl::Mask::from_int_unchecked(value))
+    }
+
+    /// Converts a vector of integers to a mask, where 0 represents `false` and -1
+    /// represents `true`.
+    ///
+    /// # Panics
+    /// Panics if any lane is not 0 or -1.
+    #[inline]
+    pub fn from_int(value: Simd<Element, LANES>) -> Self {
+        assert!(Element::valid(value), "all values must be either 0 or -1",);
+        unsafe { Self::from_int_unchecked(value) }
+    }
+
+    /// Converts the mask to a vector of integers, where 0 represents `false` and -1
+    /// represents `true`.
+    #[inline]
+    pub fn to_int(self) -> Simd<Element, LANES> {
+        self.0.to_int()
+    }
+
+    /// Tests the value of the specified lane.
+    ///
+    /// # Safety
+    /// `lane` must be less than `LANES`.
+    #[inline]
+    pub unsafe fn test_unchecked(&self, lane: usize) -> bool {
+        self.0.test_unchecked(lane)
+    }
+
+    /// Tests the value of the specified lane.
+    ///
+    /// # Panics
+    /// Panics if `lane` is greater than or equal to the number of lanes in the vector.
+    #[inline]
+    pub fn test(&self, lane: usize) -> bool {
+        assert!(lane < LANES, "lane index out of range");
+        unsafe { self.test_unchecked(lane) }
+    }
+
+    /// Sets the value of the specified lane.
+    ///
+    /// # Safety
+    /// `lane` must be less than `LANES`.
+    #[inline]
+    pub unsafe fn set_unchecked(&mut self, lane: usize, value: bool) {
+        self.0.set_unchecked(lane, value);
+    }
+
+    /// Sets the value of the specified lane.
+    ///
+    /// # Panics
+    /// Panics if `lane` is greater than or equal to the number of lanes in the vector.
+    #[inline]
+    pub fn set(&mut self, lane: usize, value: bool) {
+        assert!(lane < LANES, "lane index out of range");
+        unsafe {
+            self.set_unchecked(lane, value);
+        }
+    }
+
+    /// Convert this mask to a bitmask, with one bit set per lane.
+    pub fn to_bitmask(self) -> [u8; LaneCount::<LANES>::BITMASK_LEN] {
+        self.0.to_bitmask()
+    }
+
+    /// Convert a bitmask to a mask.
+    pub fn from_bitmask(bitmask: [u8; LaneCount::<LANES>::BITMASK_LEN]) -> Self {
+        Self(mask_impl::Mask::from_bitmask(bitmask))
+    }
+
+    /// Returns true if any lane is set, or false otherwise.
+    #[inline]
+    pub fn any(self) -> bool {
+        self.0.any()
+    }
+
+    /// Returns true if all lanes are set, or false otherwise.
+    #[inline]
+    pub fn all(self) -> bool {
+        self.0.all()
+    }
 }
 
-define_opaque_mask! {
-    /// Mask for vectors with `LANES` 16-bit elements.
-    ///
-    /// The layout of this type is unspecified.
-    struct Mask16<const LANES: usize>(mask_impl::Mask16<LANES>);
-    @bits SimdI16
+// vector/array conversion
+impl<Element, const LANES: usize> From<[bool; LANES]> for Mask<Element, LANES>
+where
+    Element: MaskElement,
+    LaneCount<LANES>: SupportedLaneCount,
+{
+    fn from(array: [bool; LANES]) -> Self {
+        Self::from_array(array)
+    }
 }
 
-define_opaque_mask! {
-    /// Mask for vectors with `LANES` 32-bit elements.
-    ///
-    /// The layout of this type is unspecified.
-    struct Mask32<const LANES: usize>(mask_impl::Mask32<LANES>);
-    @bits SimdI32
+impl<Element, const LANES: usize> From<Mask<Element, LANES>> for [bool; LANES]
+where
+    Element: MaskElement,
+    LaneCount<LANES>: SupportedLaneCount,
+{
+    fn from(vector: Mask<Element, LANES>) -> Self {
+        vector.to_array()
+    }
 }
 
-define_opaque_mask! {
-    /// Mask for vectors with `LANES` 64-bit elements.
-    ///
-    /// The layout of this type is unspecified.
-    struct Mask64<const LANES: usize>(mask_impl::Mask64<LANES>);
-    @bits SimdI64
+impl<Element, const LANES: usize> Default for Mask<Element, LANES>
+where
+    Element: MaskElement,
+    LaneCount<LANES>: SupportedLaneCount,
+{
+    #[inline]
+    fn default() -> Self {
+        Self::splat(false)
+    }
 }
 
-define_opaque_mask! {
-    /// Mask for vectors with `LANES` pointer-width elements.
-    ///
-    /// The layout of this type is unspecified.
-    struct MaskSize<const LANES: usize>(mask_impl::MaskSize<LANES>);
-    @bits SimdIsize
+impl<Element, const LANES: usize> PartialEq for Mask<Element, LANES>
+where
+    Element: MaskElement + PartialEq,
+    LaneCount<LANES>: SupportedLaneCount,
+{
+    #[inline]
+    fn eq(&self, other: &Self) -> bool {
+        self.0 == other.0
+    }
 }
 
+impl<Element, const LANES: usize> PartialOrd for Mask<Element, LANES>
+where
+    Element: MaskElement + PartialOrd,
+    LaneCount<LANES>: SupportedLaneCount,
+{
+    #[inline]
+    fn partial_cmp(&self, other: &Self) -> Option<core::cmp::Ordering> {
+        self.0.partial_cmp(&other.0)
+    }
+}
+
+impl<Element, const LANES: usize> core::fmt::Debug for Mask<Element, LANES>
+where
+    Element: MaskElement + core::fmt::Debug,
+    LaneCount<LANES>: SupportedLaneCount,
+{
+    fn fmt(&self, f: &mut core::fmt::Formatter) -> core::fmt::Result {
+        f.debug_list()
+            .entries((0..LANES).map(|lane| self.test(lane)))
+            .finish()
+    }
+}
+
+impl<Element, const LANES: usize> core::ops::BitAnd for Mask<Element, LANES>
+where
+    Element: MaskElement,
+    LaneCount<LANES>: SupportedLaneCount,
+{
+    type Output = Self;
+    #[inline]
+    fn bitand(self, rhs: Self) -> Self {
+        Self(self.0 & rhs.0)
+    }
+}
+
+impl<Element, const LANES: usize> core::ops::BitAnd<bool> for Mask<Element, LANES>
+where
+    Element: MaskElement,
+    LaneCount<LANES>: SupportedLaneCount,
+{
+    type Output = Self;
+    #[inline]
+    fn bitand(self, rhs: bool) -> Self {
+        self & Self::splat(rhs)
+    }
+}
+
+impl<Element, const LANES: usize> core::ops::BitAnd<Mask<Element, LANES>> for bool
+where
+    Element: MaskElement,
+    LaneCount<LANES>: SupportedLaneCount,
+{
+    type Output = Mask<Element, LANES>;
+    #[inline]
+    fn bitand(self, rhs: Mask<Element, LANES>) -> Mask<Element, LANES> {
+        Mask::splat(self) & rhs
+    }
+}
+
+impl<Element, const LANES: usize> core::ops::BitOr for Mask<Element, LANES>
+where
+    Element: MaskElement,
+    LaneCount<LANES>: SupportedLaneCount,
+{
+    type Output = Self;
+    #[inline]
+    fn bitor(self, rhs: Self) -> Self {
+        Self(self.0 | rhs.0)
+    }
+}
+
+impl<Element, const LANES: usize> core::ops::BitOr<bool> for Mask<Element, LANES>
+where
+    Element: MaskElement,
+    LaneCount<LANES>: SupportedLaneCount,
+{
+    type Output = Self;
+    #[inline]
+    fn bitor(self, rhs: bool) -> Self {
+        self | Self::splat(rhs)
+    }
+}
+
+impl<Element, const LANES: usize> core::ops::BitOr<Mask<Element, LANES>> for bool
+where
+    Element: MaskElement,
+    LaneCount<LANES>: SupportedLaneCount,
+{
+    type Output = Mask<Element, LANES>;
+    #[inline]
+    fn bitor(self, rhs: Mask<Element, LANES>) -> Mask<Element, LANES> {
+        Mask::splat(self) | rhs
+    }
+}
+
+impl<Element, const LANES: usize> core::ops::BitXor for Mask<Element, LANES>
+where
+    Element: MaskElement,
+    LaneCount<LANES>: SupportedLaneCount,
+{
+    type Output = Self;
+    #[inline]
+    fn bitxor(self, rhs: Self) -> Self::Output {
+        Self(self.0 ^ rhs.0)
+    }
+}
+
+impl<Element, const LANES: usize> core::ops::BitXor<bool> for Mask<Element, LANES>
+where
+    Element: MaskElement,
+    LaneCount<LANES>: SupportedLaneCount,
+{
+    type Output = Self;
+    #[inline]
+    fn bitxor(self, rhs: bool) -> Self::Output {
+        self ^ Self::splat(rhs)
+    }
+}
+
+impl<Element, const LANES: usize> core::ops::BitXor<Mask<Element, LANES>> for bool
+where
+    Element: MaskElement,
+    LaneCount<LANES>: SupportedLaneCount,
+{
+    type Output = Mask<Element, LANES>;
+    #[inline]
+    fn bitxor(self, rhs: Mask<Element, LANES>) -> Self::Output {
+        Mask::splat(self) ^ rhs
+    }
+}
+
+impl<Element, const LANES: usize> core::ops::Not for Mask<Element, LANES>
+where
+    Element: MaskElement,
+    LaneCount<LANES>: SupportedLaneCount,
+{
+    type Output = Mask<Element, LANES>;
+    #[inline]
+    fn not(self) -> Self::Output {
+        Self(!self.0)
+    }
+}
+
+impl<Element, const LANES: usize> core::ops::BitAndAssign for Mask<Element, LANES>
+where
+    Element: MaskElement,
+    LaneCount<LANES>: SupportedLaneCount,
+{
+    #[inline]
+    fn bitand_assign(&mut self, rhs: Self) {
+        self.0 = self.0 & rhs.0;
+    }
+}
+
+impl<Element, const LANES: usize> core::ops::BitAndAssign<bool> for Mask<Element, LANES>
+where
+    Element: MaskElement,
+    LaneCount<LANES>: SupportedLaneCount,
+{
+    #[inline]
+    fn bitand_assign(&mut self, rhs: bool) {
+        *self &= Self::splat(rhs);
+    }
+}
+
+impl<Element, const LANES: usize> core::ops::BitOrAssign for Mask<Element, LANES>
+where
+    Element: MaskElement,
+    LaneCount<LANES>: SupportedLaneCount,
+{
+    #[inline]
+    fn bitor_assign(&mut self, rhs: Self) {
+        self.0 = self.0 | rhs.0;
+    }
+}
+
+impl<Element, const LANES: usize> core::ops::BitOrAssign<bool> for Mask<Element, LANES>
+where
+    Element: MaskElement,
+    LaneCount<LANES>: SupportedLaneCount,
+{
+    #[inline]
+    fn bitor_assign(&mut self, rhs: bool) {
+        *self |= Self::splat(rhs);
+    }
+}
+
+impl<Element, const LANES: usize> core::ops::BitXorAssign for Mask<Element, LANES>
+where
+    Element: MaskElement,
+    LaneCount<LANES>: SupportedLaneCount,
+{
+    #[inline]
+    fn bitxor_assign(&mut self, rhs: Self) {
+        self.0 = self.0 ^ rhs.0;
+    }
+}
+
+impl<Element, const LANES: usize> core::ops::BitXorAssign<bool> for Mask<Element, LANES>
+where
+    Element: MaskElement,
+    LaneCount<LANES>: SupportedLaneCount,
+{
+    #[inline]
+    fn bitxor_assign(&mut self, rhs: bool) {
+        *self ^= Self::splat(rhs);
+    }
+}
+
+/// A SIMD mask of `LANES` 8-bit values.
+pub type Mask8<const LANES: usize> = Mask<i8, LANES>;
+
+/// A SIMD mask of `LANES` 16-bit values.
+pub type Mask16<const LANES: usize> = Mask<i16, LANES>;
+
+/// A SIMD mask of `LANES` 32-bit values.
+pub type Mask32<const LANES: usize> = Mask<i32, LANES>;
+
+/// A SIMD mask of `LANES` 64-bit values.
+pub type Mask64<const LANES: usize> = Mask<i64, LANES>;
+
+/// A SIMD mask of `LANES` pointer-width values.
+pub type MaskSize<const LANES: usize> = Mask<isize, LANES>;
+
 /// Vector of eight 8-bit masks
 pub type mask8x8 = Mask8<8>;
 
@@ -488,7 +529,7 @@ macro_rules! impl_from {
             crate::LaneCount<LANES>: crate::SupportedLaneCount,
         {
             fn from(value: $from<LANES>) -> Self {
-                Self(value.0.into())
+                Self(value.0.convert())
             }
         }
         )*
diff --git a/crates/core_simd/src/masks/bitmask.rs b/crates/core_simd/src/masks/bitmask.rs
index 69edd5235872..2b8309494510 100644
--- a/crates/core_simd/src/masks/bitmask.rs
+++ b/crates/core_simd/src/masks/bitmask.rs
@@ -1,38 +1,26 @@
-use crate::{LaneCount, SupportedLaneCount};
-
-/// Helper trait for limiting int conversion types
-pub trait ConvertToInt {}
-impl<const LANES: usize> ConvertToInt for crate::SimdI8<LANES> where
-    LaneCount<LANES>: SupportedLaneCount
-{
-}
-impl<const LANES: usize> ConvertToInt for crate::SimdI16<LANES> where
-    LaneCount<LANES>: SupportedLaneCount
-{
-}
-impl<const LANES: usize> ConvertToInt for crate::SimdI32<LANES> where
-    LaneCount<LANES>: SupportedLaneCount
-{
-}
-impl<const LANES: usize> ConvertToInt for crate::SimdI64<LANES> where
-    LaneCount<LANES>: SupportedLaneCount
-{
-}
-impl<const LANES: usize> ConvertToInt for crate::SimdIsize<LANES> where
-    LaneCount<LANES>: SupportedLaneCount
-{
-}
+use crate::{LaneCount, MaskElement, Simd, SupportedLaneCount};
+use core::marker::PhantomData;
 
 /// A mask where each lane is represented by a single bit.
 #[repr(transparent)]
-pub struct BitMask<const LANES: usize>(<LaneCount<LANES> as SupportedLaneCount>::BitMask)
+pub struct Mask<Element, const LANES: usize>(
+    <LaneCount<LANES> as SupportedLaneCount>::BitMask,
+    PhantomData<Element>,
+)
 where
+    Element: MaskElement,
     LaneCount<LANES>: SupportedLaneCount;
 
-impl<const LANES: usize> Copy for BitMask<LANES> where LaneCount<LANES>: SupportedLaneCount {}
-
-impl<const LANES: usize> Clone for BitMask<LANES>
+impl<Element, const LANES: usize> Copy for Mask<Element, LANES>
 where
+    Element: MaskElement,
+    LaneCount<LANES>: SupportedLaneCount,
+{
+}
+
+impl<Element, const LANES: usize> Clone for Mask<Element, LANES>
+where
+    Element: MaskElement,
     LaneCount<LANES>: SupportedLaneCount,
 {
     fn clone(&self) -> Self {
@@ -40,8 +28,9 @@ where
     }
 }
 
-impl<const LANES: usize> PartialEq for BitMask<LANES>
+impl<Element, const LANES: usize> PartialEq for Mask<Element, LANES>
 where
+    Element: MaskElement,
     LaneCount<LANES>: SupportedLaneCount,
 {
     fn eq(&self, other: &Self) -> bool {
@@ -49,8 +38,9 @@ where
     }
 }
 
-impl<const LANES: usize> PartialOrd for BitMask<LANES>
+impl<Element, const LANES: usize> PartialOrd for Mask<Element, LANES>
 where
+    Element: MaskElement,
     LaneCount<LANES>: SupportedLaneCount,
 {
     fn partial_cmp(&self, other: &Self) -> Option<core::cmp::Ordering> {
@@ -58,10 +48,16 @@ where
     }
 }
 
-impl<const LANES: usize> Eq for BitMask<LANES> where LaneCount<LANES>: SupportedLaneCount {}
-
-impl<const LANES: usize> Ord for BitMask<LANES>
+impl<Element, const LANES: usize> Eq for Mask<Element, LANES>
 where
+    Element: MaskElement,
+    LaneCount<LANES>: SupportedLaneCount,
+{
+}
+
+impl<Element, const LANES: usize> Ord for Mask<Element, LANES>
+where
+    Element: MaskElement,
     LaneCount<LANES>: SupportedLaneCount,
 {
     fn cmp(&self, other: &Self) -> core::cmp::Ordering {
@@ -69,8 +65,9 @@ where
     }
 }
 
-impl<const LANES: usize> BitMask<LANES>
+impl<Element, const LANES: usize> Mask<Element, LANES>
 where
+    Element: MaskElement,
     LaneCount<LANES>: SupportedLaneCount,
 {
     #[inline]
@@ -84,7 +81,7 @@ where
         if LANES % 8 > 0 {
             *mask.as_mut().last_mut().unwrap() &= u8::MAX >> (8 - LANES % 8);
         }
-        Self(mask)
+        Self(mask, PhantomData)
     }
 
     #[inline]
@@ -98,33 +95,28 @@ where
     }
 
     #[inline]
-    pub fn to_int<V>(self) -> V
-    where
-        V: ConvertToInt + Default + core::ops::Not<Output = V>,
-    {
+    pub fn to_int(self) -> Simd<Element, LANES> {
         unsafe {
             let mask: <LaneCount<LANES> as SupportedLaneCount>::IntBitMask =
                 core::mem::transmute_copy(&self);
-            crate::intrinsics::simd_select_bitmask(mask, !V::default(), V::default())
+            crate::intrinsics::simd_select_bitmask(
+                mask,
+                Simd::splat(Element::TRUE),
+                Simd::splat(Element::FALSE),
+            )
         }
     }
 
     #[inline]
-    pub unsafe fn from_int_unchecked<V>(value: V) -> Self
-    where
-        V: crate::Vector,
-    {
+    pub unsafe fn from_int_unchecked(value: Simd<Element, LANES>) -> Self {
         // TODO remove the transmute when rustc is more flexible
         assert_eq!(
-            core::mem::size_of::<<crate::LaneCount::<LANES> as crate::SupportedLaneCount>::BitMask>(
-            ),
-            core::mem::size_of::<
-                <crate::LaneCount::<LANES> as crate::SupportedLaneCount>::IntBitMask,
-            >(),
+            core::mem::size_of::<<LaneCount::<LANES> as SupportedLaneCount>::BitMask>(),
+            core::mem::size_of::<<LaneCount::<LANES> as SupportedLaneCount>::IntBitMask>(),
         );
         let mask: <LaneCount<LANES> as SupportedLaneCount>::IntBitMask =
             crate::intrinsics::simd_bitmask(value);
-        Self(core::mem::transmute_copy(&mask))
+        Self(core::mem::transmute_copy(&mask), PhantomData)
     }
 
     #[inline]
@@ -136,7 +128,15 @@ where
     #[inline]
     pub fn from_bitmask(bitmask: [u8; LaneCount::<LANES>::BITMASK_LEN]) -> Self {
         // Safety: these are the same type and we are laundering the generic
-        Self(unsafe { core::mem::transmute_copy(&bitmask) })
+        Self(unsafe { core::mem::transmute_copy(&bitmask) }, PhantomData)
+    }
+
+    #[inline]
+    pub fn convert<T>(self) -> Mask<T, LANES>
+    where
+        T: MaskElement,
+    {
+        unsafe { core::mem::transmute_copy(&self) }
     }
 
     #[inline]
@@ -150,10 +150,11 @@ where
     }
 }
 
-impl<const LANES: usize> core::ops::BitAnd for BitMask<LANES>
+impl<Element, const LANES: usize> core::ops::BitAnd for Mask<Element, LANES>
 where
+    Element: MaskElement,
     LaneCount<LANES>: SupportedLaneCount,
-    <LaneCount<LANES> as SupportedLaneCount>::BitMask: Default + AsRef<[u8]> + AsMut<[u8]>,
+    <LaneCount<LANES> as SupportedLaneCount>::BitMask: AsRef<[u8]> + AsMut<[u8]>,
 {
     type Output = Self;
     #[inline]
@@ -165,10 +166,11 @@ where
     }
 }
 
-impl<const LANES: usize> core::ops::BitOr for BitMask<LANES>
+impl<Element, const LANES: usize> core::ops::BitOr for Mask<Element, LANES>
 where
+    Element: MaskElement,
     LaneCount<LANES>: SupportedLaneCount,
-    <LaneCount<LANES> as SupportedLaneCount>::BitMask: Default + AsRef<[u8]> + AsMut<[u8]>,
+    <LaneCount<LANES> as SupportedLaneCount>::BitMask: AsRef<[u8]> + AsMut<[u8]>,
 {
     type Output = Self;
     #[inline]
@@ -180,8 +182,9 @@ where
     }
 }
 
-impl<const LANES: usize> core::ops::BitXor for BitMask<LANES>
+impl<Element, const LANES: usize> core::ops::BitXor for Mask<Element, LANES>
 where
+    Element: MaskElement,
     LaneCount<LANES>: SupportedLaneCount,
 {
     type Output = Self;
@@ -194,8 +197,9 @@ where
     }
 }
 
-impl<const LANES: usize> core::ops::Not for BitMask<LANES>
+impl<Element, const LANES: usize> core::ops::Not for Mask<Element, LANES>
 where
+    Element: MaskElement,
     LaneCount<LANES>: SupportedLaneCount,
 {
     type Output = Self;
@@ -210,9 +214,3 @@ where
         self
     }
 }
-
-pub type Mask8<const LANES: usize> = BitMask<LANES>;
-pub type Mask16<const LANES: usize> = BitMask<LANES>;
-pub type Mask32<const LANES: usize> = BitMask<LANES>;
-pub type Mask64<const LANES: usize> = BitMask<LANES>;
-pub type MaskSize<const LANES: usize> = BitMask<LANES>;
diff --git a/crates/core_simd/src/masks/full_masks.rs b/crates/core_simd/src/masks/full_masks.rs
index 2923cf1964a0..858c99032a31 100644
--- a/crates/core_simd/src/masks/full_masks.rs
+++ b/crates/core_simd/src/masks/full_masks.rs
@@ -1,264 +1,221 @@
 //! Masks that take up full SIMD vector registers.
 
-macro_rules! define_mask {
+use super::MaskElement;
+use crate::{LaneCount, Simd, SupportedLaneCount};
+
+#[repr(transparent)]
+pub struct Mask<Element, const LANES: usize>(Simd<Element, LANES>)
+where
+    Element: MaskElement,
+    LaneCount<LANES>: SupportedLaneCount;
+
+impl<Element, const LANES: usize> Copy for Mask<Element, LANES>
+where
+    Element: MaskElement,
+    LaneCount<LANES>: SupportedLaneCount,
+{
+}
+
+impl<Element, const LANES: usize> Clone for Mask<Element, LANES>
+where
+    Element: MaskElement,
+    LaneCount<LANES>: SupportedLaneCount,
+{
+    #[inline]
+    fn clone(&self) -> Self {
+        *self
+    }
+}
+
+impl<Element, const LANES: usize> PartialEq for Mask<Element, LANES>
+where
+    Element: MaskElement + PartialEq,
+    LaneCount<LANES>: SupportedLaneCount,
+{
+    fn eq(&self, other: &Self) -> bool {
+        self.0.eq(&other.0)
+    }
+}
+
+impl<Element, const LANES: usize> PartialOrd for Mask<Element, LANES>
+where
+    Element: MaskElement + PartialOrd,
+    LaneCount<LANES>: SupportedLaneCount,
+{
+    fn partial_cmp(&self, other: &Self) -> Option<core::cmp::Ordering> {
+        self.0.partial_cmp(&other.0)
+    }
+}
+
+impl<Element, const LANES: usize> Eq for Mask<Element, LANES>
+where
+    Element: MaskElement + Eq,
+    LaneCount<LANES>: SupportedLaneCount,
+{
+}
+
+impl<Element, const LANES: usize> Ord for Mask<Element, LANES>
+where
+    Element: MaskElement + Ord,
+    LaneCount<LANES>: SupportedLaneCount,
+{
+    fn cmp(&self, other: &Self) -> core::cmp::Ordering {
+        self.0.cmp(&other.0)
+    }
+}
+
+impl<Element, const LANES: usize> Mask<Element, LANES>
+where
+    Element: MaskElement,
+    LaneCount<LANES>: SupportedLaneCount,
+{
+    pub fn splat(value: bool) -> Self {
+        Self(Simd::splat(if value {
+            Element::TRUE
+        } else {
+            Element::FALSE
+        }))
+    }
+
+    #[inline]
+    pub unsafe fn test_unchecked(&self, lane: usize) -> bool {
+        Element::eq(self.0[lane], Element::TRUE)
+    }
+
+    #[inline]
+    pub unsafe fn set_unchecked(&mut self, lane: usize, value: bool) {
+        self.0[lane] = if value { Element::TRUE } else { Element::FALSE }
+    }
+
+    #[inline]
+    pub fn to_int(self) -> Simd<Element, LANES> {
+        self.0
+    }
+
+    #[inline]
+    pub unsafe fn from_int_unchecked(value: Simd<Element, LANES>) -> Self {
+        Self(value)
+    }
+
+    #[inline]
+    pub fn convert<T>(self) -> Mask<T, LANES>
+    where
+        T: MaskElement,
     {
-        $(#[$attr:meta])*
-        struct $name:ident<const $lanes:ident: usize>(
-            crate::$type:ident<$lanes2:ident>
-        );
-    } => {
-        $(#[$attr])*
-        #[repr(transparent)]
-        pub struct $name<const $lanes: usize>(crate::$type<$lanes>)
-        where
-            crate::LaneCount<$lanes>: crate::SupportedLaneCount;
+        unsafe { Mask(crate::intrinsics::simd_cast(self.0)) }
+    }
 
-        impl_full_mask_reductions! { $name, $type }
+    #[inline]
+    pub fn to_bitmask(self) -> [u8; LaneCount::<LANES>::BITMASK_LEN] {
+        unsafe {
+            // TODO remove the transmute when rustc can use arrays of u8 as bitmasks
+            assert_eq!(
+                core::mem::size_of::<<LaneCount::<LANES> as SupportedLaneCount>::IntBitMask>(),
+                LaneCount::<LANES>::BITMASK_LEN,
+            );
+            let bitmask: <LaneCount<LANES> as SupportedLaneCount>::IntBitMask =
+                crate::intrinsics::simd_bitmask(self.0);
+            let mut bitmask: [u8; LaneCount::<LANES>::BITMASK_LEN] =
+                core::mem::transmute_copy(&bitmask);
 
-        impl<const LANES: usize> Copy for $name<LANES>
-        where
-            crate::LaneCount<LANES>: crate::SupportedLaneCount,
-        {}
-
-        impl<const LANES: usize> Clone for $name<LANES>
-        where
-            crate::LaneCount<LANES>: crate::SupportedLaneCount,
-        {
-            #[inline]
-            fn clone(&self) -> Self {
-                *self
-            }
-        }
-
-        impl<const LANES: usize> PartialEq for $name<LANES>
-        where
-            crate::LaneCount<LANES>: crate::SupportedLaneCount,
-        {
-            fn eq(&self, other: &Self) -> bool {
-                self.0 == other.0
-            }
-        }
-
-        impl<const LANES: usize> PartialOrd for $name<LANES>
-        where
-            crate::LaneCount<LANES>: crate::SupportedLaneCount,
-        {
-            fn partial_cmp(&self, other: &Self) -> Option<core::cmp::Ordering> {
-                self.0.partial_cmp(&other.0)
-            }
-        }
-
-        impl<const LANES: usize> Eq for $name<LANES>
-        where
-            crate::LaneCount<LANES>: crate::SupportedLaneCount,
-        {}
-
-        impl<const LANES: usize> Ord for $name<LANES>
-        where
-            crate::LaneCount<LANES>: crate::SupportedLaneCount,
-        {
-            fn cmp(&self, other: &Self) -> core::cmp::Ordering {
-                self.0.cmp(&other.0)
-            }
-        }
-
-        impl<const LANES: usize> $name<LANES>
-        where
-            crate::LaneCount<LANES>: crate::SupportedLaneCount,
-        {
-            pub fn splat(value: bool) -> Self {
-                Self(
-                    <crate::$type<LANES>>::splat(
-                        if value {
-                            -1
-                        } else {
-                            0
-                        }
-                    ),
-                )
-            }
-
-            #[inline]
-            pub unsafe fn test_unchecked(&self, lane: usize) -> bool {
-                self.0[lane] == -1
-            }
-
-            #[inline]
-            pub unsafe fn set_unchecked(&mut self, lane: usize, value: bool) {
-                self.0[lane] = if value {
-                    -1
-                } else {
-                    0
+            // There is a bug where LLVM appears to implement this operation with the wrong
+            // bit order.
+            // TODO fix this in a better way
+            if cfg!(any(target_arch = "mips", target_arch = "mips64")) {
+                for x in bitmask.as_mut() {
+                    *x = x.reverse_bits();
                 }
             }
 
-            #[inline]
-            pub fn to_int(self) -> crate::$type<LANES> {
-                self.0
-            }
+            bitmask
+        }
+    }
 
-            #[inline]
-            pub unsafe fn from_int_unchecked(value: crate::$type<LANES>) -> Self {
-                Self(value)
-            }
-
-            #[inline]
-            pub fn to_bitmask(self) -> [u8; crate::LaneCount::<LANES>::BITMASK_LEN] {
-                unsafe {
-                    // TODO remove the transmute when rustc can use arrays of u8 as bitmasks
-                    assert_eq!(
-                        core::mem::size_of::<<crate::LaneCount::<LANES> as crate::SupportedLaneCount>::IntBitMask>(),
-                        crate::LaneCount::<LANES>::BITMASK_LEN,
-                    );
-                    let bitmask: <crate::LaneCount::<LANES> as crate::SupportedLaneCount>::IntBitMask = crate::intrinsics::simd_bitmask(self.0);
-                    let mut bitmask: [u8; crate::LaneCount::<LANES>::BITMASK_LEN] = core::mem::transmute_copy(&bitmask);
-
-                    // There is a bug where LLVM appears to implement this operation with the wrong
-                    // bit order.
-                    // TODO fix this in a better way
-                    if cfg!(any(target_arch = "mips", target_arch = "mips64")) {
-                        for x in bitmask.as_mut() {
-                            *x = x.reverse_bits();
-                        }
-                    }
-
-                    bitmask
+    #[inline]
+    pub fn from_bitmask(mut bitmask: [u8; LaneCount::<LANES>::BITMASK_LEN]) -> Self {
+        unsafe {
+            // There is a bug where LLVM appears to implement this operation with the wrong
+            // bit order.
+            // TODO fix this in a better way
+            if cfg!(any(target_arch = "mips", target_arch = "mips64")) {
+                for x in bitmask.as_mut() {
+                    *x = x.reverse_bits();
                 }
             }
 
-            #[inline]
-            pub fn from_bitmask(mut bitmask: [u8; crate::LaneCount::<LANES>::BITMASK_LEN]) -> Self {
-                unsafe {
-                    // There is a bug where LLVM appears to implement this operation with the wrong
-                    // bit order.
-                    // TODO fix this in a better way
-                    if cfg!(any(target_arch = "mips", target_arch = "mips64")) {
-                        for x in bitmask.as_mut() {
-                            *x = x.reverse_bits();
-                        }
-                    }
+            // TODO remove the transmute when rustc can use arrays of u8 as bitmasks
+            assert_eq!(
+                core::mem::size_of::<<LaneCount::<LANES> as SupportedLaneCount>::IntBitMask>(),
+                LaneCount::<LANES>::BITMASK_LEN,
+            );
+            let bitmask: <LaneCount<LANES> as SupportedLaneCount>::IntBitMask =
+                core::mem::transmute_copy(&bitmask);
 
-                    // TODO remove the transmute when rustc can use arrays of u8 as bitmasks
-                    assert_eq!(
-                        core::mem::size_of::<<crate::LaneCount::<LANES> as crate::SupportedLaneCount>::IntBitMask>(),
-                        crate::LaneCount::<LANES>::BITMASK_LEN,
-                    );
-                    let bitmask: <crate::LaneCount::<LANES> as crate::SupportedLaneCount>::IntBitMask = core::mem::transmute_copy(&bitmask);
-
-                    Self::from_int_unchecked(crate::intrinsics::simd_select_bitmask(
-                        bitmask,
-                        Self::splat(true).to_int(),
-                        Self::splat(false).to_int(),
-                    ))
-                }
-            }
-        }
-
-        impl<const LANES: usize> core::convert::From<$name<LANES>> for crate::$type<LANES>
-        where
-            crate::LaneCount<LANES>: crate::SupportedLaneCount,
-        {
-            fn from(value: $name<LANES>) -> Self {
-                value.0
-            }
-        }
-
-        impl<const LANES: usize> core::ops::BitAnd for $name<LANES>
-        where
-            crate::LaneCount<LANES>: crate::SupportedLaneCount,
-        {
-            type Output = Self;
-            #[inline]
-            fn bitand(self, rhs: Self) -> Self {
-                Self(self.0 & rhs.0)
-            }
-        }
-
-        impl<const LANES: usize> core::ops::BitOr for $name<LANES>
-        where
-            crate::LaneCount<LANES>: crate::SupportedLaneCount,
-        {
-            type Output = Self;
-            #[inline]
-            fn bitor(self, rhs: Self) -> Self {
-                Self(self.0 | rhs.0)
-            }
-        }
-
-        impl<const LANES: usize> core::ops::BitXor for $name<LANES>
-        where
-            crate::LaneCount<LANES>: crate::SupportedLaneCount,
-        {
-            type Output = Self;
-            #[inline]
-            fn bitxor(self, rhs: Self) -> Self::Output {
-                Self(self.0 ^ rhs.0)
-            }
-        }
-
-        impl<const LANES: usize> core::ops::Not for $name<LANES>
-        where
-            crate::LaneCount<LANES>: crate::SupportedLaneCount,
-        {
-            type Output = Self;
-            #[inline]
-            fn not(self) -> Self::Output {
-                Self(!self.0)
-            }
+            Self::from_int_unchecked(crate::intrinsics::simd_select_bitmask(
+                bitmask,
+                Self::splat(true).to_int(),
+                Self::splat(false).to_int(),
+            ))
         }
     }
 }
 
-define_mask! {
-    /// A mask equivalent to [SimdI8](crate::SimdI8), where all bits in the lane must be either set
-    /// or unset.
-    struct Mask8<const LANES: usize>(crate::SimdI8<LANES>);
-}
-
-define_mask! {
-    /// A mask equivalent to [SimdI16](crate::SimdI16), where all bits in the lane must be either set
-    /// or unset.
-    struct Mask16<const LANES: usize>(crate::SimdI16<LANES>);
-}
-
-define_mask! {
-    /// A mask equivalent to [SimdI32](crate::SimdI32), where all bits in the lane must be either set
-    /// or unset.
-    struct Mask32<const LANES: usize>(crate::SimdI32<LANES>);
-}
-
-define_mask! {
-    /// A mask equivalent to [SimdI64](crate::SimdI64), where all bits in the lane must be either set
-    /// or unset.
-    struct Mask64<const LANES: usize>(crate::SimdI64<LANES>);
-}
-
-define_mask! {
-    /// A mask equivalent to [SimdIsize](crate::SimdIsize), where all bits in the lane must be either set
-    /// or unset.
-    struct MaskSize<const LANES: usize>(crate::SimdIsize<LANES>);
-}
-
-macro_rules! impl_from {
-    { $from:ident ($from_inner:ident) => $($to:ident ($to_inner:ident)),* } => {
-        $(
-        impl<const LANES: usize> From<$from<LANES>> for $to<LANES>
-        where
-            crate::LaneCount<LANES>: crate::SupportedLaneCount,
-        {
-            fn from(value: $from<LANES>) -> Self {
-                let mut new = Self::splat(false);
-                for i in 0..LANES {
-                    unsafe { new.set_unchecked(i, value.test_unchecked(i)) }
-                }
-                new
-            }
-        }
-        )*
+impl<Element, const LANES: usize> core::convert::From<Mask<Element, LANES>> for Simd<Element, LANES>
+where
+    Element: MaskElement,
+    LaneCount<LANES>: SupportedLaneCount,
+{
+    fn from(value: Mask<Element, LANES>) -> Self {
+        value.0
     }
 }
-impl_from! { Mask8 (SimdI8) => Mask16 (SimdI16), Mask32 (SimdI32), Mask64 (SimdI64), MaskSize (SimdIsize) }
-impl_from! { Mask16 (SimdI16) => Mask32 (SimdI32), Mask64 (SimdI64), MaskSize (SimdIsize), Mask8 (SimdI8) }
-impl_from! { Mask32 (SimdI32) => Mask64 (SimdI64), MaskSize (SimdIsize), Mask8 (SimdI8), Mask16 (SimdI16) }
-impl_from! { Mask64 (SimdI64) => MaskSize (SimdIsize), Mask8 (SimdI8), Mask16 (SimdI16), Mask32 (SimdI32) }
-impl_from! { MaskSize (SimdIsize) => Mask8 (SimdI8), Mask16 (SimdI16), Mask32 (SimdI32), Mask64 (SimdI64) }
+
+impl<Element, const LANES: usize> core::ops::BitAnd for Mask<Element, LANES>
+where
+    Element: MaskElement,
+    LaneCount<LANES>: SupportedLaneCount,
+{
+    type Output = Self;
+    #[inline]
+    fn bitand(self, rhs: Self) -> Self {
+        unsafe { Self(crate::intrinsics::simd_and(self.0, rhs.0)) }
+    }
+}
+
+impl<Element, const LANES: usize> core::ops::BitOr for Mask<Element, LANES>
+where
+    Element: MaskElement,
+    LaneCount<LANES>: SupportedLaneCount,
+{
+    type Output = Self;
+    #[inline]
+    fn bitor(self, rhs: Self) -> Self {
+        unsafe { Self(crate::intrinsics::simd_or(self.0, rhs.0)) }
+    }
+}
+
+impl<Element, const LANES: usize> core::ops::BitXor for Mask<Element, LANES>
+where
+    Element: MaskElement,
+    LaneCount<LANES>: SupportedLaneCount,
+{
+    type Output = Self;
+    #[inline]
+    fn bitxor(self, rhs: Self) -> Self {
+        unsafe { Self(crate::intrinsics::simd_xor(self.0, rhs.0)) }
+    }
+}
+
+impl<Element, const LANES: usize> core::ops::Not for Mask<Element, LANES>
+where
+    Element: MaskElement,
+    LaneCount<LANES>: SupportedLaneCount,
+{
+    type Output = Self;
+    #[inline]
+    fn not(self) -> Self::Output {
+        Self::splat(true) ^ self
+    }
+}
+
+impl_full_mask_reductions! {}
diff --git a/crates/core_simd/src/ops.rs b/crates/core_simd/src/ops.rs
index c75090aab9c5..67bafd73b144 100644
--- a/crates/core_simd/src/ops.rs
+++ b/crates/core_simd/src/ops.rs
@@ -1,4 +1,27 @@
-use crate::{LaneCount, SupportedLaneCount};
+use crate::{LaneCount, Simd, SimdElement, SupportedLaneCount};
+
+impl<I, Element, const LANES: usize> core::ops::Index<I> for Simd<Element, LANES>
+where
+    Element: SimdElement,
+    LaneCount<LANES>: SupportedLaneCount,
+    I: core::slice::SliceIndex<[Element]>,
+{
+    type Output = I::Output;
+    fn index(&self, index: I) -> &Self::Output {
+        &self.as_array()[index]
+    }
+}
+
+impl<I, Element, const LANES: usize> core::ops::IndexMut<I> for Simd<Element, LANES>
+where
+    Element: SimdElement,
+    LaneCount<LANES>: SupportedLaneCount,
+    I: core::slice::SliceIndex<[Element]>,
+{
+    fn index_mut(&mut self, index: I) -> &mut Self::Output {
+        &mut self.as_mut_array()[index]
+    }
+}
 
 /// Checks if the right-hand side argument of a left- or right-shift would cause overflow.
 fn invalid_shift_rhs<T>(rhs: T) -> bool
@@ -191,31 +214,6 @@ macro_rules! impl_op {
         }
     };
 
-    { impl Index for $type:ident, $scalar:ty } => {
-        impl<I, const LANES: usize> core::ops::Index<I> for crate::$type<LANES>
-        where
-            LaneCount<LANES>: SupportedLaneCount,
-            I: core::slice::SliceIndex<[$scalar]>,
-        {
-            type Output = I::Output;
-            fn index(&self, index: I) -> &Self::Output {
-                let slice: &[_] = self.as_ref();
-                &slice[index]
-            }
-        }
-
-        impl<I, const LANES: usize> core::ops::IndexMut<I> for crate::$type<LANES>
-        where
-            LaneCount<LANES>: SupportedLaneCount,
-            I: core::slice::SliceIndex<[$scalar]>,
-        {
-            fn index_mut(&mut self, index: I) -> &mut Self::Output {
-                let slice: &mut [_] = self.as_mut();
-                &mut slice[index]
-            }
-        }
-    };
-
     // generic binary op with assignment when output is `Self`
     { @binary $type:ident, $scalar:ty, $trait:ident :: $trait_fn:ident, $assign_trait:ident :: $assign_trait_fn:ident, $intrinsic:ident } => {
         impl_ref_ops! {
@@ -301,7 +299,6 @@ macro_rules! impl_float_ops {
                 impl_op! { impl Div for $vector, $scalar }
                 impl_op! { impl Rem for $vector, $scalar }
                 impl_op! { impl Neg for $vector, $scalar }
-                impl_op! { impl Index for $vector, $scalar }
             )*
         )*
     };
@@ -319,7 +316,6 @@ macro_rules! impl_unsigned_int_ops {
                 impl_op! { impl BitOr  for $vector, $scalar }
                 impl_op! { impl BitXor for $vector, $scalar }
                 impl_op! { impl Not for $vector, $scalar }
-                impl_op! { impl Index for $vector, $scalar }
 
                 // Integers panic on divide by 0
                 impl_ref_ops! {
diff --git a/crates/core_simd/src/reduction.rs b/crates/core_simd/src/reduction.rs
index df227d09e342..b9c24d027b62 100644
--- a/crates/core_simd/src/reduction.rs
+++ b/crates/core_simd/src/reduction.rs
@@ -103,10 +103,11 @@ macro_rules! impl_float_reductions {
 }
 
 macro_rules! impl_full_mask_reductions {
-    { $name:ident, $bits_ty:ident } => {
-        impl<const LANES: usize> $name<LANES>
+    {} => {
+        impl<Element, const LANES: usize> Mask<Element, LANES>
         where
-            crate::LaneCount<LANES>: crate::SupportedLaneCount,
+            Element: MaskElement,
+            LaneCount<LANES>: SupportedLaneCount,
         {
             #[inline]
             pub fn any(self) -> bool {
@@ -120,24 +121,3 @@ macro_rules! impl_full_mask_reductions {
         }
     }
 }
-
-macro_rules! impl_opaque_mask_reductions {
-    { $name:ident, $bits_ty:ident } => {
-        impl<const LANES: usize> $name<LANES>
-        where
-            crate::LaneCount<LANES>: crate::SupportedLaneCount,
-        {
-            /// Returns true if any lane is set, or false otherwise.
-            #[inline]
-            pub fn any(self) -> bool {
-                self.0.any()
-            }
-
-            /// Returns true if all lanes are set, or false otherwise.
-            #[inline]
-            pub fn all(self) -> bool {
-                self.0.all()
-            }
-        }
-    }
-}

From ea0280539cfee50c02fb5ed87960390d2d68008b Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Sat, 7 Aug 2021 05:19:06 +0000
Subject: [PATCH 209/249] Implement select generically

---
 crates/core_simd/src/select.rs | 142 ++++++++++++++++-----------------
 crates/core_simd/src/vector.rs |  14 ++--
 2 files changed, 77 insertions(+), 79 deletions(-)

diff --git a/crates/core_simd/src/select.rs b/crates/core_simd/src/select.rs
index d70e8a66b95f..1f7ea854a933 100644
--- a/crates/core_simd/src/select.rs
+++ b/crates/core_simd/src/select.rs
@@ -1,3 +1,5 @@
+use crate::{LaneCount, Mask, MaskElement, Simd, SimdElement, SupportedLaneCount};
+
 mod sealed {
     pub trait Sealed {}
 }
@@ -9,79 +11,75 @@ pub trait Select<Mask>: Sealed {
     fn select(mask: Mask, true_values: Self, false_values: Self) -> Self;
 }
 
-macro_rules! impl_select {
-    {
-        $mask:ident ($bits_ty:ident): $($type:ident),*
-    } => {
-        $(
-        impl<const LANES: usize> Sealed for crate::$type<LANES> where crate::LaneCount<LANES>: crate::SupportedLaneCount {}
-        impl<const LANES: usize> Select<crate::$mask<LANES>> for crate::$type<LANES>
-        where
-            crate::LaneCount<LANES>: crate::SupportedLaneCount,
-        {
-            #[doc(hidden)]
-            #[inline]
-            fn select(mask: crate::$mask<LANES>, true_values: Self, false_values: Self) -> Self {
-                unsafe { crate::intrinsics::simd_select(mask.to_int(), true_values, false_values) }
-            }
-        }
-        )*
+impl<Element, const LANES: usize> Sealed for Simd<Element, LANES>
+where
+    Element: SimdElement,
+    LaneCount<LANES>: SupportedLaneCount,
+{
+}
 
-        impl<const LANES: usize> Sealed for crate::$mask<LANES>
-        where
-            crate::LaneCount<LANES>: crate::SupportedLaneCount,
-        {}
-
-        impl<const LANES: usize> Select<Self> for crate::$mask<LANES>
-        where
-            crate::LaneCount<LANES>: crate::SupportedLaneCount,
-        {
-            #[doc(hidden)]
-            #[inline]
-            fn select(mask: Self, true_values: Self, false_values: Self) -> Self {
-                mask & true_values | !mask & false_values
-            }
-        }
-
-        impl<const LANES: usize> crate::$mask<LANES>
-        where
-            crate::LaneCount<LANES>: crate::SupportedLaneCount,
-        {
-            /// Choose lanes from two vectors.
-            ///
-            /// For each lane in the mask, choose the corresponding lane from `true_values` if
-            /// that lane mask is true, and `false_values` if that lane mask is false.
-            ///
-            /// ```
-            /// # #![feature(portable_simd)]
-            /// # use core_simd::{Mask32, SimdI32};
-            /// let a = SimdI32::from_array([0, 1, 2, 3]);
-            /// let b = SimdI32::from_array([4, 5, 6, 7]);
-            /// let mask = Mask32::from_array([true, false, false, true]);
-            /// let c = mask.select(a, b);
-            /// assert_eq!(c.to_array(), [0, 5, 6, 3]);
-            /// ```
-            ///
-            /// `select` can also be used on masks:
-            /// ```
-            /// # #![feature(portable_simd)]
-            /// # use core_simd::Mask32;
-            /// let a = Mask32::from_array([true, true, false, false]);
-            /// let b = Mask32::from_array([false, false, true, true]);
-            /// let mask = Mask32::from_array([true, false, false, true]);
-            /// let c = mask.select(a, b);
-            /// assert_eq!(c.to_array(), [true, false, true, false]);
-            /// ```
-            #[inline]
-            pub fn select<S: Select<Self>>(self, true_values: S, false_values: S) -> S {
-                S::select(self, true_values, false_values)
-            }
-        }
+impl<Element, const LANES: usize> Select<Mask<Element::Mask, LANES>> for Simd<Element, LANES>
+where
+    Element: SimdElement,
+    LaneCount<LANES>: SupportedLaneCount,
+{
+    #[inline]
+    fn select(mask: Mask<Element::Mask, LANES>, true_values: Self, false_values: Self) -> Self {
+        unsafe { crate::intrinsics::simd_select(mask.to_int(), true_values, false_values) }
     }
 }
 
-impl_select! { Mask8 (SimdI8): SimdU8, SimdI8 }
-impl_select! { Mask16 (SimdI16): SimdU16, SimdI16 }
-impl_select! { Mask32 (SimdI32): SimdU32, SimdI32, SimdF32}
-impl_select! { Mask64 (SimdI64): SimdU64, SimdI64, SimdF64}
-impl_select! { MaskSize (SimdIsize): SimdUsize, SimdIsize }
+impl<Element, const LANES: usize> Sealed for Mask<Element, LANES>
+where
+    Element: MaskElement,
+    LaneCount<LANES>: SupportedLaneCount,
+{
+}
+
+impl<Element, const LANES: usize> Select<Self> for Mask<Element, LANES>
+where
+    Element: MaskElement,
+    LaneCount<LANES>: SupportedLaneCount,
+{
+    #[doc(hidden)]
+    #[inline]
+    fn select(mask: Self, true_values: Self, false_values: Self) -> Self {
+        mask & true_values | !mask & false_values
+    }
+}
+
+impl<Element, const LANES: usize> Mask<Element, LANES>
+where
+    Element: MaskElement,
+    LaneCount<LANES>: SupportedLaneCount,
+{
+    /// Choose lanes from two vectors.
+    ///
+    /// For each lane in the mask, choose the corresponding lane from `true_values` if
+    /// that lane mask is true, and `false_values` if that lane mask is false.
+    ///
+    /// ```
+    /// # #![feature(portable_simd)]
+    /// # use core_simd::{Mask32, SimdI32};
+    /// let a = SimdI32::from_array([0, 1, 2, 3]);
+    /// let b = SimdI32::from_array([4, 5, 6, 7]);
+    /// let mask = Mask32::from_array([true, false, false, true]);
+    /// let c = mask.select(a, b);
+    /// assert_eq!(c.to_array(), [0, 5, 6, 3]);
+    /// ```
+    ///
+    /// `select` can also be used on masks:
+    /// ```
+    /// # #![feature(portable_simd)]
+    /// # use core_simd::Mask32;
+    /// let a = Mask32::from_array([true, true, false, false]);
+    /// let b = Mask32::from_array([false, false, true, true]);
+    /// let mask = Mask32::from_array([true, false, false, true]);
+    /// let c = mask.select(a, b);
+    /// assert_eq!(c.to_array(), [true, false, true, false]);
+    /// ```
+    #[inline]
+    pub fn select<S: Select<Self>>(self, true_values: S, false_values: S) -> S {
+        S::select(self, true_values, false_values)
+    }
+}
diff --git a/crates/core_simd/src/vector.rs b/crates/core_simd/src/vector.rs
index 9fb6d17d40af..03c2f93a9c29 100644
--- a/crates/core_simd/src/vector.rs
+++ b/crates/core_simd/src/vector.rs
@@ -9,7 +9,7 @@ pub use uint::*;
 // Vectors of pointers are not for public use at the current time.
 pub(crate) mod ptr;
 
-use crate::{LaneCount, SupportedLaneCount};
+use crate::{LaneCount, MaskElement, SupportedLaneCount};
 
 /// A SIMD vector of `LANES` elements of type `Element`.
 #[repr(simd)]
@@ -338,32 +338,32 @@ use sealed::Sealed;
 /// Marker trait for types that may be used as SIMD vector elements.
 pub unsafe trait SimdElement: Sealed + Copy {
     /// The mask element type corresponding to this element type.
-    type Mask: SimdElement;
+    type Mask: MaskElement;
 }
 
 impl Sealed for u8 {}
 unsafe impl SimdElement for u8 {
-    type Mask = u8;
+    type Mask = i8;
 }
 
 impl Sealed for u16 {}
 unsafe impl SimdElement for u16 {
-    type Mask = u16;
+    type Mask = i16;
 }
 
 impl Sealed for u32 {}
 unsafe impl SimdElement for u32 {
-    type Mask = u32;
+    type Mask = i32;
 }
 
 impl Sealed for u64 {}
 unsafe impl SimdElement for u64 {
-    type Mask = u64;
+    type Mask = i64;
 }
 
 impl Sealed for usize {}
 unsafe impl SimdElement for usize {
-    type Mask = usize;
+    type Mask = isize;
 }
 
 impl Sealed for i8 {}

From e6d95e47983e69e63abfb080efafc2c7fa1f6c67 Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Sat, 7 Aug 2021 05:47:54 +0000
Subject: [PATCH 210/249] Implement comparisons generically

---
 crates/core_simd/src/comparisons.rs | 110 +++++++++++-----------------
 1 file changed, 41 insertions(+), 69 deletions(-)

diff --git a/crates/core_simd/src/comparisons.rs b/crates/core_simd/src/comparisons.rs
index c5e9be9015fe..c094f680a59d 100644
--- a/crates/core_simd/src/comparisons.rs
+++ b/crates/core_simd/src/comparisons.rs
@@ -1,77 +1,49 @@
-use crate::{LaneCount, SupportedLaneCount};
+use crate::{LaneCount, Mask, Simd, SimdElement, SupportedLaneCount};
 
-macro_rules! implement_mask_ops {
-    { $($vector:ident => $mask:ident ($inner_ty:ident),)* } => {
-        $(
-            impl<const LANES: usize> crate::$vector<LANES>
-            where
-                LaneCount<LANES>: SupportedLaneCount,
-            {
-                /// Test if each lane is equal to the corresponding lane in `other`.
-                #[inline]
-                pub fn lanes_eq(self, other: Self) -> crate::$mask<LANES> {
-                    unsafe {
-                        crate::$mask::from_int_unchecked(crate::intrinsics::simd_eq(self, other))
-                    }
-                }
+impl<Element, const LANES: usize> Simd<Element, LANES>
+where
+    Element: SimdElement + PartialEq,
+    LaneCount<LANES>: SupportedLaneCount,
+{
+    /// Test if each lane is equal to the corresponding lane in `other`.
+    #[inline]
+    pub fn lanes_eq(self, other: Self) -> Mask<Element::Mask, LANES> {
+        unsafe { Mask::from_int_unchecked(crate::intrinsics::simd_eq(self, other)) }
+    }
 
-                /// Test if each lane is not equal to the corresponding lane in `other`.
-                #[inline]
-                pub fn lanes_ne(self, other: Self) -> crate::$mask<LANES> {
-                    unsafe {
-                        crate::$mask::from_int_unchecked(crate::intrinsics::simd_ne(self, other))
-                    }
-                }
-
-                /// Test if each lane is less than the corresponding lane in `other`.
-                #[inline]
-                pub fn lanes_lt(self, other: Self) -> crate::$mask<LANES> {
-                    unsafe {
-                        crate::$mask::from_int_unchecked(crate::intrinsics::simd_lt(self, other))
-                    }
-                }
-
-                /// Test if each lane is greater than the corresponding lane in `other`.
-                #[inline]
-                pub fn lanes_gt(self, other: Self) -> crate::$mask<LANES> {
-                    unsafe {
-                        crate::$mask::from_int_unchecked(crate::intrinsics::simd_gt(self, other))
-                    }
-                }
-
-                /// Test if each lane is less than or equal to the corresponding lane in `other`.
-                #[inline]
-                pub fn lanes_le(self, other: Self) -> crate::$mask<LANES> {
-                    unsafe {
-                        crate::$mask::from_int_unchecked(crate::intrinsics::simd_le(self, other))
-                    }
-                }
-
-                /// Test if each lane is greater than or equal to the corresponding lane in `other`.
-                #[inline]
-                pub fn lanes_ge(self, other: Self) -> crate::$mask<LANES> {
-                    unsafe {
-                        crate::$mask::from_int_unchecked(crate::intrinsics::simd_ge(self, other))
-                    }
-                }
-            }
-        )*
+    /// Test if each lane is not equal to the corresponding lane in `other`.
+    #[inline]
+    pub fn lanes_ne(self, other: Self) -> Mask<Element::Mask, LANES> {
+        unsafe { Mask::from_int_unchecked(crate::intrinsics::simd_ne(self, other)) }
     }
 }
 
-implement_mask_ops! {
-    SimdI8 => Mask8 (SimdI8),
-    SimdI16 => Mask16 (SimdI16),
-    SimdI32 => Mask32 (SimdI32),
-    SimdI64 => Mask64 (SimdI64),
-    SimdIsize => MaskSize (SimdIsize),
+impl<Element, const LANES: usize> Simd<Element, LANES>
+where
+    Element: SimdElement + PartialOrd,
+    LaneCount<LANES>: SupportedLaneCount,
+{
+    /// Test if each lane is less than the corresponding lane in `other`.
+    #[inline]
+    pub fn lanes_lt(self, other: Self) -> Mask<Element::Mask, LANES> {
+        unsafe { Mask::from_int_unchecked(crate::intrinsics::simd_lt(self, other)) }
+    }
 
-    SimdU8 => Mask8 (SimdI8),
-    SimdU16 => Mask16 (SimdI16),
-    SimdU32 => Mask32 (SimdI32),
-    SimdU64 => Mask64 (SimdI64),
-    SimdUsize => MaskSize (SimdIsize),
+    /// Test if each lane is greater than the corresponding lane in `other`.
+    #[inline]
+    pub fn lanes_gt(self, other: Self) -> Mask<Element::Mask, LANES> {
+        unsafe { Mask::from_int_unchecked(crate::intrinsics::simd_gt(self, other)) }
+    }
 
-    SimdF32 => Mask32 (SimdI32),
-    SimdF64 => Mask64 (SimdI64),
+    /// Test if each lane is less than or equal to the corresponding lane in `other`.
+    #[inline]
+    pub fn lanes_le(self, other: Self) -> Mask<Element::Mask, LANES> {
+        unsafe { Mask::from_int_unchecked(crate::intrinsics::simd_le(self, other)) }
+    }
+
+    /// Test if each lane is greater than or equal to the corresponding lane in `other`.
+    #[inline]
+    pub fn lanes_ge(self, other: Self) -> Mask<Element::Mask, LANES> {
+        unsafe { Mask::from_int_unchecked(crate::intrinsics::simd_ge(self, other)) }
+    }
 }

From e11286a362a844a40dfa1c74f4d656568e7bdf07 Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Sat, 7 Aug 2021 16:53:08 +0000
Subject: [PATCH 211/249] Remove unused transmute file

---
 crates/core_simd/src/transmute.rs | 0
 1 file changed, 0 insertions(+), 0 deletions(-)
 delete mode 100644 crates/core_simd/src/transmute.rs

diff --git a/crates/core_simd/src/transmute.rs b/crates/core_simd/src/transmute.rs
deleted file mode 100644
index e69de29bb2d1..000000000000

From 5ed57b4c85a7a9c104b093afbf54d5b7e25069de Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Sat, 7 Aug 2021 19:28:27 +0000
Subject: [PATCH 212/249] Remove most usage of type aliases

---
 crates/core_simd/src/iter.rs             | 56 +++++++++--------
 crates/core_simd/src/masks.rs            | 18 +++---
 crates/core_simd/src/masks/full_masks.rs | 12 +++-
 crates/core_simd/src/math.rs             | 76 ++++++++++++------------
 crates/core_simd/src/reduction.rs        | 46 +++++++-------
 crates/core_simd/src/round.rs            | 20 ++++---
 crates/core_simd/src/to_bytes.rs         | 32 +++++-----
 crates/core_simd/src/vector/float.rs     |  2 -
 crates/core_simd/src/vector/int.rs       |  2 -
 crates/core_simd/src/vector/uint.rs      | 13 ----
 crates/core_simd/src/vendor/arm.rs       | 16 ++---
 crates/core_simd/src/vendor/x86.rs       |  4 +-
 12 files changed, 145 insertions(+), 152 deletions(-)

diff --git a/crates/core_simd/src/iter.rs b/crates/core_simd/src/iter.rs
index 0020ea5f2016..f403f4d90473 100644
--- a/crates/core_simd/src/iter.rs
+++ b/crates/core_simd/src/iter.rs
@@ -1,54 +1,58 @@
-use crate::{LaneCount, SupportedLaneCount};
+use crate::{LaneCount, Simd, SupportedLaneCount};
+use core::{
+    iter::{Product, Sum},
+    ops::{Add, Mul},
+};
 
 macro_rules! impl_traits {
-    { $type:ident } => {
-        impl<const LANES: usize> core::iter::Sum<Self> for crate::$type<LANES>
+    { $type:ty } => {
+        impl<const LANES: usize> Sum<Self> for Simd<$type, LANES>
         where
             LaneCount<LANES>: SupportedLaneCount,
         {
-            fn sum<I: core::iter::Iterator<Item = Self>>(iter: I) -> Self {
-                iter.fold(Default::default(), core::ops::Add::add)
+            fn sum<I: Iterator<Item = Self>>(iter: I) -> Self {
+                iter.fold(Simd::splat(0 as $type), Add::add)
             }
         }
 
-        impl<const LANES: usize> core::iter::Product<Self> for crate::$type<LANES>
+        impl<const LANES: usize> core::iter::Product<Self> for Simd<$type, LANES>
         where
             LaneCount<LANES>: SupportedLaneCount,
         {
-            fn product<I: core::iter::Iterator<Item = Self>>(iter: I) -> Self {
-                iter.fold(Default::default(), core::ops::Mul::mul)
+            fn product<I: Iterator<Item = Self>>(iter: I) -> Self {
+                iter.fold(Simd::splat(1 as $type), Mul::mul)
             }
         }
 
-        impl<'a, const LANES: usize> core::iter::Sum<&'a Self> for crate::$type<LANES>
+        impl<'a, const LANES: usize> Sum<&'a Self> for Simd<$type, LANES>
         where
             LaneCount<LANES>: SupportedLaneCount,
         {
-            fn sum<I: core::iter::Iterator<Item = &'a Self>>(iter: I) -> Self {
-                iter.fold(Default::default(), core::ops::Add::add)
+            fn sum<I: Iterator<Item = &'a Self>>(iter: I) -> Self {
+                iter.fold(Simd::splat(0 as $type), Add::add)
             }
         }
 
-        impl<'a, const LANES: usize> core::iter::Product<&'a Self> for crate::$type<LANES>
+        impl<'a, const LANES: usize> Product<&'a Self> for Simd<$type, LANES>
         where
             LaneCount<LANES>: SupportedLaneCount,
         {
-            fn product<I: core::iter::Iterator<Item = &'a Self>>(iter: I) -> Self {
-                iter.fold(Default::default(), core::ops::Mul::mul)
+            fn product<I: Iterator<Item = &'a Self>>(iter: I) -> Self {
+                iter.fold(Simd::splat(1 as $type), Mul::mul)
             }
         }
     }
 }
 
-impl_traits! { SimdF32 }
-impl_traits! { SimdF64 }
-impl_traits! { SimdU8 }
-impl_traits! { SimdU16 }
-impl_traits! { SimdU32 }
-impl_traits! { SimdU64 }
-impl_traits! { SimdUsize }
-impl_traits! { SimdI8 }
-impl_traits! { SimdI16 }
-impl_traits! { SimdI32 }
-impl_traits! { SimdI64 }
-impl_traits! { SimdIsize }
+impl_traits! { f32 }
+impl_traits! { f64 }
+impl_traits! { u8 }
+impl_traits! { u16 }
+impl_traits! { u32 }
+impl_traits! { u64 }
+impl_traits! { usize }
+impl_traits! { i8 }
+impl_traits! { i16 }
+impl_traits! { i32 }
+impl_traits! { i64 }
+impl_traits! { isize }
diff --git a/crates/core_simd/src/masks.rs b/crates/core_simd/src/masks.rs
index d4a0cff4e238..b7bde44b384f 100644
--- a/crates/core_simd/src/masks.rs
+++ b/crates/core_simd/src/masks.rs
@@ -522,21 +522,21 @@ pub type masksizex4 = MaskSize<4>;
 pub type masksizex8 = MaskSize<8>;
 
 macro_rules! impl_from {
-    { $from:ident ($from_inner:ident) => $($to:ident ($to_inner:ident)),* } => {
+    { $from:ty  => $($to:ty),* } => {
         $(
-        impl<const LANES: usize> From<$from<LANES>> for $to<LANES>
+        impl<const LANES: usize> From<Mask<$from, LANES>> for Mask<$to, LANES>
         where
-            crate::LaneCount<LANES>: crate::SupportedLaneCount,
+            LaneCount<LANES>: SupportedLaneCount,
         {
-            fn from(value: $from<LANES>) -> Self {
+            fn from(value: Mask<$from, LANES>) -> Self {
                 Self(value.0.convert())
             }
         }
         )*
     }
 }
-impl_from! { Mask8 (SimdI8) => Mask16 (SimdI16), Mask32 (SimdI32), Mask64 (SimdI64), MaskSize (SimdIsize) }
-impl_from! { Mask16 (SimdI16) => Mask32 (SimdI32), Mask64 (SimdI64), MaskSize (SimdIsize), Mask8 (SimdI8) }
-impl_from! { Mask32 (SimdI32) => Mask64 (SimdI64), MaskSize (SimdIsize), Mask8 (SimdI8), Mask16 (SimdI16) }
-impl_from! { Mask64 (SimdI64) => MaskSize (SimdIsize), Mask8 (SimdI8), Mask16 (SimdI16), Mask32 (SimdI32) }
-impl_from! { MaskSize (SimdIsize) => Mask8 (SimdI8), Mask16 (SimdI16), Mask32 (SimdI32), Mask64 (SimdI64) }
+impl_from! { i8 => i16, i32, i64, isize }
+impl_from! { i16 => i32, i64, isize, i8 }
+impl_from! { i32 => i64, isize, i8, i16 }
+impl_from! { i64 => isize, i8, i16, i32 }
+impl_from! { isize => i8, i16, i32, i64 }
diff --git a/crates/core_simd/src/masks/full_masks.rs b/crates/core_simd/src/masks/full_masks.rs
index 858c99032a31..b45ace3791d3 100644
--- a/crates/core_simd/src/masks/full_masks.rs
+++ b/crates/core_simd/src/masks/full_masks.rs
@@ -158,6 +158,16 @@ where
             ))
         }
     }
+
+    #[inline]
+    pub fn any(self) -> bool {
+        unsafe { crate::intrinsics::simd_reduce_any(self.to_int()) }
+    }
+
+    #[inline]
+    pub fn all(self) -> bool {
+        unsafe { crate::intrinsics::simd_reduce_all(self.to_int()) }
+    }
 }
 
 impl<Element, const LANES: usize> core::convert::From<Mask<Element, LANES>> for Simd<Element, LANES>
@@ -217,5 +227,3 @@ where
         Self::splat(true) ^ self
     }
 }
-
-impl_full_mask_reductions! {}
diff --git a/crates/core_simd/src/math.rs b/crates/core_simd/src/math.rs
index 28720eb13e3c..7affecbafd68 100644
--- a/crates/core_simd/src/math.rs
+++ b/crates/core_simd/src/math.rs
@@ -1,6 +1,8 @@
+use crate::{LaneCount, Simd, SupportedLaneCount};
+
 macro_rules! impl_uint_arith {
-    ($(($name:ident, $n:ident)),+) => {
-        $( impl<const LANES: usize> $name<LANES> where crate::LaneCount<LANES>: crate::SupportedLaneCount {
+    ($($ty:ty),+) => {
+        $( impl<const LANES: usize> Simd<$ty, LANES> where LaneCount<LANES>: SupportedLaneCount {
 
             /// Lanewise saturating add.
             ///
@@ -8,9 +10,9 @@ macro_rules! impl_uint_arith {
             /// ```
             /// # #![feature(portable_simd)]
             /// # use core_simd::*;
-            #[doc = concat!("# use core::", stringify!($n), "::MAX;")]
-            #[doc = concat!("let x = ", stringify!($name), "::from_array([2, 1, 0, MAX]);")]
-            #[doc = concat!("let max = ", stringify!($name), "::splat(MAX);")]
+            #[doc = concat!("# use core::", stringify!($ty), "::MAX;")]
+            /// let x = Simd::from_array([2, 1, 0, MAX]);
+            /// let max = Simd::splat(MAX);
             /// let unsat = x + max;
             /// let sat = x.saturating_add(max);
             /// assert_eq!(x - 1, unsat);
@@ -27,13 +29,13 @@ macro_rules! impl_uint_arith {
             /// ```
             /// # #![feature(portable_simd)]
             /// # use core_simd::*;
-            #[doc = concat!("# use core::", stringify!($n), "::MAX;")]
-            #[doc = concat!("let x = ", stringify!($name), "::from_array([2, 1, 0, MAX]);")]
-            #[doc = concat!("let max = ", stringify!($name), "::splat(MAX);")]
+            #[doc = concat!("# use core::", stringify!($ty), "::MAX;")]
+            /// let x = Simd::from_array([2, 1, 0, MAX]);
+            /// let max = Simd::splat(MAX);
             /// let unsat = x - max;
             /// let sat = x.saturating_sub(max);
             /// assert_eq!(unsat, x + 1);
-            #[doc = concat!("assert_eq!(sat, ", stringify!($name), "::splat(0));")]
+            /// assert_eq!(sat, Simd::splat(0));
             #[inline]
             pub fn saturating_sub(self, second: Self) -> Self {
                 unsafe { crate::intrinsics::simd_saturating_sub(self, second) }
@@ -43,8 +45,8 @@ macro_rules! impl_uint_arith {
 }
 
 macro_rules! impl_int_arith {
-    ($(($name:ident, $n:ident)),+) => {
-        $( impl<const LANES: usize> $name<LANES> where crate::LaneCount<LANES>: crate::SupportedLaneCount {
+    ($($ty:ty),+) => {
+        $( impl<const LANES: usize> Simd<$ty, LANES> where LaneCount<LANES>: SupportedLaneCount {
 
             /// Lanewise saturating add.
             ///
@@ -52,13 +54,13 @@ macro_rules! impl_int_arith {
             /// ```
             /// # #![feature(portable_simd)]
             /// # use core_simd::*;
-            #[doc = concat!("# use core::", stringify!($n), "::{MIN, MAX};")]
-            #[doc = concat!("let x = ", stringify!($name), "::from_array([MIN, 0, 1, MAX]);")]
-            #[doc = concat!("let max = ", stringify!($name), "::splat(MAX);")]
+            #[doc = concat!("# use core::", stringify!($ty), "::{MIN, MAX};")]
+            /// let x = Simd::from_array([MIN, 0, 1, MAX]);
+            /// let max = Simd::splat(MAX);
             /// let unsat = x + max;
             /// let sat = x.saturating_add(max);
-            #[doc = concat!("assert_eq!(unsat, ", stringify!($name), "::from_array([-1, MAX, MIN, -2]));")]
-            #[doc = concat!("assert_eq!(sat, ", stringify!($name), "::from_array([-1, MAX, MAX, MAX]));")]
+            /// assert_eq!(unsat, Simd::from_array([-1, MAX, MIN, -2]));
+            /// assert_eq!(sat, Simd::from_array([-1, MAX, MAX, MAX]));
             /// ```
             #[inline]
             pub fn saturating_add(self, second: Self) -> Self {
@@ -71,13 +73,13 @@ macro_rules! impl_int_arith {
             /// ```
             /// # #![feature(portable_simd)]
             /// # use core_simd::*;
-            #[doc = concat!("# use core::", stringify!($n), "::{MIN, MAX};")]
-            #[doc = concat!("let x = ", stringify!($name), "::from_array([MIN, -2, -1, MAX]);")]
-            #[doc = concat!("let max = ", stringify!($name), "::splat(MAX);")]
+            #[doc = concat!("# use core::", stringify!($ty), "::{MIN, MAX};")]
+            /// let x = Simd::from_array([MIN, -2, -1, MAX]);
+            /// let max = Simd::splat(MAX);
             /// let unsat = x - max;
             /// let sat = x.saturating_sub(max);
-            #[doc = concat!("assert_eq!(unsat, ", stringify!($name), "::from_array([1, MAX, MIN, 0]));")]
-            #[doc = concat!("assert_eq!(sat, ", stringify!($name), "::from_array([MIN, MIN, MIN, 0]));")]
+            /// assert_eq!(unsat, Simd::from_array([1, MAX, MIN, 0]));
+            /// assert_eq!(sat, Simd::from_array([MIN, MIN, MIN, 0]));
             #[inline]
             pub fn saturating_sub(self, second: Self) -> Self {
                 unsafe { crate::intrinsics::simd_saturating_sub(self, second) }
@@ -90,13 +92,13 @@ macro_rules! impl_int_arith {
             /// ```
             /// # #![feature(portable_simd)]
             /// # use core_simd::*;
-            #[doc = concat!("# use core::", stringify!($n), "::{MIN, MAX};")]
-            #[doc = concat!("let xs = ", stringify!($name), "::from_array([MIN, MIN +1, -5, 0]);")]
-            #[doc = concat!("assert_eq!(xs.abs(), ", stringify!($name), "::from_array([MIN, MAX, 5, 0]));")]
+            #[doc = concat!("# use core::", stringify!($ty), "::{MIN, MAX};")]
+            /// let xs = Simd::from_array([MIN, MIN +1, -5, 0]);
+            /// assert_eq!(xs.abs(), Simd::from_array([MIN, MAX, 5, 0]));
             /// ```
             #[inline]
             pub fn abs(self) -> Self {
-                const SHR: $n = <$n>::BITS as $n - 1;
+                const SHR: $ty = <$ty>::BITS as $ty - 1;
                 let m = self >> SHR;
                 (self^m) - m
             }
@@ -108,17 +110,17 @@ macro_rules! impl_int_arith {
             /// ```
             /// # #![feature(portable_simd)]
             /// # use core_simd::*;
-            #[doc = concat!("# use core::", stringify!($n), "::{MIN, MAX};")]
-            #[doc = concat!("let xs = ", stringify!($name), "::from_array([MIN, -2, 0, 3]);")]
+            #[doc = concat!("# use core::", stringify!($ty), "::{MIN, MAX};")]
+            /// let xs = Simd::from_array([MIN, -2, 0, 3]);
             /// let unsat = xs.abs();
             /// let sat = xs.saturating_abs();
-            #[doc = concat!("assert_eq!(unsat, ", stringify!($name), "::from_array([MIN, 2, 0, 3]));")]
-            #[doc = concat!("assert_eq!(sat, ", stringify!($name), "::from_array([MAX, 2, 0, 3]));")]
+            /// assert_eq!(unsat, Simd::from_array([MIN, 2, 0, 3]));
+            /// assert_eq!(sat, Simd::from_array([MAX, 2, 0, 3]));
             /// ```
             #[inline]
             pub fn saturating_abs(self) -> Self {
                 // arith shift for -1 or 0 mask based on sign bit, giving 2s complement
-                const SHR: $n = <$n>::BITS as $n - 1;
+                const SHR: $ty = <$ty>::BITS as $ty - 1;
                 let m = self >> SHR;
                 (self^m).saturating_sub(m)
             }
@@ -130,12 +132,12 @@ macro_rules! impl_int_arith {
             /// ```
             /// # #![feature(portable_simd)]
             /// # use core_simd::*;
-            #[doc = concat!("# use core::", stringify!($n), "::{MIN, MAX};")]
-            #[doc = concat!("let x = ", stringify!($name), "::from_array([MIN, -2, 3, MAX]);")]
+            #[doc = concat!("# use core::", stringify!($ty), "::{MIN, MAX};")]
+            /// let x = Simd::from_array([MIN, -2, 3, MAX]);
             /// let unsat = -x;
             /// let sat = x.saturating_neg();
-            #[doc = concat!("assert_eq!(unsat, ", stringify!($name), "::from_array([MIN, 2, -3, MIN + 1]));")]
-            #[doc = concat!("assert_eq!(sat, ", stringify!($name), "::from_array([MAX, 2, -3, MIN + 1]));")]
+            /// assert_eq!(unsat, Simd::from_array([MIN, 2, -3, MIN + 1]));
+            /// assert_eq!(sat, Simd::from_array([MAX, 2, -3, MIN + 1]));
             /// ```
             #[inline]
             pub fn saturating_neg(self) -> Self {
@@ -145,7 +147,5 @@ macro_rules! impl_int_arith {
     }
 }
 
-use crate::vector::*;
-
-impl_uint_arith! { (SimdU8, u8), (SimdU16, u16), (SimdU32, u32), (SimdU64, u64), (SimdUsize, usize) }
-impl_int_arith! { (SimdI8, i8), (SimdI16, i16), (SimdI32, i32), (SimdI64, i64), (SimdIsize, isize) }
+impl_uint_arith! { u8, u16, u32, u64, usize }
+impl_int_arith! { i8, i16, i32, i64, isize }
diff --git a/crates/core_simd/src/reduction.rs b/crates/core_simd/src/reduction.rs
index b9c24d027b62..943d2856e359 100644
--- a/crates/core_simd/src/reduction.rs
+++ b/crates/core_simd/src/reduction.rs
@@ -1,8 +1,10 @@
+use crate::{LaneCount, Simd, SupportedLaneCount};
+
 macro_rules! impl_integer_reductions {
-    { $name:ident, $scalar:ty } => {
-        impl<const LANES: usize> crate::$name<LANES>
+    { $scalar:ty } => {
+        impl<const LANES: usize> Simd<$scalar, LANES>
         where
-            crate::LaneCount<LANES>: crate::SupportedLaneCount,
+            LaneCount<LANES>: SupportedLaneCount,
         {
             /// Horizontal wrapping add.  Returns the sum of the lanes of the vector, with wrapping addition.
             #[inline]
@@ -52,11 +54,22 @@ macro_rules! impl_integer_reductions {
     }
 }
 
+impl_integer_reductions! { i8 }
+impl_integer_reductions! { i16 }
+impl_integer_reductions! { i32 }
+impl_integer_reductions! { i64 }
+impl_integer_reductions! { isize }
+impl_integer_reductions! { u8 }
+impl_integer_reductions! { u16 }
+impl_integer_reductions! { u32 }
+impl_integer_reductions! { u64 }
+impl_integer_reductions! { usize }
+
 macro_rules! impl_float_reductions {
-    { $name:ident, $scalar:ty } => {
-        impl<const LANES: usize> crate::$name<LANES>
+    { $scalar:ty } => {
+        impl<const LANES: usize> Simd<$scalar, LANES>
         where
-            crate::LaneCount<LANES>: crate::SupportedLaneCount,
+            LaneCount<LANES>: SupportedLaneCount,
         {
 
             /// Horizontal add.  Returns the sum of the lanes of the vector.
@@ -102,22 +115,5 @@ macro_rules! impl_float_reductions {
     }
 }
 
-macro_rules! impl_full_mask_reductions {
-    {} => {
-        impl<Element, const LANES: usize> Mask<Element, LANES>
-        where
-            Element: MaskElement,
-            LaneCount<LANES>: SupportedLaneCount,
-        {
-            #[inline]
-            pub fn any(self) -> bool {
-                unsafe { crate::intrinsics::simd_reduce_any(self.to_int()) }
-            }
-
-            #[inline]
-            pub fn all(self) -> bool {
-                unsafe { crate::intrinsics::simd_reduce_all(self.to_int()) }
-            }
-        }
-    }
-}
+impl_float_reductions! { f32 }
+impl_float_reductions! { f64 }
diff --git a/crates/core_simd/src/round.rs b/crates/core_simd/src/round.rs
index c284ade463fc..96d46b9a1232 100644
--- a/crates/core_simd/src/round.rs
+++ b/crates/core_simd/src/round.rs
@@ -1,11 +1,13 @@
+use crate::{LaneCount, Simd, SupportedLaneCount};
+
 macro_rules! implement {
     {
-        $type:ident, $int_type:ident
+        $type:ty, $int_type:ty
     } => {
         #[cfg(feature = "std")]
-        impl<const LANES: usize> crate::$type<LANES>
+        impl<const LANES: usize> Simd<$type, LANES>
         where
-            crate::LaneCount<LANES>: crate::SupportedLaneCount,
+            LaneCount<LANES>: SupportedLaneCount,
         {
             /// Returns the smallest integer greater than or equal to each lane.
             #[must_use = "method returns a new vector and does not mutate the original value"]
@@ -43,9 +45,9 @@ macro_rules! implement {
             }
         }
 
-        impl<const LANES: usize> crate::$type<LANES>
+        impl<const LANES: usize> Simd<$type, LANES>
         where
-            crate::LaneCount<LANES>: crate::SupportedLaneCount,
+            LaneCount<LANES>: SupportedLaneCount,
         {
             /// Rounds toward zero and converts to the same-width integer type, assuming that
             /// the value is finite and fits in that type.
@@ -57,19 +59,19 @@ macro_rules! implement {
             /// * Not be infinite
             /// * Be representable in the return type, after truncating off its fractional part
             #[inline]
-            pub unsafe fn to_int_unchecked(self) -> crate::$int_type<LANES> {
+            pub unsafe fn to_int_unchecked(self) -> Simd<$int_type, LANES> {
                 crate::intrinsics::simd_cast(self)
             }
 
             /// Creates a floating-point vector from an integer vector.  Rounds values that are
             /// not exactly representable.
             #[inline]
-            pub fn round_from_int(value: crate::$int_type<LANES>) -> Self {
+            pub fn round_from_int(value: Simd<$int_type, LANES>) -> Self {
                 unsafe { crate::intrinsics::simd_cast(value) }
             }
         }
     }
 }
 
-implement! { SimdF32, SimdI32 }
-implement! { SimdF64, SimdI64 }
+implement! { f32, i32 }
+implement! { f64, i64 }
diff --git a/crates/core_simd/src/to_bytes.rs b/crates/core_simd/src/to_bytes.rs
index 31d7dfebe1a7..bd818f532118 100644
--- a/crates/core_simd/src/to_bytes.rs
+++ b/crates/core_simd/src/to_bytes.rs
@@ -1,39 +1,39 @@
 macro_rules! impl_to_bytes {
-    { $name:ident, $size:literal } => {
-        impl<const LANES: usize> crate::$name<LANES>
+    { $ty:ty, $size:literal } => {
+        impl<const LANES: usize> crate::Simd<$ty, LANES>
         where
             crate::LaneCount<LANES>: crate::SupportedLaneCount,
             crate::LaneCount<{{ $size * LANES }}>: crate::SupportedLaneCount,
         {
             /// Return the memory representation of this integer as a byte array in native byte
             /// order.
-            pub fn to_ne_bytes(self) -> crate::SimdU8<{{ $size * LANES }}> {
+            pub fn to_ne_bytes(self) -> crate::Simd<u8, {{ $size * LANES }}> {
                 unsafe { core::mem::transmute_copy(&self) }
             }
 
             /// Create a native endian integer value from its memory representation as a byte array
             /// in native endianness.
-            pub fn from_ne_bytes(bytes: crate::SimdU8<{{ $size * LANES }}>) -> Self {
+            pub fn from_ne_bytes(bytes: crate::Simd<u8, {{ $size * LANES }}>) -> Self {
                 unsafe { core::mem::transmute_copy(&bytes) }
             }
         }
     }
 }
 
-impl_to_bytes! { SimdU8, 1 }
-impl_to_bytes! { SimdU16, 2 }
-impl_to_bytes! { SimdU32, 4 }
-impl_to_bytes! { SimdU64, 8 }
+impl_to_bytes! { u8, 1 }
+impl_to_bytes! { u16, 2 }
+impl_to_bytes! { u32, 4 }
+impl_to_bytes! { u64, 8 }
 #[cfg(target_pointer_width = "32")]
-impl_to_bytes! { SimdUsize, 4 }
+impl_to_bytes! { usize, 4 }
 #[cfg(target_pointer_width = "64")]
-impl_to_bytes! { SimdUsize, 8 }
+impl_to_bytes! { usize, 8 }
 
-impl_to_bytes! { SimdI8, 1 }
-impl_to_bytes! { SimdI16, 2 }
-impl_to_bytes! { SimdI32, 4 }
-impl_to_bytes! { SimdI64, 8 }
+impl_to_bytes! { i8, 1 }
+impl_to_bytes! { i16, 2 }
+impl_to_bytes! { i32, 4 }
+impl_to_bytes! { i64, 8 }
 #[cfg(target_pointer_width = "32")]
-impl_to_bytes! { SimdIsize, 4 }
+impl_to_bytes! { isize, 4 }
 #[cfg(target_pointer_width = "64")]
-impl_to_bytes! { SimdIsize, 8 }
+impl_to_bytes! { isize, 8 }
diff --git a/crates/core_simd/src/vector/float.rs b/crates/core_simd/src/vector/float.rs
index 40959d66872c..840ad049d2e3 100644
--- a/crates/core_simd/src/vector/float.rs
+++ b/crates/core_simd/src/vector/float.rs
@@ -7,8 +7,6 @@ use crate::{LaneCount, SupportedLaneCount};
 /// representation. Called from `define_float_vector!`.
 macro_rules! impl_float_vector {
     { $name:ident, $type:ident, $bits_ty:ident, $mask_ty:ident, $mask_impl_ty:ident } => {
-        impl_float_reductions! { $name, $type }
-
         impl<const LANES: usize> $name<LANES>
         where
             LaneCount<LANES>: SupportedLaneCount,
diff --git a/crates/core_simd/src/vector/int.rs b/crates/core_simd/src/vector/int.rs
index 74c4a0f2fb63..3dad2abbe7cc 100644
--- a/crates/core_simd/src/vector/int.rs
+++ b/crates/core_simd/src/vector/int.rs
@@ -5,8 +5,6 @@ use crate::{LaneCount, SupportedLaneCount};
 /// Implements additional integer traits (Eq, Ord, Hash) on the specified vector `$name`, holding multiple `$lanes` of `$type`.
 macro_rules! impl_integer_vector {
     { $name:ident, $type:ty, $mask_ty:ident, $mask_impl_ty:ident } => {
-        impl_integer_reductions! { $name, $type }
-
         impl<const LANES: usize> $name<LANES>
         where
             LaneCount<LANES>: SupportedLaneCount,
diff --git a/crates/core_simd/src/vector/uint.rs b/crates/core_simd/src/vector/uint.rs
index 6dfcbe795939..ba6dab930901 100644
--- a/crates/core_simd/src/vector/uint.rs
+++ b/crates/core_simd/src/vector/uint.rs
@@ -1,12 +1,5 @@
 #![allow(non_camel_case_types)]
 
-/// Implements additional integer traits (Eq, Ord, Hash) on the specified vector `$name`, holding multiple `$lanes` of `$type`.
-macro_rules! impl_unsigned_vector {
-    { $name:ident, $type:ty } => {
-        impl_integer_reductions! { $name, $type }
-    }
-}
-
 /// A SIMD vector of containing `LANES` `u8` values.
 pub type SimdU8<const LANES: usize> = crate::Simd<u8, LANES>;
 
@@ -22,12 +15,6 @@ pub type SimdU64<const LANES: usize> = crate::Simd<u64, LANES>;
 /// A SIMD vector of containing `LANES` `usize` values.
 pub type SimdUsize<const LANES: usize> = crate::Simd<usize, LANES>;
 
-impl_unsigned_vector! { SimdUsize, usize }
-impl_unsigned_vector! { SimdU16, u16 }
-impl_unsigned_vector! { SimdU32, u32 }
-impl_unsigned_vector! { SimdU64, u64 }
-impl_unsigned_vector! { SimdU8, u8 }
-
 /// Vector of two `usize` values
 pub type usizex2 = SimdUsize<2>;
 
diff --git a/crates/core_simd/src/vendor/arm.rs b/crates/core_simd/src/vendor/arm.rs
index 1a1e9bed1e1d..e39173a9c3c4 100644
--- a/crates/core_simd/src/vendor/arm.rs
+++ b/crates/core_simd/src/vendor/arm.rs
@@ -28,26 +28,26 @@ from_transmute! { unsafe u32x4 => uint32x4_t }
 from_transmute! { unsafe i32x2 => int32x2_t }
 from_transmute! { unsafe i32x4 => int32x4_t }
 
-from_transmute! { unsafe SimdU64<1> => uint64x1_t }
+from_transmute! { unsafe Simd<u64, 1> => uint64x1_t }
 from_transmute! { unsafe u64x2 => uint64x2_t }
-from_transmute! { unsafe SimdI64<1> => int64x1_t }
+from_transmute! { unsafe Simd<i64, 1> => int64x1_t }
 from_transmute! { unsafe i64x2 => int64x2_t }
-from_transmute! { unsafe SimdU64<1> => poly64x1_t }
+from_transmute! { unsafe Simd<u64, 1> => poly64x1_t }
 from_transmute! { unsafe u64x2 => poly64x2_t }
 
 #[cfg(target_arch = "arm")]
 mod arm {
     use super::*;
-    from_transmute! { unsafe SimdU8<4> => uint8x4_t }
-    from_transmute! { unsafe SimdI8<4> => int8x4_t }
+    from_transmute! { unsafe Simd<u8, 4> => uint8x4_t }
+    from_transmute! { unsafe Simd<i8, 4> => int8x4_t }
 
-    from_transmute! { unsafe SimdU16<2> => uint16x2_t }
-    from_transmute! { unsafe SimdI16<2> => int16x2_t }
+    from_transmute! { unsafe Simd<u16, 2> => uint16x2_t }
+    from_transmute! { unsafe Simd<i16, 2> => int16x2_t }
 }
 
 #[cfg(target_arch = "aarch64")]
 mod aarch64 {
     use super::*;
-    from_transmute! { unsafe SimdF64<1> => float64x1_t }
+    from_transmute! { unsafe Simd<f64, 1> => float64x1_t }
     from_transmute! { unsafe f64x2 => float64x2_t }
 }
diff --git a/crates/core_simd/src/vendor/x86.rs b/crates/core_simd/src/vendor/x86.rs
index 4de57de057e5..0090c3756481 100644
--- a/crates/core_simd/src/vendor/x86.rs
+++ b/crates/core_simd/src/vendor/x86.rs
@@ -45,10 +45,10 @@ mod p32 {
     use super::*;
     from_transmute! { unsafe usizex4 => __m128i }
     from_transmute! { unsafe usizex8 => __m256i }
-    from_transmute! { unsafe SimdUsize<16> => __m512i }
+    from_transmute! { unsafe Simd<usize, 16> => __m512i }
     from_transmute! { unsafe isizex4 => __m128i }
     from_transmute! { unsafe isizex8 => __m256i }
-    from_transmute! { unsafe SimdIsize<16> => __m512i }
+    from_transmute! { unsafe Simd<isize, 16> => __m512i }
 }
 
 #[cfg(target_pointer_width = "64")]

From 88f79d4a6f609ffec80f83e4c9a71ff8939b4384 Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Sat, 7 Aug 2021 20:20:20 +0000
Subject: [PATCH 213/249] Remove aliases from op trait impls

---
 crates/core_simd/src/ops.rs | 619 +++++++++++++++++-------------------
 1 file changed, 298 insertions(+), 321 deletions(-)

diff --git a/crates/core_simd/src/ops.rs b/crates/core_simd/src/ops.rs
index 67bafd73b144..651498817c38 100644
--- a/crates/core_simd/src/ops.rs
+++ b/crates/core_simd/src/ops.rs
@@ -155,40 +155,40 @@ macro_rules! impl_ref_ops {
 
 /// Automatically implements operators over vectors and scalars for a particular vector.
 macro_rules! impl_op {
-    { impl Add for $type:ident, $scalar:ty } => {
-        impl_op! { @binary $type, $scalar, Add::add, AddAssign::add_assign, simd_add }
+    { impl Add for $scalar:ty } => {
+        impl_op! { @binary $scalar, Add::add, AddAssign::add_assign, simd_add }
     };
-    { impl Sub for $type:ident, $scalar:ty } => {
-        impl_op! { @binary $type, $scalar, Sub::sub, SubAssign::sub_assign, simd_sub }
+    { impl Sub for $scalar:ty } => {
+        impl_op! { @binary $scalar, Sub::sub, SubAssign::sub_assign, simd_sub }
     };
-    { impl Mul for $type:ident, $scalar:ty } => {
-        impl_op! { @binary $type, $scalar, Mul::mul, MulAssign::mul_assign, simd_mul }
+    { impl Mul for $scalar:ty } => {
+        impl_op! { @binary $scalar, Mul::mul, MulAssign::mul_assign, simd_mul }
     };
-    { impl Div for $type:ident, $scalar:ty } => {
-        impl_op! { @binary $type, $scalar, Div::div, DivAssign::div_assign, simd_div }
+    { impl Div for $scalar:ty } => {
+        impl_op! { @binary $scalar, Div::div, DivAssign::div_assign, simd_div }
     };
-    { impl Rem for $type:ident, $scalar:ty } => {
-        impl_op! { @binary $type, $scalar, Rem::rem, RemAssign::rem_assign, simd_rem }
+    { impl Rem for $scalar:ty } => {
+        impl_op! { @binary $scalar, Rem::rem, RemAssign::rem_assign, simd_rem }
     };
-    { impl Shl for $type:ident, $scalar:ty } => {
-        impl_op! { @binary $type, $scalar, Shl::shl, ShlAssign::shl_assign, simd_shl }
+    { impl Shl for $scalar:ty } => {
+        impl_op! { @binary $scalar, Shl::shl, ShlAssign::shl_assign, simd_shl }
     };
-    { impl Shr for $type:ident, $scalar:ty } => {
-        impl_op! { @binary $type, $scalar, Shr::shr, ShrAssign::shr_assign, simd_shr }
+    { impl Shr for $scalar:ty } => {
+        impl_op! { @binary $scalar, Shr::shr, ShrAssign::shr_assign, simd_shr }
     };
-    { impl BitAnd for $type:ident, $scalar:ty } => {
-        impl_op! { @binary $type, $scalar, BitAnd::bitand, BitAndAssign::bitand_assign, simd_and }
+    { impl BitAnd for $scalar:ty } => {
+        impl_op! { @binary $scalar, BitAnd::bitand, BitAndAssign::bitand_assign, simd_and }
     };
-    { impl BitOr for $type:ident, $scalar:ty } => {
-        impl_op! { @binary $type, $scalar, BitOr::bitor, BitOrAssign::bitor_assign, simd_or }
+    { impl BitOr for $scalar:ty } => {
+        impl_op! { @binary $scalar, BitOr::bitor, BitOrAssign::bitor_assign, simd_or }
     };
-    { impl BitXor for $type:ident, $scalar:ty } => {
-        impl_op! { @binary $type, $scalar, BitXor::bitxor, BitXorAssign::bitxor_assign, simd_xor }
+    { impl BitXor for $scalar:ty } => {
+        impl_op! { @binary $scalar, BitXor::bitxor, BitXorAssign::bitxor_assign, simd_xor }
     };
 
-    { impl Not for $type:ident, $scalar:ty } => {
+    { impl Not for $scalar:ty } => {
         impl_ref_ops! {
-            impl<const LANES: usize> core::ops::Not for crate::$type<LANES>
+            impl<const LANES: usize> core::ops::Not for Simd<$scalar, LANES>
             where
                 LaneCount<LANES>: SupportedLaneCount,
             {
@@ -200,9 +200,9 @@ macro_rules! impl_op {
         }
     };
 
-    { impl Neg for $type:ident, $scalar:ty } => {
+    { impl Neg for $scalar:ty } => {
         impl_ref_ops! {
-            impl<const LANES: usize> core::ops::Neg for crate::$type<LANES>
+            impl<const LANES: usize> core::ops::Neg for Simd<$scalar, LANES>
             where
                 LaneCount<LANES>: SupportedLaneCount,
             {
@@ -215,9 +215,9 @@ macro_rules! impl_op {
     };
 
     // generic binary op with assignment when output is `Self`
-    { @binary $type:ident, $scalar:ty, $trait:ident :: $trait_fn:ident, $assign_trait:ident :: $assign_trait_fn:ident, $intrinsic:ident } => {
+    { @binary $scalar:ty, $trait:ident :: $trait_fn:ident, $assign_trait:ident :: $assign_trait_fn:ident, $intrinsic:ident } => {
         impl_ref_ops! {
-            impl<const LANES: usize> core::ops::$trait<Self> for crate::$type<LANES>
+            impl<const LANES: usize> core::ops::$trait<Self> for Simd<$scalar, LANES>
             where
                 LaneCount<LANES>: SupportedLaneCount,
             {
@@ -233,7 +233,7 @@ macro_rules! impl_op {
         }
 
         impl_ref_ops! {
-            impl<const LANES: usize> core::ops::$trait<$scalar> for crate::$type<LANES>
+            impl<const LANES: usize> core::ops::$trait<$scalar> for Simd<$scalar, LANES>
             where
                 LaneCount<LANES>: SupportedLaneCount,
             {
@@ -247,21 +247,21 @@ macro_rules! impl_op {
         }
 
         impl_ref_ops! {
-            impl<const LANES: usize> core::ops::$trait<crate::$type<LANES>> for $scalar
+            impl<const LANES: usize> core::ops::$trait<Simd<$scalar, LANES>> for $scalar
             where
                 LaneCount<LANES>: SupportedLaneCount,
             {
-                type Output = crate::$type<LANES>;
+                type Output = Simd<$scalar, LANES>;
 
                 #[inline]
-                fn $trait_fn(self, rhs: crate::$type<LANES>) -> Self::Output {
-                    core::ops::$trait::$trait_fn(crate::$type::splat(self), rhs)
+                fn $trait_fn(self, rhs: Simd<$scalar, LANES>) -> Self::Output {
+                    core::ops::$trait::$trait_fn(Simd::splat(self), rhs)
                 }
             }
         }
 
         impl_ref_ops! {
-            impl<const LANES: usize> core::ops::$assign_trait<Self> for crate::$type<LANES>
+            impl<const LANES: usize> core::ops::$assign_trait<Self> for Simd<$scalar, LANES>
             where
                 LaneCount<LANES>: SupportedLaneCount,
             {
@@ -275,7 +275,7 @@ macro_rules! impl_op {
         }
 
         impl_ref_ops! {
-            impl<const LANES: usize> core::ops::$assign_trait<$scalar> for crate::$type<LANES>
+            impl<const LANES: usize> core::ops::$assign_trait<$scalar> for Simd<$scalar, LANES>
             where
                 LaneCount<LANES>: SupportedLaneCount,
             {
@@ -290,377 +290,354 @@ macro_rules! impl_op {
 
 /// Implements floating-point operators for the provided types.
 macro_rules! impl_float_ops {
-    { $($scalar:ty => $($vector:ident),*;)* } => {
-        $( // scalar
-            $( // vector
-                impl_op! { impl Add for $vector, $scalar }
-                impl_op! { impl Sub for $vector, $scalar }
-                impl_op! { impl Mul for $vector, $scalar }
-                impl_op! { impl Div for $vector, $scalar }
-                impl_op! { impl Rem for $vector, $scalar }
-                impl_op! { impl Neg for $vector, $scalar }
-            )*
+    { $($scalar:ty),* } => {
+        $(
+            impl_op! { impl Add for $scalar }
+            impl_op! { impl Sub for $scalar }
+            impl_op! { impl Mul for $scalar }
+            impl_op! { impl Div for $scalar }
+            impl_op! { impl Rem for $scalar }
+            impl_op! { impl Neg for $scalar }
         )*
     };
 }
 
 /// Implements unsigned integer operators for the provided types.
 macro_rules! impl_unsigned_int_ops {
-    { $($scalar:ty => $($vector:ident),*;)* } => {
-        $( // scalar
-            $( // vector
-                impl_op! { impl Add for $vector, $scalar }
-                impl_op! { impl Sub for $vector, $scalar }
-                impl_op! { impl Mul for $vector, $scalar }
-                impl_op! { impl BitAnd for $vector, $scalar }
-                impl_op! { impl BitOr  for $vector, $scalar }
-                impl_op! { impl BitXor for $vector, $scalar }
-                impl_op! { impl Not for $vector, $scalar }
+    { $($scalar:ty),* } => {
+        $(
+            impl_op! { impl Add for $scalar }
+            impl_op! { impl Sub for $scalar }
+            impl_op! { impl Mul for $scalar }
+            impl_op! { impl BitAnd for $scalar }
+            impl_op! { impl BitOr  for $scalar }
+            impl_op! { impl BitXor for $scalar }
+            impl_op! { impl Not for $scalar }
 
-                // Integers panic on divide by 0
-                impl_ref_ops! {
-                    impl<const LANES: usize> core::ops::Div<Self> for crate::$vector<LANES>
-                    where
-                        LaneCount<LANES>: SupportedLaneCount,
-                    {
-                        type Output = Self;
+            // Integers panic on divide by 0
+            impl_ref_ops! {
+                impl<const LANES: usize> core::ops::Div<Self> for Simd<$scalar, LANES>
+                where
+                    LaneCount<LANES>: SupportedLaneCount,
+                {
+                    type Output = Self;
 
-                        #[inline]
-                        fn div(self, rhs: Self) -> Self::Output {
-                            if rhs.as_array()
-                                .iter()
-                                .any(|x| *x == 0)
-                            {
-                                panic!("attempt to divide by zero");
-                            }
+                    #[inline]
+                    fn div(self, rhs: Self) -> Self::Output {
+                        if rhs.as_array()
+                            .iter()
+                            .any(|x| *x == 0)
+                        {
+                            panic!("attempt to divide by zero");
+                        }
 
-                            // Guards for div(MIN, -1),
-                            // this check only applies to signed ints
-                            if <$scalar>::MIN != 0 && self.as_array().iter()
-                                    .zip(rhs.as_array().iter())
-                                    .any(|(x,y)| *x == <$scalar>::MIN && *y == -1 as _) {
+                        // Guards for div(MIN, -1),
+                        // this check only applies to signed ints
+                        if <$scalar>::MIN != 0 && self.as_array().iter()
+                                .zip(rhs.as_array().iter())
+                                .any(|(x,y)| *x == <$scalar>::MIN && *y == -1 as _) {
+                            panic!("attempt to divide with overflow");
+                        }
+                        unsafe { crate::intrinsics::simd_div(self, rhs) }
+                    }
+                }
+            }
+
+            impl_ref_ops! {
+                impl<const LANES: usize> core::ops::Div<$scalar> for Simd<$scalar, LANES>
+                where
+                    LaneCount<LANES>: SupportedLaneCount,
+                {
+                    type Output = Self;
+
+                    #[inline]
+                    fn div(self, rhs: $scalar) -> Self::Output {
+                        if rhs == 0 {
+                            panic!("attempt to divide by zero");
+                        }
+                        if <$scalar>::MIN != 0 &&
+                            self.as_array().iter().any(|x| *x == <$scalar>::MIN) &&
+                            rhs == -1 as _ {
                                 panic!("attempt to divide with overflow");
-                            }
-                            unsafe { crate::intrinsics::simd_div(self, rhs) }
                         }
+                        let rhs = Self::splat(rhs);
+                        unsafe { crate::intrinsics::simd_div(self, rhs) }
                     }
                 }
+            }
 
-                impl_ref_ops! {
-                    impl<const LANES: usize> core::ops::Div<$scalar> for crate::$vector<LANES>
-                    where
-                        LaneCount<LANES>: SupportedLaneCount,
-                    {
-                        type Output = Self;
+            impl_ref_ops! {
+                impl<const LANES: usize> core::ops::Div<Simd<$scalar, LANES>> for $scalar
+                where
+                    LaneCount<LANES>: SupportedLaneCount,
+                {
+                    type Output = Simd<$scalar, LANES>;
 
-                        #[inline]
-                        fn div(self, rhs: $scalar) -> Self::Output {
-                            if rhs == 0 {
-                                panic!("attempt to divide by zero");
-                            }
-                            if <$scalar>::MIN != 0 &&
-                                self.as_array().iter().any(|x| *x == <$scalar>::MIN) &&
-                                rhs == -1 as _ {
-                                    panic!("attempt to divide with overflow");
-                            }
-                            let rhs = Self::splat(rhs);
-                            unsafe { crate::intrinsics::simd_div(self, rhs) }
-                        }
+                    #[inline]
+                    fn div(self, rhs: Simd<$scalar, LANES>) -> Self::Output {
+                        Simd::splat(self) / rhs
                     }
                 }
+            }
 
-                impl_ref_ops! {
-                    impl<const LANES: usize> core::ops::Div<crate::$vector<LANES>> for $scalar
-                    where
-                        LaneCount<LANES>: SupportedLaneCount,
-                    {
-                        type Output = crate::$vector<LANES>;
-
-                        #[inline]
-                        fn div(self, rhs: crate::$vector<LANES>) -> Self::Output {
-                            crate::$vector::splat(self) / rhs
-                        }
+            impl_ref_ops! {
+                impl<const LANES: usize> core::ops::DivAssign<Self> for Simd<$scalar, LANES>
+                where
+                    LaneCount<LANES>: SupportedLaneCount,
+                {
+                    #[inline]
+                    fn div_assign(&mut self, rhs: Self) {
+                        *self = *self / rhs;
                     }
                 }
+            }
 
-                impl_ref_ops! {
-                    impl<const LANES: usize> core::ops::DivAssign<Self> for crate::$vector<LANES>
-                    where
-                        LaneCount<LANES>: SupportedLaneCount,
-                    {
-                        #[inline]
-                        fn div_assign(&mut self, rhs: Self) {
-                            *self = *self / rhs;
-                        }
+            impl_ref_ops! {
+                impl<const LANES: usize> core::ops::DivAssign<$scalar> for Simd<$scalar, LANES>
+                where
+                    LaneCount<LANES>: SupportedLaneCount,
+                {
+                    #[inline]
+                    fn div_assign(&mut self, rhs: $scalar) {
+                        *self = *self / rhs;
                     }
                 }
+            }
 
-                impl_ref_ops! {
-                    impl<const LANES: usize> core::ops::DivAssign<$scalar> for crate::$vector<LANES>
-                    where
-                        LaneCount<LANES>: SupportedLaneCount,
-                    {
-                        #[inline]
-                        fn div_assign(&mut self, rhs: $scalar) {
-                            *self = *self / rhs;
+            // remainder panics on zero divisor
+            impl_ref_ops! {
+                impl<const LANES: usize> core::ops::Rem<Self> for Simd<$scalar, LANES>
+                where
+                    LaneCount<LANES>: SupportedLaneCount,
+                {
+                    type Output = Self;
+
+                    #[inline]
+                    fn rem(self, rhs: Self) -> Self::Output {
+                        if rhs.as_array()
+                            .iter()
+                            .any(|x| *x == 0)
+                        {
+                            panic!("attempt to calculate the remainder with a divisor of zero");
                         }
+
+                        // Guards for rem(MIN, -1)
+                        // this branch applies the check only to signed ints
+                        if <$scalar>::MIN != 0 && self.as_array().iter()
+                                .zip(rhs.as_array().iter())
+                                .any(|(x,y)| *x == <$scalar>::MIN && *y == -1 as _) {
+                            panic!("attempt to calculate the remainder with overflow");
+                        }
+                        unsafe { crate::intrinsics::simd_rem(self, rhs) }
                     }
                 }
+            }
 
-                // remainder panics on zero divisor
-                impl_ref_ops! {
-                    impl<const LANES: usize> core::ops::Rem<Self> for crate::$vector<LANES>
-                    where
-                        LaneCount<LANES>: SupportedLaneCount,
-                    {
-                        type Output = Self;
+            impl_ref_ops! {
+                impl<const LANES: usize> core::ops::Rem<$scalar> for Simd<$scalar, LANES>
+                where
+                    LaneCount<LANES>: SupportedLaneCount,
+                {
+                    type Output = Self;
 
-                        #[inline]
-                        fn rem(self, rhs: Self) -> Self::Output {
-                            if rhs.as_array()
-                                .iter()
-                                .any(|x| *x == 0)
-                            {
-                                panic!("attempt to calculate the remainder with a divisor of zero");
-                            }
-
-                            // Guards for rem(MIN, -1)
-                            // this branch applies the check only to signed ints
-                            if <$scalar>::MIN != 0 && self.as_array().iter()
-                                    .zip(rhs.as_array().iter())
-                                    .any(|(x,y)| *x == <$scalar>::MIN && *y == -1 as _) {
+                    #[inline]
+                    fn rem(self, rhs: $scalar) -> Self::Output {
+                        if rhs == 0 {
+                            panic!("attempt to calculate the remainder with a divisor of zero");
+                        }
+                        if <$scalar>::MIN != 0 &&
+                            self.as_array().iter().any(|x| *x == <$scalar>::MIN) &&
+                            rhs == -1 as _ {
                                 panic!("attempt to calculate the remainder with overflow");
-                            }
-                            unsafe { crate::intrinsics::simd_rem(self, rhs) }
                         }
+                        let rhs = Self::splat(rhs);
+                        unsafe { crate::intrinsics::simd_rem(self, rhs) }
                     }
                 }
+            }
 
-                impl_ref_ops! {
-                    impl<const LANES: usize> core::ops::Rem<$scalar> for crate::$vector<LANES>
-                    where
-                        LaneCount<LANES>: SupportedLaneCount,
-                    {
-                        type Output = Self;
+            impl_ref_ops! {
+                impl<const LANES: usize> core::ops::Rem<Simd<$scalar, LANES>> for $scalar
+                where
+                    LaneCount<LANES>: SupportedLaneCount,
+                {
+                    type Output = Simd<$scalar, LANES>;
 
-                        #[inline]
-                        fn rem(self, rhs: $scalar) -> Self::Output {
-                            if rhs == 0 {
-                                panic!("attempt to calculate the remainder with a divisor of zero");
-                            }
-                            if <$scalar>::MIN != 0 &&
-                                self.as_array().iter().any(|x| *x == <$scalar>::MIN) &&
-                                rhs == -1 as _ {
-                                    panic!("attempt to calculate the remainder with overflow");
-                            }
-                            let rhs = Self::splat(rhs);
-                            unsafe { crate::intrinsics::simd_rem(self, rhs) }
-                        }
+                    #[inline]
+                    fn rem(self, rhs: Simd<$scalar, LANES>) -> Self::Output {
+                        Simd::splat(self) % rhs
                     }
                 }
+            }
 
-                impl_ref_ops! {
-                    impl<const LANES: usize> core::ops::Rem<crate::$vector<LANES>> for $scalar
-                    where
-                        LaneCount<LANES>: SupportedLaneCount,
-                    {
-                        type Output = crate::$vector<LANES>;
-
-                        #[inline]
-                        fn rem(self, rhs: crate::$vector<LANES>) -> Self::Output {
-                            crate::$vector::splat(self) % rhs
-                        }
+            impl_ref_ops! {
+                impl<const LANES: usize> core::ops::RemAssign<Self> for Simd<$scalar, LANES>
+                where
+                    LaneCount<LANES>: SupportedLaneCount,
+                {
+                    #[inline]
+                    fn rem_assign(&mut self, rhs: Self) {
+                        *self = *self % rhs;
                     }
                 }
+            }
 
-                impl_ref_ops! {
-                    impl<const LANES: usize> core::ops::RemAssign<Self> for crate::$vector<LANES>
-                    where
-                        LaneCount<LANES>: SupportedLaneCount,
-                    {
-                        #[inline]
-                        fn rem_assign(&mut self, rhs: Self) {
-                            *self = *self % rhs;
-                        }
+            impl_ref_ops! {
+                impl<const LANES: usize> core::ops::RemAssign<$scalar> for Simd<$scalar, LANES>
+                where
+                    LaneCount<LANES>: SupportedLaneCount,
+                {
+                    #[inline]
+                    fn rem_assign(&mut self, rhs: $scalar) {
+                        *self = *self % rhs;
                     }
                 }
+            }
 
-                impl_ref_ops! {
-                    impl<const LANES: usize> core::ops::RemAssign<$scalar> for crate::$vector<LANES>
-                    where
-                        LaneCount<LANES>: SupportedLaneCount,
-                    {
-                        #[inline]
-                        fn rem_assign(&mut self, rhs: $scalar) {
-                            *self = *self % rhs;
+            // shifts panic on overflow
+            impl_ref_ops! {
+                impl<const LANES: usize> core::ops::Shl<Self> for Simd<$scalar, LANES>
+                where
+                    LaneCount<LANES>: SupportedLaneCount,
+                {
+                    type Output = Self;
+
+                    #[inline]
+                    fn shl(self, rhs: Self) -> Self::Output {
+                        // TODO there is probably a better way of doing this
+                        if rhs.as_array()
+                            .iter()
+                            .copied()
+                            .any(invalid_shift_rhs)
+                        {
+                            panic!("attempt to shift left with overflow");
                         }
+                        unsafe { crate::intrinsics::simd_shl(self, rhs) }
                     }
                 }
+            }
 
-                // shifts panic on overflow
-                impl_ref_ops! {
-                    impl<const LANES: usize> core::ops::Shl<Self> for crate::$vector<LANES>
-                    where
-                        LaneCount<LANES>: SupportedLaneCount,
-                    {
-                        type Output = Self;
+            impl_ref_ops! {
+                impl<const LANES: usize> core::ops::Shl<$scalar> for Simd<$scalar, LANES>
+                where
+                    LaneCount<LANES>: SupportedLaneCount,
+                {
+                    type Output = Self;
 
-                        #[inline]
-                        fn shl(self, rhs: Self) -> Self::Output {
-                            // TODO there is probably a better way of doing this
-                            if rhs.as_array()
-                                .iter()
-                                .copied()
-                                .any(invalid_shift_rhs)
-                            {
-                                panic!("attempt to shift left with overflow");
-                            }
-                            unsafe { crate::intrinsics::simd_shl(self, rhs) }
+                    #[inline]
+                    fn shl(self, rhs: $scalar) -> Self::Output {
+                        if invalid_shift_rhs(rhs) {
+                            panic!("attempt to shift left with overflow");
                         }
+                        let rhs = Self::splat(rhs);
+                        unsafe { crate::intrinsics::simd_shl(self, rhs) }
                     }
                 }
+            }
 
-                impl_ref_ops! {
-                    impl<const LANES: usize> core::ops::Shl<$scalar> for crate::$vector<LANES>
-                    where
-                        LaneCount<LANES>: SupportedLaneCount,
-                    {
-                        type Output = Self;
 
-                        #[inline]
-                        fn shl(self, rhs: $scalar) -> Self::Output {
-                            if invalid_shift_rhs(rhs) {
-                                panic!("attempt to shift left with overflow");
-                            }
-                            let rhs = Self::splat(rhs);
-                            unsafe { crate::intrinsics::simd_shl(self, rhs) }
-                        }
+            impl_ref_ops! {
+                impl<const LANES: usize> core::ops::ShlAssign<Self> for Simd<$scalar, LANES>
+                where
+                    LaneCount<LANES>: SupportedLaneCount,
+                {
+                    #[inline]
+                    fn shl_assign(&mut self, rhs: Self) {
+                        *self = *self << rhs;
                     }
                 }
+            }
 
-
-                impl_ref_ops! {
-                    impl<const LANES: usize> core::ops::ShlAssign<Self> for crate::$vector<LANES>
-                    where
-                        LaneCount<LANES>: SupportedLaneCount,
-                    {
-                        #[inline]
-                        fn shl_assign(&mut self, rhs: Self) {
-                            *self = *self << rhs;
-                        }
+            impl_ref_ops! {
+                impl<const LANES: usize> core::ops::ShlAssign<$scalar> for Simd<$scalar, LANES>
+                where
+                    LaneCount<LANES>: SupportedLaneCount,
+                {
+                    #[inline]
+                    fn shl_assign(&mut self, rhs: $scalar) {
+                        *self = *self << rhs;
                     }
                 }
+            }
 
-                impl_ref_ops! {
-                    impl<const LANES: usize> core::ops::ShlAssign<$scalar> for crate::$vector<LANES>
-                    where
-                        LaneCount<LANES>: SupportedLaneCount,
-                    {
-                        #[inline]
-                        fn shl_assign(&mut self, rhs: $scalar) {
-                            *self = *self << rhs;
+            impl_ref_ops! {
+                impl<const LANES: usize> core::ops::Shr<Self> for Simd<$scalar, LANES>
+                where
+                    LaneCount<LANES>: SupportedLaneCount,
+                {
+                    type Output = Self;
+
+                    #[inline]
+                    fn shr(self, rhs: Self) -> Self::Output {
+                        // TODO there is probably a better way of doing this
+                        if rhs.as_array()
+                            .iter()
+                            .copied()
+                            .any(invalid_shift_rhs)
+                        {
+                            panic!("attempt to shift with overflow");
                         }
+                        unsafe { crate::intrinsics::simd_shr(self, rhs) }
                     }
                 }
+            }
 
-                impl_ref_ops! {
-                    impl<const LANES: usize> core::ops::Shr<Self> for crate::$vector<LANES>
-                    where
-                        LaneCount<LANES>: SupportedLaneCount,
-                    {
-                        type Output = Self;
+            impl_ref_ops! {
+                impl<const LANES: usize> core::ops::Shr<$scalar> for Simd<$scalar, LANES>
+                where
+                    LaneCount<LANES>: SupportedLaneCount,
+                {
+                    type Output = Self;
 
-                        #[inline]
-                        fn shr(self, rhs: Self) -> Self::Output {
-                            // TODO there is probably a better way of doing this
-                            if rhs.as_array()
-                                .iter()
-                                .copied()
-                                .any(invalid_shift_rhs)
-                            {
-                                panic!("attempt to shift with overflow");
-                            }
-                            unsafe { crate::intrinsics::simd_shr(self, rhs) }
+                    #[inline]
+                    fn shr(self, rhs: $scalar) -> Self::Output {
+                        if invalid_shift_rhs(rhs) {
+                            panic!("attempt to shift with overflow");
                         }
+                        let rhs = Self::splat(rhs);
+                        unsafe { crate::intrinsics::simd_shr(self, rhs) }
                     }
                 }
+            }
 
-                impl_ref_ops! {
-                    impl<const LANES: usize> core::ops::Shr<$scalar> for crate::$vector<LANES>
-                    where
-                        LaneCount<LANES>: SupportedLaneCount,
-                    {
-                        type Output = Self;
 
-                        #[inline]
-                        fn shr(self, rhs: $scalar) -> Self::Output {
-                            if invalid_shift_rhs(rhs) {
-                                panic!("attempt to shift with overflow");
-                            }
-                            let rhs = Self::splat(rhs);
-                            unsafe { crate::intrinsics::simd_shr(self, rhs) }
-                        }
+            impl_ref_ops! {
+                impl<const LANES: usize> core::ops::ShrAssign<Self> for Simd<$scalar, LANES>
+                where
+                    LaneCount<LANES>: SupportedLaneCount,
+                {
+                    #[inline]
+                    fn shr_assign(&mut self, rhs: Self) {
+                        *self = *self >> rhs;
                     }
                 }
+            }
 
-
-                impl_ref_ops! {
-                    impl<const LANES: usize> core::ops::ShrAssign<Self> for crate::$vector<LANES>
-                    where
-                        LaneCount<LANES>: SupportedLaneCount,
-                    {
-                        #[inline]
-                        fn shr_assign(&mut self, rhs: Self) {
-                            *self = *self >> rhs;
-                        }
+            impl_ref_ops! {
+                impl<const LANES: usize> core::ops::ShrAssign<$scalar> for Simd<$scalar, LANES>
+                where
+                    LaneCount<LANES>: SupportedLaneCount,
+                {
+                    #[inline]
+                    fn shr_assign(&mut self, rhs: $scalar) {
+                        *self = *self >> rhs;
                     }
                 }
-
-                impl_ref_ops! {
-                    impl<const LANES: usize> core::ops::ShrAssign<$scalar> for crate::$vector<LANES>
-                    where
-                        LaneCount<LANES>: SupportedLaneCount,
-                    {
-                        #[inline]
-                        fn shr_assign(&mut self, rhs: $scalar) {
-                            *self = *self >> rhs;
-                        }
-                    }
-                }
-            )*
+            }
         )*
     };
 }
 
 /// Implements unsigned integer operators for the provided types.
 macro_rules! impl_signed_int_ops {
-    { $($scalar:ty => $($vector:ident),*;)* } => {
-        impl_unsigned_int_ops! { $($scalar => $($vector),*;)* }
+    { $($scalar:ty),* } => {
+        impl_unsigned_int_ops! { $($scalar),* }
         $( // scalar
-            $( // vector
-                impl_op! { impl Neg for $vector, $scalar }
-            )*
+            impl_op! { impl Neg for $scalar }
         )*
     };
 }
 
-impl_unsigned_int_ops! {
-    u8 => SimdU8;
-    u16 => SimdU16;
-    u32 => SimdU32;
-    u64 => SimdU64;
-    usize => SimdUsize;
-}
-
-impl_signed_int_ops! {
-    i8 => SimdI8;
-    i16 => SimdI16;
-    i32 => SimdI32;
-    i64 => SimdI64;
-    isize => SimdIsize;
-}
-
-impl_float_ops! {
-    f32 => SimdF32;
-    f64 => SimdF64;
-}
+impl_unsigned_int_ops! { u8, u16, u32, u64, usize }
+impl_signed_int_ops! { i8, i16, i32, i64, isize }
+impl_float_ops! { f32, f64 }

From f7f29683a8f63e967c471c8cc27c7e43709bc6d1 Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Sat, 7 Aug 2021 20:38:41 +0000
Subject: [PATCH 214/249] Remove aliases from most tests

---
 crates/core_simd/tests/f32_ops.rs    |   2 +-
 crates/core_simd/tests/f64_ops.rs    |   2 +-
 crates/core_simd/tests/i16_ops.rs    |   2 +-
 crates/core_simd/tests/i32_ops.rs    |   2 +-
 crates/core_simd/tests/i64_ops.rs    |   2 +-
 crates/core_simd/tests/i8_ops.rs     |   2 +-
 crates/core_simd/tests/isize_ops.rs  |   2 +-
 crates/core_simd/tests/ops_macros.rs | 102 ++++++++++++++-------------
 crates/core_simd/tests/u16_ops.rs    |   2 +-
 crates/core_simd/tests/u32_ops.rs    |   2 +-
 crates/core_simd/tests/u64_ops.rs    |   2 +-
 crates/core_simd/tests/u8_ops.rs     |   2 +-
 crates/core_simd/tests/usize_ops.rs  |   2 +-
 13 files changed, 64 insertions(+), 62 deletions(-)

diff --git a/crates/core_simd/tests/f32_ops.rs b/crates/core_simd/tests/f32_ops.rs
index 98283110097e..414a832b1be4 100644
--- a/crates/core_simd/tests/f32_ops.rs
+++ b/crates/core_simd/tests/f32_ops.rs
@@ -2,4 +2,4 @@
 
 #[macro_use]
 mod ops_macros;
-impl_float_tests! { SimdF32, f32, i32 }
+impl_float_tests! { f32, i32 }
diff --git a/crates/core_simd/tests/f64_ops.rs b/crates/core_simd/tests/f64_ops.rs
index 0818b0c5c5a9..e0a1fa33f332 100644
--- a/crates/core_simd/tests/f64_ops.rs
+++ b/crates/core_simd/tests/f64_ops.rs
@@ -2,4 +2,4 @@
 
 #[macro_use]
 mod ops_macros;
-impl_float_tests! { SimdF64, f64, i64 }
+impl_float_tests! { f64, i64 }
diff --git a/crates/core_simd/tests/i16_ops.rs b/crates/core_simd/tests/i16_ops.rs
index 33d92faa5956..f6c5d74fbbcc 100644
--- a/crates/core_simd/tests/i16_ops.rs
+++ b/crates/core_simd/tests/i16_ops.rs
@@ -2,4 +2,4 @@
 
 #[macro_use]
 mod ops_macros;
-impl_signed_tests! { SimdI16, i16 }
+impl_signed_tests! { i16 }
diff --git a/crates/core_simd/tests/i32_ops.rs b/crates/core_simd/tests/i32_ops.rs
index 481bca23e834..69a831c52a3f 100644
--- a/crates/core_simd/tests/i32_ops.rs
+++ b/crates/core_simd/tests/i32_ops.rs
@@ -2,4 +2,4 @@
 
 #[macro_use]
 mod ops_macros;
-impl_signed_tests! { SimdI32, i32 }
+impl_signed_tests! { i32 }
diff --git a/crates/core_simd/tests/i64_ops.rs b/crates/core_simd/tests/i64_ops.rs
index 5ab0614c8480..37ac08117424 100644
--- a/crates/core_simd/tests/i64_ops.rs
+++ b/crates/core_simd/tests/i64_ops.rs
@@ -2,4 +2,4 @@
 
 #[macro_use]
 mod ops_macros;
-impl_signed_tests! { SimdI64, i64 }
+impl_signed_tests! { i64 }
diff --git a/crates/core_simd/tests/i8_ops.rs b/crates/core_simd/tests/i8_ops.rs
index 0db9ee47a9e2..11e4a5cd6a9a 100644
--- a/crates/core_simd/tests/i8_ops.rs
+++ b/crates/core_simd/tests/i8_ops.rs
@@ -2,4 +2,4 @@
 
 #[macro_use]
 mod ops_macros;
-impl_signed_tests! { SimdI8, i8 }
+impl_signed_tests! { i8 }
diff --git a/crates/core_simd/tests/isize_ops.rs b/crates/core_simd/tests/isize_ops.rs
index 8f5470b685c9..5cc9de2b7ff8 100644
--- a/crates/core_simd/tests/isize_ops.rs
+++ b/crates/core_simd/tests/isize_ops.rs
@@ -2,4 +2,4 @@
 
 #[macro_use]
 mod ops_macros;
-impl_signed_tests! { SimdIsize, isize }
+impl_signed_tests! { isize }
diff --git a/crates/core_simd/tests/ops_macros.rs b/crates/core_simd/tests/ops_macros.rs
index 81553c34aa77..0c45ea2367c9 100644
--- a/crates/core_simd/tests/ops_macros.rs
+++ b/crates/core_simd/tests/ops_macros.rs
@@ -3,19 +3,19 @@
 /// Compares the vector operation to the equivalent scalar operation.
 #[macro_export]
 macro_rules! impl_unary_op_test {
-    { $vector:ty, $scalar:ty, $trait:ident :: $fn:ident, $scalar_fn:expr } => {
+    { $scalar:ty, $trait:ident :: $fn:ident, $scalar_fn:expr } => {
         test_helpers::test_lanes! {
             fn $fn<const LANES: usize>() {
                 test_helpers::test_unary_elementwise(
-                    &<$vector as core::ops::$trait>::$fn,
+                    &<core_simd::Simd<$scalar, LANES> as core::ops::$trait>::$fn,
                     &$scalar_fn,
                     &|_| true,
                 );
             }
         }
     };
-    { $vector:ty, $scalar:ty, $trait:ident :: $fn:ident } => {
-        impl_unary_op_test! { $vector, $scalar, $trait::$fn, <$scalar as core::ops::$trait>::$fn }
+    { $scalar:ty, $trait:ident :: $fn:ident } => {
+        impl_unary_op_test! { $scalar, $trait::$fn, <$scalar as core::ops::$trait>::$fn }
     };
 }
 
@@ -24,14 +24,15 @@ macro_rules! impl_unary_op_test {
 /// Compares the vector operation to the equivalent scalar operation.
 #[macro_export]
 macro_rules! impl_binary_op_test {
-    { $vector:ty, $scalar:ty, $trait:ident :: $fn:ident, $trait_assign:ident :: $fn_assign:ident, $scalar_fn:expr } => {
+    { $scalar:ty, $trait:ident :: $fn:ident, $trait_assign:ident :: $fn_assign:ident, $scalar_fn:expr } => {
         mod $fn {
             use super::*;
+            use core_simd::Simd;
 
             test_helpers::test_lanes! {
                 fn normal<const LANES: usize>() {
                     test_helpers::test_binary_elementwise(
-                        &<$vector as core::ops::$trait>::$fn,
+                        &<Simd<$scalar, LANES> as core::ops::$trait>::$fn,
                         &$scalar_fn,
                         &|_, _| true,
                     );
@@ -39,7 +40,7 @@ macro_rules! impl_binary_op_test {
 
                 fn scalar_rhs<const LANES: usize>() {
                     test_helpers::test_binary_scalar_rhs_elementwise(
-                        &<$vector as core::ops::$trait<$scalar>>::$fn,
+                        &<Simd<$scalar, LANES> as core::ops::$trait<$scalar>>::$fn,
                         &$scalar_fn,
                         &|_, _| true,
                     );
@@ -47,7 +48,7 @@ macro_rules! impl_binary_op_test {
 
                 fn scalar_lhs<const LANES: usize>() {
                     test_helpers::test_binary_scalar_lhs_elementwise(
-                        &<$scalar as core::ops::$trait<$vector>>::$fn,
+                        &<$scalar as core::ops::$trait<Simd<$scalar, LANES>>>::$fn,
                         &$scalar_fn,
                         &|_, _| true,
                     );
@@ -55,7 +56,7 @@ macro_rules! impl_binary_op_test {
 
                 fn assign<const LANES: usize>() {
                     test_helpers::test_binary_elementwise(
-                        &|mut a, b| { <$vector as core::ops::$trait_assign>::$fn_assign(&mut a, b); a },
+                        &|mut a, b| { <Simd<$scalar, LANES> as core::ops::$trait_assign>::$fn_assign(&mut a, b); a },
                         &$scalar_fn,
                         &|_, _| true,
                     );
@@ -63,7 +64,7 @@ macro_rules! impl_binary_op_test {
 
                 fn assign_scalar_rhs<const LANES: usize>() {
                     test_helpers::test_binary_scalar_rhs_elementwise(
-                        &|mut a, b| { <$vector as core::ops::$trait_assign<$scalar>>::$fn_assign(&mut a, b); a },
+                        &|mut a, b| { <Simd<$scalar, LANES> as core::ops::$trait_assign<$scalar>>::$fn_assign(&mut a, b); a },
                         &$scalar_fn,
                         &|_, _| true,
                     );
@@ -71,8 +72,8 @@ macro_rules! impl_binary_op_test {
             }
         }
     };
-    { $vector:ty, $scalar:ty, $trait:ident :: $fn:ident, $trait_assign:ident :: $fn_assign:ident } => {
-        impl_binary_op_test! { $vector, $scalar, $trait::$fn, $trait_assign::$fn_assign, <$scalar as core::ops::$trait>::$fn }
+    { $scalar:ty, $trait:ident :: $fn:ident, $trait_assign:ident :: $fn_assign:ident } => {
+        impl_binary_op_test! { $scalar, $trait::$fn, $trait_assign::$fn_assign, <$scalar as core::ops::$trait>::$fn }
     };
 }
 
@@ -84,14 +85,15 @@ macro_rules! impl_binary_op_test {
 /// Compares the vector operation to the equivalent scalar operation.
 #[macro_export]
 macro_rules! impl_binary_checked_op_test {
-    { $vector:ty, $scalar:ty, $trait:ident :: $fn:ident, $trait_assign:ident :: $fn_assign:ident, $scalar_fn:expr, $check_fn:expr } => {
+    { $scalar:ty, $trait:ident :: $fn:ident, $trait_assign:ident :: $fn_assign:ident, $scalar_fn:expr, $check_fn:expr } => {
         mod $fn {
             use super::*;
+            use core_simd::Simd;
 
             test_helpers::test_lanes! {
                 fn normal<const LANES: usize>() {
                     test_helpers::test_binary_elementwise(
-                        &<$vector as core::ops::$trait>::$fn,
+                        &<Simd<$scalar, LANES> as core::ops::$trait>::$fn,
                         &$scalar_fn,
                         &|x, y| x.iter().zip(y.iter()).all(|(x, y)| $check_fn(*x, *y)),
                     );
@@ -99,7 +101,7 @@ macro_rules! impl_binary_checked_op_test {
 
                 fn scalar_rhs<const LANES: usize>() {
                     test_helpers::test_binary_scalar_rhs_elementwise(
-                        &<$vector as core::ops::$trait<$scalar>>::$fn,
+                        &<Simd<$scalar, LANES> as core::ops::$trait<$scalar>>::$fn,
                         &$scalar_fn,
                         &|x, y| x.iter().all(|x| $check_fn(*x, y)),
                     );
@@ -107,7 +109,7 @@ macro_rules! impl_binary_checked_op_test {
 
                 fn scalar_lhs<const LANES: usize>() {
                     test_helpers::test_binary_scalar_lhs_elementwise(
-                        &<$scalar as core::ops::$trait<$vector>>::$fn,
+                        &<$scalar as core::ops::$trait<Simd<$scalar, LANES>>>::$fn,
                         &$scalar_fn,
                         &|x, y| y.iter().all(|y| $check_fn(x, *y)),
                     );
@@ -115,7 +117,7 @@ macro_rules! impl_binary_checked_op_test {
 
                 fn assign<const LANES: usize>() {
                     test_helpers::test_binary_elementwise(
-                        &|mut a, b| { <$vector as core::ops::$trait_assign>::$fn_assign(&mut a, b); a },
+                        &|mut a, b| { <Simd<$scalar, LANES> as core::ops::$trait_assign>::$fn_assign(&mut a, b); a },
                         &$scalar_fn,
                         &|x, y| x.iter().zip(y.iter()).all(|(x, y)| $check_fn(*x, *y)),
                     )
@@ -123,7 +125,7 @@ macro_rules! impl_binary_checked_op_test {
 
                 fn assign_scalar_rhs<const LANES: usize>() {
                     test_helpers::test_binary_scalar_rhs_elementwise(
-                        &|mut a, b| { <$vector as core::ops::$trait_assign<$scalar>>::$fn_assign(&mut a, b); a },
+                        &|mut a, b| { <Simd<$scalar, LANES> as core::ops::$trait_assign<$scalar>>::$fn_assign(&mut a, b); a },
                         &$scalar_fn,
                         &|x, y| x.iter().all(|x| $check_fn(*x, y)),
                     )
@@ -131,8 +133,8 @@ macro_rules! impl_binary_checked_op_test {
             }
         }
     };
-    { $vector:ty, $scalar:ty, $trait:ident :: $fn:ident, $trait_assign:ident :: $fn_assign:ident, $check_fn:expr } => {
-        impl_binary_nonzero_rhs_op_test! { $vector, $scalar, $trait::$fn, $trait_assign::$fn_assign, <$scalar as core::ops::$trait>::$fn, $check_fn }
+    { $scalar:ty, $trait:ident :: $fn:ident, $trait_assign:ident :: $fn_assign:ident, $check_fn:expr } => {
+        impl_binary_checked_op_test! { $scalar, $trait::$fn, $trait_assign::$fn_assign, <$scalar as core::ops::$trait>::$fn, $check_fn }
     };
 }
 
@@ -216,9 +218,9 @@ macro_rules! impl_common_integer_tests {
 /// Implement tests for signed integers.
 #[macro_export]
 macro_rules! impl_signed_tests {
-    { $vector:ident, $scalar:tt } => {
+    { $scalar:tt } => {
         mod $scalar {
-            type Vector<const LANES: usize> = core_simd::$vector<LANES>;
+            type Vector<const LANES: usize> = core_simd::Simd<Scalar, LANES>;
             type Scalar = $scalar;
 
             impl_common_integer_tests! { Vector, Scalar }
@@ -305,18 +307,18 @@ macro_rules! impl_signed_tests {
                 }
             }
 
-            impl_binary_op_test!(Vector<LANES>, Scalar, Add::add, AddAssign::add_assign, Scalar::wrapping_add);
-            impl_binary_op_test!(Vector<LANES>, Scalar, Sub::sub, SubAssign::sub_assign, Scalar::wrapping_sub);
-            impl_binary_op_test!(Vector<LANES>, Scalar, Mul::mul, MulAssign::mul_assign, Scalar::wrapping_mul);
+            impl_binary_op_test!(Scalar, Add::add, AddAssign::add_assign, Scalar::wrapping_add);
+            impl_binary_op_test!(Scalar, Sub::sub, SubAssign::sub_assign, Scalar::wrapping_sub);
+            impl_binary_op_test!(Scalar, Mul::mul, MulAssign::mul_assign, Scalar::wrapping_mul);
 
             // Exclude Div and Rem panicking cases
-            impl_binary_checked_op_test!(Vector<LANES>, Scalar, Div::div, DivAssign::div_assign, Scalar::wrapping_div, |x, y| y != 0 && !(x == Scalar::MIN && y == -1));
-            impl_binary_checked_op_test!(Vector<LANES>, Scalar, Rem::rem, RemAssign::rem_assign, Scalar::wrapping_rem, |x, y| y != 0 && !(x == Scalar::MIN && y == -1));
+            impl_binary_checked_op_test!(Scalar, Div::div, DivAssign::div_assign, Scalar::wrapping_div, |x, y| y != 0 && !(x == Scalar::MIN && y == -1));
+            impl_binary_checked_op_test!(Scalar, Rem::rem, RemAssign::rem_assign, Scalar::wrapping_rem, |x, y| y != 0 && !(x == Scalar::MIN && y == -1));
 
-            impl_unary_op_test!(Vector<LANES>, Scalar, Not::not);
-            impl_binary_op_test!(Vector<LANES>, Scalar, BitAnd::bitand, BitAndAssign::bitand_assign);
-            impl_binary_op_test!(Vector<LANES>, Scalar, BitOr::bitor, BitOrAssign::bitor_assign);
-            impl_binary_op_test!(Vector<LANES>, Scalar, BitXor::bitxor, BitXorAssign::bitxor_assign);
+            impl_unary_op_test!(Scalar, Not::not);
+            impl_binary_op_test!(Scalar, BitAnd::bitand, BitAndAssign::bitand_assign);
+            impl_binary_op_test!(Scalar, BitOr::bitor, BitOrAssign::bitor_assign);
+            impl_binary_op_test!(Scalar, BitXor::bitxor, BitXorAssign::bitxor_assign);
         }
     }
 }
@@ -324,9 +326,9 @@ macro_rules! impl_signed_tests {
 /// Implement tests for unsigned integers.
 #[macro_export]
 macro_rules! impl_unsigned_tests {
-    { $vector:ident, $scalar:tt } => {
+    { $scalar:tt } => {
         mod $scalar {
-            type Vector<const LANES: usize> = core_simd::$vector<LANES>;
+            type Vector<const LANES: usize> = core_simd::Simd<Scalar, LANES>;
             type Scalar = $scalar;
 
             impl_common_integer_tests! { Vector, Scalar }
@@ -339,18 +341,18 @@ macro_rules! impl_unsigned_tests {
                 }
             }
 
-            impl_binary_op_test!(Vector<LANES>, Scalar, Add::add, AddAssign::add_assign, Scalar::wrapping_add);
-            impl_binary_op_test!(Vector<LANES>, Scalar, Sub::sub, SubAssign::sub_assign, Scalar::wrapping_sub);
-            impl_binary_op_test!(Vector<LANES>, Scalar, Mul::mul, MulAssign::mul_assign, Scalar::wrapping_mul);
+            impl_binary_op_test!(Scalar, Add::add, AddAssign::add_assign, Scalar::wrapping_add);
+            impl_binary_op_test!(Scalar, Sub::sub, SubAssign::sub_assign, Scalar::wrapping_sub);
+            impl_binary_op_test!(Scalar, Mul::mul, MulAssign::mul_assign, Scalar::wrapping_mul);
 
             // Exclude Div and Rem panicking cases
-            impl_binary_checked_op_test!(Vector<LANES>, Scalar, Div::div, DivAssign::div_assign, Scalar::wrapping_div, |_, y| y != 0);
-            impl_binary_checked_op_test!(Vector<LANES>, Scalar, Rem::rem, RemAssign::rem_assign, Scalar::wrapping_rem, |_, y| y != 0);
+            impl_binary_checked_op_test!(Scalar, Div::div, DivAssign::div_assign, Scalar::wrapping_div, |_, y| y != 0);
+            impl_binary_checked_op_test!(Scalar, Rem::rem, RemAssign::rem_assign, Scalar::wrapping_rem, |_, y| y != 0);
 
-            impl_unary_op_test!(Vector<LANES>, Scalar, Not::not);
-            impl_binary_op_test!(Vector<LANES>, Scalar, BitAnd::bitand, BitAndAssign::bitand_assign);
-            impl_binary_op_test!(Vector<LANES>, Scalar, BitOr::bitor, BitOrAssign::bitor_assign);
-            impl_binary_op_test!(Vector<LANES>, Scalar, BitXor::bitxor, BitXorAssign::bitxor_assign);
+            impl_unary_op_test!(Scalar, Not::not);
+            impl_binary_op_test!(Scalar, BitAnd::bitand, BitAndAssign::bitand_assign);
+            impl_binary_op_test!(Scalar, BitOr::bitor, BitOrAssign::bitor_assign);
+            impl_binary_op_test!(Scalar, BitXor::bitxor, BitXorAssign::bitxor_assign);
         }
     }
 }
@@ -358,17 +360,17 @@ macro_rules! impl_unsigned_tests {
 /// Implement tests for floating point numbers.
 #[macro_export]
 macro_rules! impl_float_tests {
-    { $vector:ident, $scalar:tt, $int_scalar:tt } => {
+    { $scalar:tt, $int_scalar:tt } => {
         mod $scalar {
-            type Vector<const LANES: usize> = core_simd::$vector<LANES>;
+            type Vector<const LANES: usize> = core_simd::Simd<Scalar, LANES>;
             type Scalar = $scalar;
 
-            impl_unary_op_test!(Vector<LANES>, Scalar, Neg::neg);
-            impl_binary_op_test!(Vector<LANES>, Scalar, Add::add, AddAssign::add_assign);
-            impl_binary_op_test!(Vector<LANES>, Scalar, Sub::sub, SubAssign::sub_assign);
-            impl_binary_op_test!(Vector<LANES>, Scalar, Mul::mul, MulAssign::mul_assign);
-            impl_binary_op_test!(Vector<LANES>, Scalar, Div::div, DivAssign::div_assign);
-            impl_binary_op_test!(Vector<LANES>, Scalar, Rem::rem, RemAssign::rem_assign);
+            impl_unary_op_test!(Scalar, Neg::neg);
+            impl_binary_op_test!(Scalar, Add::add, AddAssign::add_assign);
+            impl_binary_op_test!(Scalar, Sub::sub, SubAssign::sub_assign);
+            impl_binary_op_test!(Scalar, Mul::mul, MulAssign::mul_assign);
+            impl_binary_op_test!(Scalar, Div::div, DivAssign::div_assign);
+            impl_binary_op_test!(Scalar, Rem::rem, RemAssign::rem_assign);
 
             test_helpers::test_lanes! {
                 fn is_sign_positive<const LANES: usize>() {
diff --git a/crates/core_simd/tests/u16_ops.rs b/crates/core_simd/tests/u16_ops.rs
index d220dae64568..9ae3bd6a47d0 100644
--- a/crates/core_simd/tests/u16_ops.rs
+++ b/crates/core_simd/tests/u16_ops.rs
@@ -2,4 +2,4 @@
 
 #[macro_use]
 mod ops_macros;
-impl_unsigned_tests! { SimdU16, u16 }
+impl_unsigned_tests! { u16 }
diff --git a/crates/core_simd/tests/u32_ops.rs b/crates/core_simd/tests/u32_ops.rs
index f27cc30a17fc..de34b73d6526 100644
--- a/crates/core_simd/tests/u32_ops.rs
+++ b/crates/core_simd/tests/u32_ops.rs
@@ -2,4 +2,4 @@
 
 #[macro_use]
 mod ops_macros;
-impl_unsigned_tests! { SimdU32, u32 }
+impl_unsigned_tests! { u32 }
diff --git a/crates/core_simd/tests/u64_ops.rs b/crates/core_simd/tests/u64_ops.rs
index ec3df39c53c3..8ee5a318c83d 100644
--- a/crates/core_simd/tests/u64_ops.rs
+++ b/crates/core_simd/tests/u64_ops.rs
@@ -2,4 +2,4 @@
 
 #[macro_use]
 mod ops_macros;
-impl_unsigned_tests! { SimdU64, u64 }
+impl_unsigned_tests! { u64 }
diff --git a/crates/core_simd/tests/u8_ops.rs b/crates/core_simd/tests/u8_ops.rs
index 2c52a52b9216..6d7211121284 100644
--- a/crates/core_simd/tests/u8_ops.rs
+++ b/crates/core_simd/tests/u8_ops.rs
@@ -2,4 +2,4 @@
 
 #[macro_use]
 mod ops_macros;
-impl_unsigned_tests! { SimdU8, u8 }
+impl_unsigned_tests! { u8 }
diff --git a/crates/core_simd/tests/usize_ops.rs b/crates/core_simd/tests/usize_ops.rs
index 070edc4e266f..9c7b1687a085 100644
--- a/crates/core_simd/tests/usize_ops.rs
+++ b/crates/core_simd/tests/usize_ops.rs
@@ -2,4 +2,4 @@
 
 #[macro_use]
 mod ops_macros;
-impl_unsigned_tests! { SimdUsize, usize }
+impl_unsigned_tests! { usize }

From 275889f7f464614069e88ea8efbf41c560da0a06 Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Sat, 7 Aug 2021 21:06:40 +0000
Subject: [PATCH 215/249] Remove remaining usage of aliases

---
 crates/core_simd/src/permute.rs      | 22 +++++------
 crates/core_simd/src/vector.rs       | 58 ++++++++++++++--------------
 crates/core_simd/src/vector/float.rs | 48 +++++++++++------------
 crates/core_simd/src/vector/int.rs   | 20 +++++-----
 crates/core_simd/src/vector/ptr.rs   | 10 ++---
 crates/core_simd/tests/permute.rs    | 10 ++---
 crates/core_simd/tests/to_bytes.rs   |  6 +--
 7 files changed, 87 insertions(+), 87 deletions(-)

diff --git a/crates/core_simd/src/permute.rs b/crates/core_simd/src/permute.rs
index 4e377d68915b..e1a085fd76db 100644
--- a/crates/core_simd/src/permute.rs
+++ b/crates/core_simd/src/permute.rs
@@ -15,12 +15,12 @@ macro_rules! impl_shuffle_lane {
             ///
             /// ```
             /// #![feature(portable_simd)]
-            /// # use core_simd::*;
-            /// let a = f32x4::from_array([1.0, 2.0, 3.0, 4.0]);
-            /// let b = f32x4::from_array([5.0, 6.0, 7.0, 8.0]);
+            /// # use core_simd::Simd;
+            /// let a = Simd::from_array([1.0, 2.0, 3.0, 4.0]);
+            /// let b = Simd::from_array([5.0, 6.0, 7.0, 8.0]);
             /// const IDXS: [u32; 4] = [4,0,3,7];
-            /// let c = f32x4::shuffle::<IDXS>(a,b);
-            /// assert_eq!(f32x4::from_array([5.0, 1.0, 4.0, 8.0]), c);
+            /// let c = Simd::<_, 4>::shuffle::<IDXS>(a,b);
+            /// assert_eq!(Simd::from_array([5.0, 1.0, 4.0, 8.0]), c);
             /// ```
             #[inline]
             pub fn shuffle<const IDX: [u32; $n]>(self, second: Self) -> Self {
@@ -56,9 +56,9 @@ macro_rules! impl_shuffle_lane {
             ///
             /// ```
             /// #![feature(portable_simd)]
-            /// # use core_simd::SimdU32;
-            /// let a = SimdU32::from_array([0, 1, 2, 3]);
-            /// let b = SimdU32::from_array([4, 5, 6, 7]);
+            /// # use core_simd::Simd;
+            /// let a = Simd::from_array([0, 1, 2, 3]);
+            /// let b = Simd::from_array([4, 5, 6, 7]);
             /// let (x, y) = a.interleave(b);
             /// assert_eq!(x.to_array(), [0, 4, 1, 5]);
             /// assert_eq!(y.to_array(), [2, 6, 3, 7]);
@@ -108,9 +108,9 @@ macro_rules! impl_shuffle_lane {
             ///
             /// ```
             /// #![feature(portable_simd)]
-            /// # use core_simd::SimdU32;
-            /// let a = SimdU32::from_array([0, 4, 1, 5]);
-            /// let b = SimdU32::from_array([2, 6, 3, 7]);
+            /// # use core_simd::Simd;
+            /// let a = Simd::from_array([0, 4, 1, 5]);
+            /// let b = Simd::from_array([2, 6, 3, 7]);
             /// let (x, y) = a.deinterleave(b);
             /// assert_eq!(x.to_array(), [0, 1, 2, 3]);
             /// assert_eq!(y.to_array(), [4, 5, 6, 7]);
diff --git a/crates/core_simd/src/vector.rs b/crates/core_simd/src/vector.rs
index 03c2f93a9c29..26c9bc0af6e1 100644
--- a/crates/core_simd/src/vector.rs
+++ b/crates/core_simd/src/vector.rs
@@ -9,7 +9,7 @@ pub use uint::*;
 // Vectors of pointers are not for public use at the current time.
 pub(crate) mod ptr;
 
-use crate::{LaneCount, MaskElement, SupportedLaneCount};
+use crate::{LaneCount, Mask, MaskElement, SupportedLaneCount};
 
 /// A SIMD vector of `LANES` elements of type `Element`.
 #[repr(simd)]
@@ -54,16 +54,16 @@ where
     /// # #![feature(portable_simd)]
     /// # use core_simd::*;
     /// let vec: Vec<i32> = vec![10, 11, 12, 13, 14, 15, 16, 17, 18];
-    /// let idxs = SimdUsize::<4>::from_array([9, 3, 0, 5]);
-    /// let alt = SimdI32::from_array([-5, -4, -3, -2]);
+    /// let idxs = Simd::from_array([9, 3, 0, 5]);
+    /// let alt = Simd::from_array([-5, -4, -3, -2]);
     ///
-    /// let result = SimdI32::<4>::gather_or(&vec, idxs, alt); // Note the lane that is out-of-bounds.
-    /// assert_eq!(result, SimdI32::from_array([-5, 13, 10, 15]));
+    /// let result = Simd::gather_or(&vec, idxs, alt); // Note the lane that is out-of-bounds.
+    /// assert_eq!(result, Simd::from_array([-5, 13, 10, 15]));
     /// ```
     #[must_use]
     #[inline]
-    pub fn gather_or(slice: &[Element], idxs: crate::SimdUsize<LANES>, or: Self) -> Self {
-        Self::gather_select(slice, crate::MaskSize::splat(true), idxs, or)
+    pub fn gather_or(slice: &[Element], idxs: Simd<usize, LANES>, or: Self) -> Self {
+        Self::gather_select(slice, Mask::splat(true), idxs, or)
     }
 
     /// SIMD gather: construct a SIMD vector by reading from a slice, using potentially discontiguous indices.
@@ -72,14 +72,14 @@ where
     /// # #![feature(portable_simd)]
     /// # use core_simd::*;
     /// let vec: Vec<i32> = vec![10, 11, 12, 13, 14, 15, 16, 17, 18];
-    /// let idxs = SimdUsize::<4>::from_array([9, 3, 0, 5]);
+    /// let idxs = Simd::from_array([9, 3, 0, 5]);
     ///
-    /// let result = SimdI32::<4>::gather_or_default(&vec, idxs); // Note the lane that is out-of-bounds.
-    /// assert_eq!(result, SimdI32::from_array([0, 13, 10, 15]));
+    /// let result = Simd::gather_or_default(&vec, idxs); // Note the lane that is out-of-bounds.
+    /// assert_eq!(result, Simd::from_array([0, 13, 10, 15]));
     /// ```
     #[must_use]
     #[inline]
-    pub fn gather_or_default(slice: &[Element], idxs: crate::SimdUsize<LANES>) -> Self
+    pub fn gather_or_default(slice: &[Element], idxs: Simd<usize, LANES>) -> Self
     where
         Element: Default,
     {
@@ -92,22 +92,22 @@ where
     /// # #![feature(portable_simd)]
     /// # use core_simd::*;
     /// let vec: Vec<i32> = vec![10, 11, 12, 13, 14, 15, 16, 17, 18];
-    /// let idxs = SimdUsize::<4>::from_array([9, 3, 0, 5]);
-    /// let alt = SimdI32::from_array([-5, -4, -3, -2]);
-    /// let mask = MaskSize::from_array([true, true, true, false]); // Note the mask of the last lane.
+    /// let idxs = Simd::from_array([9, 3, 0, 5]);
+    /// let alt = Simd::from_array([-5, -4, -3, -2]);
+    /// let mask = Mask::from_array([true, true, true, false]); // Note the mask of the last lane.
     ///
-    /// let result = SimdI32::<4>::gather_select(&vec, mask, idxs, alt); // Note the lane that is out-of-bounds.
-    /// assert_eq!(result, SimdI32::from_array([-5, 13, 10, -2]));
+    /// let result = Simd::gather_select(&vec, mask, idxs, alt); // Note the lane that is out-of-bounds.
+    /// assert_eq!(result, Simd::from_array([-5, 13, 10, -2]));
     /// ```
     #[must_use]
     #[inline]
     pub fn gather_select(
         slice: &[Element],
-        mask: crate::MaskSize<LANES>,
-        idxs: crate::SimdUsize<LANES>,
+        mask: Mask<isize, LANES>,
+        idxs: Simd<usize, LANES>,
         or: Self,
     ) -> Self {
-        let mask = (mask & idxs.lanes_lt(crate::SimdUsize::splat(slice.len()))).to_int();
+        let mask = (mask & idxs.lanes_lt(Simd::splat(slice.len()))).to_int();
         let base_ptr = crate::vector::ptr::SimdConstPtr::splat(slice.as_ptr());
         // Ferris forgive me, I have done pointer arithmetic here.
         let ptrs = base_ptr.wrapping_add(idxs);
@@ -122,15 +122,15 @@ where
     /// # #![feature(portable_simd)]
     /// # use core_simd::*;
     /// let mut vec: Vec<i32> = vec![10, 11, 12, 13, 14, 15, 16, 17, 18];
-    /// let idxs = SimdUsize::<4>::from_array([9, 3, 0, 0]);
-    /// let vals = SimdI32::from_array([-27, 82, -41, 124]);
+    /// let idxs = Simd::from_array([9, 3, 0, 0]);
+    /// let vals = Simd::from_array([-27, 82, -41, 124]);
     ///
     /// vals.scatter(&mut vec, idxs); // index 0 receives two writes.
     /// assert_eq!(vec, vec![124, 11, 12, 82, 14, 15, 16, 17, 18]);
     /// ```
     #[inline]
-    pub fn scatter(self, slice: &mut [Element], idxs: crate::SimdUsize<LANES>) {
-        self.scatter_select(slice, crate::MaskSize::splat(true), idxs)
+    pub fn scatter(self, slice: &mut [Element], idxs: Simd<usize, LANES>) {
+        self.scatter_select(slice, Mask::splat(true), idxs)
     }
 
     /// SIMD scatter: write a SIMD vector's values into a slice, using potentially discontiguous indices.
@@ -140,9 +140,9 @@ where
     /// # #![feature(portable_simd)]
     /// # use core_simd::*;
     /// let mut vec: Vec<i32> = vec![10, 11, 12, 13, 14, 15, 16, 17, 18];
-    /// let idxs = SimdUsize::<4>::from_array([9, 3, 0, 0]);
-    /// let vals = SimdI32::from_array([-27, 82, -41, 124]);
-    /// let mask = MaskSize::from_array([true, true, true, false]); // Note the mask of the last lane.
+    /// let idxs = Simd::from_array([9, 3, 0, 0]);
+    /// let vals = Simd::from_array([-27, 82, -41, 124]);
+    /// let mask = Mask::from_array([true, true, true, false]); // Note the mask of the last lane.
     ///
     /// vals.scatter_select(&mut vec, mask, idxs); // index 0's second write is masked, thus omitted.
     /// assert_eq!(vec, vec![-41, 11, 12, 82, 14, 15, 16, 17, 18]);
@@ -151,11 +151,11 @@ where
     pub fn scatter_select(
         self,
         slice: &mut [Element],
-        mask: crate::MaskSize<LANES>,
-        idxs: crate::SimdUsize<LANES>,
+        mask: Mask<isize, LANES>,
+        idxs: Simd<usize, LANES>,
     ) {
         // We must construct our scatter mask before we derive a pointer!
-        let mask = (mask & idxs.lanes_lt(crate::SimdUsize::splat(slice.len()))).to_int();
+        let mask = (mask & idxs.lanes_lt(Simd::splat(slice.len()))).to_int();
         // SAFETY: This block works with *mut T derived from &mut 'a [T],
         // which means it is delicate in Rust's borrowing model, circa 2021:
         // &mut 'a [T] asserts uniqueness, so deriving &'a [T] invalidates live *mut Ts!
diff --git a/crates/core_simd/src/vector/float.rs b/crates/core_simd/src/vector/float.rs
index 840ad049d2e3..96aacdfcca10 100644
--- a/crates/core_simd/src/vector/float.rs
+++ b/crates/core_simd/src/vector/float.rs
@@ -1,29 +1,29 @@
 #![allow(non_camel_case_types)]
 
-use crate::{LaneCount, SupportedLaneCount};
+use crate::{LaneCount, Mask, Simd, SupportedLaneCount};
 
-/// Implements inherent methods for a float vector `$name` containing multiple
+/// Implements inherent methods for a float vector containing multiple
 /// `$lanes` of float `$type`, which uses `$bits_ty` as its binary
-/// representation. Called from `define_float_vector!`.
+/// representation.
 macro_rules! impl_float_vector {
-    { $name:ident, $type:ident, $bits_ty:ident, $mask_ty:ident, $mask_impl_ty:ident } => {
-        impl<const LANES: usize> $name<LANES>
+    { $type:ty, $bits_ty:ty, $mask_ty:ty } => {
+        impl<const LANES: usize> Simd<$type, LANES>
         where
             LaneCount<LANES>: SupportedLaneCount,
         {
             /// Raw transmutation to an unsigned integer vector type with the
             /// same size and number of lanes.
             #[inline]
-            pub fn to_bits(self) -> crate::$bits_ty<LANES> {
-                assert_eq!(core::mem::size_of::<Self>(), core::mem::size_of::<crate::$bits_ty<LANES>>());
+            pub fn to_bits(self) -> Simd<$bits_ty, LANES> {
+                assert_eq!(core::mem::size_of::<Self>(), core::mem::size_of::<Simd<$bits_ty, LANES>>());
                 unsafe { core::mem::transmute_copy(&self) }
             }
 
             /// Raw transmutation from an unsigned integer vector type with the
             /// same size and number of lanes.
             #[inline]
-            pub fn from_bits(bits: crate::$bits_ty<LANES>) -> Self {
-                assert_eq!(core::mem::size_of::<Self>(), core::mem::size_of::<crate::$bits_ty<LANES>>());
+            pub fn from_bits(bits: Simd<$bits_ty, LANES>) -> Self {
+                assert_eq!(core::mem::size_of::<Self>(), core::mem::size_of::<Simd<$bits_ty, LANES>>());
                 unsafe { core::mem::transmute_copy(&bits) }
             }
 
@@ -64,58 +64,58 @@ macro_rules! impl_float_vector {
             #[inline]
             pub fn to_degrees(self) -> Self {
                 // to_degrees uses a special constant for better precision, so extract that constant
-                self * Self::splat($type::to_degrees(1.))
+                self * Self::splat(<$type>::to_degrees(1.))
             }
 
             /// Converts each lane from degrees to radians.
             #[inline]
             pub fn to_radians(self) -> Self {
-                self * Self::splat($type::to_radians(1.))
+                self * Self::splat(<$type>::to_radians(1.))
             }
 
             /// Returns true for each lane if it has a positive sign, including
             /// `+0.0`, `NaN`s with positive sign bit and positive infinity.
             #[inline]
-            pub fn is_sign_positive(self) -> crate::$mask_ty<LANES> {
+            pub fn is_sign_positive(self) -> Mask<$mask_ty, LANES> {
                 !self.is_sign_negative()
             }
 
             /// Returns true for each lane if it has a negative sign, including
             /// `-0.0`, `NaN`s with negative sign bit and negative infinity.
             #[inline]
-            pub fn is_sign_negative(self) -> crate::$mask_ty<LANES> {
-                let sign_bits = self.to_bits() & crate::$bits_ty::splat((!0 >> 1) + 1);
-                sign_bits.lanes_gt(crate::$bits_ty::splat(0))
+            pub fn is_sign_negative(self) -> Mask<$mask_ty, LANES> {
+                let sign_bits = self.to_bits() & Simd::splat((!0 >> 1) + 1);
+                sign_bits.lanes_gt(Simd::splat(0))
             }
 
             /// Returns true for each lane if its value is `NaN`.
             #[inline]
-            pub fn is_nan(self) -> crate::$mask_ty<LANES> {
+            pub fn is_nan(self) -> Mask<$mask_ty, LANES> {
                 self.lanes_ne(self)
             }
 
             /// Returns true for each lane if its value is positive infinity or negative infinity.
             #[inline]
-            pub fn is_infinite(self) -> crate::$mask_ty<LANES> {
+            pub fn is_infinite(self) -> Mask<$mask_ty, LANES> {
                 self.abs().lanes_eq(Self::splat(<$type>::INFINITY))
             }
 
             /// Returns true for each lane if its value is neither infinite nor `NaN`.
             #[inline]
-            pub fn is_finite(self) -> crate::$mask_ty<LANES> {
+            pub fn is_finite(self) -> Mask<$mask_ty, LANES> {
                 self.abs().lanes_lt(Self::splat(<$type>::INFINITY))
             }
 
             /// Returns true for each lane if its value is subnormal.
             #[inline]
-            pub fn is_subnormal(self) -> crate::$mask_ty<LANES> {
-                self.abs().lanes_ne(Self::splat(0.0)) & (self.to_bits() & Self::splat(<$type>::INFINITY).to_bits()).lanes_eq(crate::$bits_ty::splat(0))
+            pub fn is_subnormal(self) -> Mask<$mask_ty, LANES> {
+                self.abs().lanes_ne(Self::splat(0.0)) & (self.to_bits() & Self::splat(<$type>::INFINITY).to_bits()).lanes_eq(Simd::splat(0))
             }
 
             /// Returns true for each lane if its value is neither neither zero, infinite,
             /// subnormal, or `NaN`.
             #[inline]
-            pub fn is_normal(self) -> crate::$mask_ty<LANES> {
+            pub fn is_normal(self) -> Mask<$mask_ty, LANES> {
                 !(self.abs().lanes_eq(Self::splat(0.0)) | self.is_nan() | self.is_subnormal() | self.is_infinite())
             }
 
@@ -126,7 +126,7 @@ macro_rules! impl_float_vector {
             /// * `NAN` if the number is `NAN`
             #[inline]
             pub fn signum(self) -> Self {
-                self.is_nan().select(Self::splat($type::NAN), Self::splat(1.0).copysign(self))
+                self.is_nan().select(Self::splat(<$type>::NAN), Self::splat(1.0).copysign(self))
             }
 
             /// Returns each lane with the magnitude of `self` and the sign of `sign`.
@@ -189,8 +189,8 @@ pub type SimdF32<const LANES: usize> = crate::Simd<f32, LANES>;
 /// A SIMD vector of containing `LANES` `f64` values.
 pub type SimdF64<const LANES: usize> = crate::Simd<f64, LANES>;
 
-impl_float_vector! { SimdF32, f32, SimdU32, Mask32, SimdI32 }
-impl_float_vector! { SimdF64, f64, SimdU64, Mask64, SimdI64 }
+impl_float_vector! { f32, u32, i32 }
+impl_float_vector! { f64, u64, i64 }
 
 /// Vector of two `f32` values
 pub type f32x2 = SimdF32<2>;
diff --git a/crates/core_simd/src/vector/int.rs b/crates/core_simd/src/vector/int.rs
index 3dad2abbe7cc..38d90ad62c0d 100644
--- a/crates/core_simd/src/vector/int.rs
+++ b/crates/core_simd/src/vector/int.rs
@@ -1,23 +1,23 @@
 #![allow(non_camel_case_types)]
 
-use crate::{LaneCount, SupportedLaneCount};
+use crate::{LaneCount, Mask, Simd, SupportedLaneCount};
 
 /// Implements additional integer traits (Eq, Ord, Hash) on the specified vector `$name`, holding multiple `$lanes` of `$type`.
 macro_rules! impl_integer_vector {
-    { $name:ident, $type:ty, $mask_ty:ident, $mask_impl_ty:ident } => {
-        impl<const LANES: usize> $name<LANES>
+    { $type:ty } => {
+        impl<const LANES: usize> Simd<$type, LANES>
         where
             LaneCount<LANES>: SupportedLaneCount,
         {
             /// Returns true for each positive lane and false if it is zero or negative.
             #[inline]
-            pub fn is_positive(self) -> crate::$mask_ty<LANES> {
+            pub fn is_positive(self) -> Mask<$type, LANES> {
                 self.lanes_gt(Self::splat(0))
             }
 
             /// Returns true for each negative lane and false if it is zero or positive.
             #[inline]
-            pub fn is_negative(self) -> crate::$mask_ty<LANES> {
+            pub fn is_negative(self) -> Mask<$type, LANES> {
                 self.lanes_lt(Self::splat(0))
             }
 
@@ -51,11 +51,11 @@ pub type SimdI64<const LANES: usize> = crate::Simd<i64, LANES>;
 /// A SIMD vector of containing `LANES` `isize` values.
 pub type SimdIsize<const LANES: usize> = crate::Simd<isize, LANES>;
 
-impl_integer_vector! { SimdIsize, isize, MaskSize, SimdIsize }
-impl_integer_vector! { SimdI16, i16, Mask16, SimdI16 }
-impl_integer_vector! { SimdI32, i32, Mask32, SimdI32 }
-impl_integer_vector! { SimdI64, i64, Mask64, SimdI64 }
-impl_integer_vector! { SimdI8, i8, Mask8, SimdI8 }
+impl_integer_vector! { isize }
+impl_integer_vector! { i16 }
+impl_integer_vector! { i32 }
+impl_integer_vector! { i64 }
+impl_integer_vector! { i8 }
 
 /// Vector of two `isize` values
 pub type isizex2 = SimdIsize<2>;
diff --git a/crates/core_simd/src/vector/ptr.rs b/crates/core_simd/src/vector/ptr.rs
index 9dd1bfd0f365..fc4082a4b555 100644
--- a/crates/core_simd/src/vector/ptr.rs
+++ b/crates/core_simd/src/vector/ptr.rs
@@ -1,5 +1,5 @@
 //! Private implementation details of public gather/scatter APIs.
-use crate::{LaneCount, SimdUsize, SupportedLaneCount};
+use crate::{LaneCount, Simd, SupportedLaneCount};
 use core::mem;
 
 /// A vector of *const T.
@@ -20,9 +20,9 @@ where
 
     #[inline]
     #[must_use]
-    pub fn wrapping_add(self, addend: SimdUsize<LANES>) -> Self {
+    pub fn wrapping_add(self, addend: Simd<usize, LANES>) -> Self {
         unsafe {
-            let x: SimdUsize<LANES> = mem::transmute_copy(&self);
+            let x: Simd<usize, LANES> = mem::transmute_copy(&self);
             mem::transmute_copy(&{ x + (addend * mem::size_of::<T>()) })
         }
     }
@@ -46,9 +46,9 @@ where
 
     #[inline]
     #[must_use]
-    pub fn wrapping_add(self, addend: SimdUsize<LANES>) -> Self {
+    pub fn wrapping_add(self, addend: Simd<usize, LANES>) -> Self {
         unsafe {
-            let x: SimdUsize<LANES> = mem::transmute_copy(&self);
+            let x: Simd<usize, LANES> = mem::transmute_copy(&self);
             mem::transmute_copy(&{ x + (addend * mem::size_of::<T>()) })
         }
     }
diff --git a/crates/core_simd/tests/permute.rs b/crates/core_simd/tests/permute.rs
index 4c771002528f..ea52e8f5ca73 100644
--- a/crates/core_simd/tests/permute.rs
+++ b/crates/core_simd/tests/permute.rs
@@ -1,6 +1,6 @@
 #![feature(portable_simd)]
 
-use core_simd::SimdU32;
+use core_simd::Simd;
 
 #[cfg(target_arch = "wasm32")]
 use wasm_bindgen_test::*;
@@ -11,7 +11,7 @@ wasm_bindgen_test_configure!(run_in_browser);
 #[test]
 #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
 fn simple_shuffle() {
-    let a = SimdU32::from_array([2, 4, 1, 9]);
+    let a = Simd::from_array([2, 4, 1, 9]);
     let b = a;
     assert_eq!(a.shuffle::<{ [3, 1, 4, 6] }>(b).to_array(), [9, 4, 2, 1]);
 }
@@ -19,15 +19,15 @@ fn simple_shuffle() {
 #[test]
 #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
 fn reverse() {
-    let a = SimdU32::from_array([0, 1, 2, 3, 4, 5, 6, 7]);
+    let a = Simd::from_array([0, 1, 2, 3, 4, 5, 6, 7]);
     assert_eq!(a.reverse().to_array(), [7, 6, 5, 4, 3, 2, 1, 0]);
 }
 
 #[test]
 #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
 fn interleave() {
-    let a = SimdU32::from_array([0, 1, 2, 3, 4, 5, 6, 7]);
-    let b = SimdU32::from_array([8, 9, 10, 11, 12, 13, 14, 15]);
+    let a = Simd::from_array([0, 1, 2, 3, 4, 5, 6, 7]);
+    let b = Simd::from_array([8, 9, 10, 11, 12, 13, 14, 15]);
     let (lo, hi) = a.interleave(b);
     assert_eq!(lo.to_array(), [0, 8, 1, 9, 2, 10, 3, 11]);
     assert_eq!(hi.to_array(), [4, 12, 5, 13, 6, 14, 7, 15]);
diff --git a/crates/core_simd/tests/to_bytes.rs b/crates/core_simd/tests/to_bytes.rs
index 22c97c95d927..c66c9d5bd36f 100644
--- a/crates/core_simd/tests/to_bytes.rs
+++ b/crates/core_simd/tests/to_bytes.rs
@@ -2,13 +2,13 @@
 #![allow(incomplete_features)]
 #![cfg(feature = "const_evaluatable_checked")]
 
-use core_simd::SimdU32;
+use core_simd::Simd;
 
 #[test]
 fn byte_convert() {
-    let int = SimdU32::from_array([0xdeadbeef, 0x8badf00d]);
+    let int = Simd::<u32, 2>::from_array([0xdeadbeef, 0x8badf00d]);
     let bytes = int.to_ne_bytes();
     assert_eq!(int[0].to_ne_bytes(), bytes[..4]);
     assert_eq!(int[1].to_ne_bytes(), bytes[4..]);
-    assert_eq!(SimdU32::from_ne_bytes(bytes), int);
+    assert_eq!(Simd::<u32, 2>::from_ne_bytes(bytes), int);
 }

From 40142ac034088c0ca149d4ca511bc854c70ff238 Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Sat, 7 Aug 2021 21:15:24 +0000
Subject: [PATCH 216/249] Remove aliases

---
 crates/core_simd/src/select.rs       | 16 ++++----
 crates/core_simd/src/vector/float.rs | 20 ++++------
 crates/core_simd/src/vector/int.rs   | 55 ++++++++++------------------
 crates/core_simd/src/vector/uint.rs  | 55 +++++++++++-----------------
 crates/core_simd/tests/round.rs      |  8 ++--
 5 files changed, 60 insertions(+), 94 deletions(-)

diff --git a/crates/core_simd/src/select.rs b/crates/core_simd/src/select.rs
index 1f7ea854a933..710d23a71d03 100644
--- a/crates/core_simd/src/select.rs
+++ b/crates/core_simd/src/select.rs
@@ -60,10 +60,10 @@ where
     ///
     /// ```
     /// # #![feature(portable_simd)]
-    /// # use core_simd::{Mask32, SimdI32};
-    /// let a = SimdI32::from_array([0, 1, 2, 3]);
-    /// let b = SimdI32::from_array([4, 5, 6, 7]);
-    /// let mask = Mask32::from_array([true, false, false, true]);
+    /// # use core_simd::{Mask, Simd};
+    /// let a = Simd::from_array([0, 1, 2, 3]);
+    /// let b = Simd::from_array([4, 5, 6, 7]);
+    /// let mask = Mask::from_array([true, false, false, true]);
     /// let c = mask.select(a, b);
     /// assert_eq!(c.to_array(), [0, 5, 6, 3]);
     /// ```
@@ -71,10 +71,10 @@ where
     /// `select` can also be used on masks:
     /// ```
     /// # #![feature(portable_simd)]
-    /// # use core_simd::Mask32;
-    /// let a = Mask32::from_array([true, true, false, false]);
-    /// let b = Mask32::from_array([false, false, true, true]);
-    /// let mask = Mask32::from_array([true, false, false, true]);
+    /// # use core_simd::Mask;
+    /// let a = Mask::<i32, 4>::from_array([true, true, false, false]);
+    /// let b = Mask::<i32, 4>::from_array([false, false, true, true]);
+    /// let mask = Mask::<i32, 4>::from_array([true, false, false, true]);
     /// let c = mask.select(a, b);
     /// assert_eq!(c.to_array(), [true, false, true, false]);
     /// ```
diff --git a/crates/core_simd/src/vector/float.rs b/crates/core_simd/src/vector/float.rs
index 96aacdfcca10..6ef88ddebc68 100644
--- a/crates/core_simd/src/vector/float.rs
+++ b/crates/core_simd/src/vector/float.rs
@@ -183,32 +183,26 @@ macro_rules! impl_float_vector {
     };
 }
 
-/// A SIMD vector of containing `LANES` `f32` values.
-pub type SimdF32<const LANES: usize> = crate::Simd<f32, LANES>;
-
-/// A SIMD vector of containing `LANES` `f64` values.
-pub type SimdF64<const LANES: usize> = crate::Simd<f64, LANES>;
-
 impl_float_vector! { f32, u32, i32 }
 impl_float_vector! { f64, u64, i64 }
 
 /// Vector of two `f32` values
-pub type f32x2 = SimdF32<2>;
+pub type f32x2 = Simd<f32, 2>;
 
 /// Vector of four `f32` values
-pub type f32x4 = SimdF32<4>;
+pub type f32x4 = Simd<f32, 4>;
 
 /// Vector of eight `f32` values
-pub type f32x8 = SimdF32<8>;
+pub type f32x8 = Simd<f32, 8>;
 
 /// Vector of 16 `f32` values
-pub type f32x16 = SimdF32<16>;
+pub type f32x16 = Simd<f32, 16>;
 
 /// Vector of two `f64` values
-pub type f64x2 = SimdF64<2>;
+pub type f64x2 = Simd<f64, 2>;
 
 /// Vector of four `f64` values
-pub type f64x4 = SimdF64<4>;
+pub type f64x4 = Simd<f64, 4>;
 
 /// Vector of eight `f64` values
-pub type f64x8 = SimdF64<8>;
+pub type f64x8 = Simd<f64, 8>;
diff --git a/crates/core_simd/src/vector/int.rs b/crates/core_simd/src/vector/int.rs
index 38d90ad62c0d..5f435e16b687 100644
--- a/crates/core_simd/src/vector/int.rs
+++ b/crates/core_simd/src/vector/int.rs
@@ -36,21 +36,6 @@ macro_rules! impl_integer_vector {
     }
 }
 
-/// A SIMD vector of containing `LANES` `i8` values.
-pub type SimdI8<const LANES: usize> = crate::Simd<i8, LANES>;
-
-/// A SIMD vector of containing `LANES` `i16` values.
-pub type SimdI16<const LANES: usize> = crate::Simd<i16, LANES>;
-
-/// A SIMD vector of containing `LANES` `i32` values.
-pub type SimdI32<const LANES: usize> = crate::Simd<i32, LANES>;
-
-/// A SIMD vector of containing `LANES` `i64` values.
-pub type SimdI64<const LANES: usize> = crate::Simd<i64, LANES>;
-
-/// A SIMD vector of containing `LANES` `isize` values.
-pub type SimdIsize<const LANES: usize> = crate::Simd<isize, LANES>;
-
 impl_integer_vector! { isize }
 impl_integer_vector! { i16 }
 impl_integer_vector! { i32 }
@@ -58,61 +43,61 @@ impl_integer_vector! { i64 }
 impl_integer_vector! { i8 }
 
 /// Vector of two `isize` values
-pub type isizex2 = SimdIsize<2>;
+pub type isizex2 = Simd<isize, 2>;
 
 /// Vector of four `isize` values
-pub type isizex4 = SimdIsize<4>;
+pub type isizex4 = Simd<isize, 4>;
 
 /// Vector of eight `isize` values
-pub type isizex8 = SimdIsize<8>;
+pub type isizex8 = Simd<isize, 8>;
 
 /// Vector of two `i16` values
-pub type i16x2 = SimdI16<2>;
+pub type i16x2 = Simd<i16, 2>;
 
 /// Vector of four `i16` values
-pub type i16x4 = SimdI16<4>;
+pub type i16x4 = Simd<i16, 4>;
 
 /// Vector of eight `i16` values
-pub type i16x8 = SimdI16<8>;
+pub type i16x8 = Simd<i16, 8>;
 
 /// Vector of 16 `i16` values
-pub type i16x16 = SimdI16<16>;
+pub type i16x16 = Simd<i16, 16>;
 
 /// Vector of 32 `i16` values
-pub type i16x32 = SimdI16<32>;
+pub type i16x32 = Simd<i16, 32>;
 
 /// Vector of two `i32` values
-pub type i32x2 = SimdI32<2>;
+pub type i32x2 = Simd<i32, 2>;
 
 /// Vector of four `i32` values
-pub type i32x4 = SimdI32<4>;
+pub type i32x4 = Simd<i32, 4>;
 
 /// Vector of eight `i32` values
-pub type i32x8 = SimdI32<8>;
+pub type i32x8 = Simd<i32, 8>;
 
 /// Vector of 16 `i32` values
-pub type i32x16 = SimdI32<16>;
+pub type i32x16 = Simd<i32, 16>;
 
 /// Vector of two `i64` values
-pub type i64x2 = SimdI64<2>;
+pub type i64x2 = Simd<i64, 2>;
 
 /// Vector of four `i64` values
-pub type i64x4 = SimdI64<4>;
+pub type i64x4 = Simd<i64, 4>;
 
 /// Vector of eight `i64` values
-pub type i64x8 = SimdI64<8>;
+pub type i64x8 = Simd<i64, 8>;
 
 /// Vector of four `i8` values
-pub type i8x4 = SimdI8<4>;
+pub type i8x4 = Simd<i8, 4>;
 
 /// Vector of eight `i8` values
-pub type i8x8 = SimdI8<8>;
+pub type i8x8 = Simd<i8, 8>;
 
 /// Vector of 16 `i8` values
-pub type i8x16 = SimdI8<16>;
+pub type i8x16 = Simd<i8, 16>;
 
 /// Vector of 32 `i8` values
-pub type i8x32 = SimdI8<32>;
+pub type i8x32 = Simd<i8, 32>;
 
 /// Vector of 64 `i8` values
-pub type i8x64 = SimdI8<64>;
+pub type i8x64 = Simd<i8, 64>;
diff --git a/crates/core_simd/src/vector/uint.rs b/crates/core_simd/src/vector/uint.rs
index ba6dab930901..b3dd199a5463 100644
--- a/crates/core_simd/src/vector/uint.rs
+++ b/crates/core_simd/src/vector/uint.rs
@@ -1,76 +1,63 @@
 #![allow(non_camel_case_types)]
 
-/// A SIMD vector of containing `LANES` `u8` values.
-pub type SimdU8<const LANES: usize> = crate::Simd<u8, LANES>;
-
-/// A SIMD vector of containing `LANES` `u16` values.
-pub type SimdU16<const LANES: usize> = crate::Simd<u16, LANES>;
-
-/// A SIMD vector of containing `LANES` `u32` values.
-pub type SimdU32<const LANES: usize> = crate::Simd<u32, LANES>;
-
-/// A SIMD vector of containing `LANES` `u64` values.
-pub type SimdU64<const LANES: usize> = crate::Simd<u64, LANES>;
-
-/// A SIMD vector of containing `LANES` `usize` values.
-pub type SimdUsize<const LANES: usize> = crate::Simd<usize, LANES>;
+use crate::Simd;
 
 /// Vector of two `usize` values
-pub type usizex2 = SimdUsize<2>;
+pub type usizex2 = Simd<usize, 2>;
 
 /// Vector of four `usize` values
-pub type usizex4 = SimdUsize<4>;
+pub type usizex4 = Simd<usize, 4>;
 
 /// Vector of eight `usize` values
-pub type usizex8 = SimdUsize<8>;
+pub type usizex8 = Simd<usize, 8>;
 
 /// Vector of two `u16` values
-pub type u16x2 = SimdU16<2>;
+pub type u16x2 = Simd<u16, 2>;
 
 /// Vector of four `u16` values
-pub type u16x4 = SimdU16<4>;
+pub type u16x4 = Simd<u16, 4>;
 
 /// Vector of eight `u16` values
-pub type u16x8 = SimdU16<8>;
+pub type u16x8 = Simd<u16, 8>;
 
 /// Vector of 16 `u16` values
-pub type u16x16 = SimdU16<16>;
+pub type u16x16 = Simd<u16, 16>;
 
 /// Vector of 32 `u16` values
-pub type u16x32 = SimdU16<32>;
+pub type u16x32 = Simd<u16, 32>;
 
 /// Vector of two `u32` values
-pub type u32x2 = SimdU32<2>;
+pub type u32x2 = Simd<u32, 2>;
 
 /// Vector of four `u32` values
-pub type u32x4 = SimdU32<4>;
+pub type u32x4 = Simd<u32, 4>;
 
 /// Vector of eight `u32` values
-pub type u32x8 = SimdU32<8>;
+pub type u32x8 = Simd<u32, 8>;
 
 /// Vector of 16 `u32` values
-pub type u32x16 = SimdU32<16>;
+pub type u32x16 = Simd<u32, 16>;
 
 /// Vector of two `u64` values
-pub type u64x2 = SimdU64<2>;
+pub type u64x2 = Simd<u64, 2>;
 
 /// Vector of four `u64` values
-pub type u64x4 = SimdU64<4>;
+pub type u64x4 = Simd<u64, 4>;
 
 /// Vector of eight `u64` values
-pub type u64x8 = SimdU64<8>;
+pub type u64x8 = Simd<u64, 8>;
 
 /// Vector of four `u8` values
-pub type u8x4 = SimdU8<4>;
+pub type u8x4 = Simd<u8, 4>;
 
 /// Vector of eight `u8` values
-pub type u8x8 = SimdU8<8>;
+pub type u8x8 = Simd<u8, 8>;
 
 /// Vector of 16 `u8` values
-pub type u8x16 = SimdU8<16>;
+pub type u8x16 = Simd<u8, 16>;
 
 /// Vector of 32 `u8` values
-pub type u8x32 = SimdU8<32>;
+pub type u8x32 = Simd<u8, 32>;
 
 /// Vector of 64 `u8` values
-pub type u8x64 = SimdU8<64>;
+pub type u8x64 = Simd<u8, 64>;
diff --git a/crates/core_simd/tests/round.rs b/crates/core_simd/tests/round.rs
index 37044a751125..11d617a6c2c5 100644
--- a/crates/core_simd/tests/round.rs
+++ b/crates/core_simd/tests/round.rs
@@ -1,9 +1,9 @@
 #![feature(portable_simd)]
 
 macro_rules! float_rounding_test {
-    { $vector:ident, $scalar:tt, $int_scalar:tt } => {
+    { $scalar:tt, $int_scalar:tt } => {
         mod $scalar {
-            type Vector<const LANES: usize> = core_simd::$vector<LANES>;
+            type Vector<const LANES: usize> = core_simd::Simd<$scalar, LANES>;
             type Scalar = $scalar;
             type IntScalar = $int_scalar;
 
@@ -88,5 +88,5 @@ macro_rules! float_rounding_test {
     }
 }
 
-float_rounding_test! { SimdF32, f32, i32 }
-float_rounding_test! { SimdF64, f64, i64 }
+float_rounding_test! { f32, i32 }
+float_rounding_test! { f64, i64 }

From 00165ed5beea0dbbbb950afba692b4c804485c03 Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Sat, 7 Aug 2021 21:22:10 +0000
Subject: [PATCH 217/249] Remove mask aliases

---
 crates/core_simd/src/masks.rs   | 51 ++++++++++++---------------------
 crates/core_simd/tests/masks.rs | 36 +++++++++++------------
 2 files changed, 36 insertions(+), 51 deletions(-)

diff --git a/crates/core_simd/src/masks.rs b/crates/core_simd/src/masks.rs
index b7bde44b384f..b433712a3291 100644
--- a/crates/core_simd/src/masks.rs
+++ b/crates/core_simd/src/masks.rs
@@ -452,74 +452,59 @@ where
     }
 }
 
-/// A SIMD mask of `LANES` 8-bit values.
-pub type Mask8<const LANES: usize> = Mask<i8, LANES>;
-
-/// A SIMD mask of `LANES` 16-bit values.
-pub type Mask16<const LANES: usize> = Mask<i16, LANES>;
-
-/// A SIMD mask of `LANES` 32-bit values.
-pub type Mask32<const LANES: usize> = Mask<i32, LANES>;
-
-/// A SIMD mask of `LANES` 64-bit values.
-pub type Mask64<const LANES: usize> = Mask<i64, LANES>;
-
-/// A SIMD mask of `LANES` pointer-width values.
-pub type MaskSize<const LANES: usize> = Mask<isize, LANES>;
-
 /// Vector of eight 8-bit masks
-pub type mask8x8 = Mask8<8>;
+pub type mask8x8 = Mask<i8, 8>;
 
 /// Vector of 16 8-bit masks
-pub type mask8x16 = Mask8<16>;
+pub type mask8x16 = Mask<i8, 16>;
 
 /// Vector of 32 8-bit masks
-pub type mask8x32 = Mask8<32>;
+pub type mask8x32 = Mask<i8, 32>;
 
 /// Vector of 16 8-bit masks
-pub type mask8x64 = Mask8<64>;
+pub type mask8x64 = Mask<i8, 64>;
 
 /// Vector of four 16-bit masks
-pub type mask16x4 = Mask16<4>;
+pub type mask16x4 = Mask<i16, 4>;
 
 /// Vector of eight 16-bit masks
-pub type mask16x8 = Mask16<8>;
+pub type mask16x8 = Mask<i16, 8>;
 
 /// Vector of 16 16-bit masks
-pub type mask16x16 = Mask16<16>;
+pub type mask16x16 = Mask<i16, 16>;
 
 /// Vector of 32 16-bit masks
-pub type mask16x32 = Mask32<32>;
+pub type mask16x32 = Mask<i32, 32>;
 
 /// Vector of two 32-bit masks
-pub type mask32x2 = Mask32<2>;
+pub type mask32x2 = Mask<i32, 2>;
 
 /// Vector of four 32-bit masks
-pub type mask32x4 = Mask32<4>;
+pub type mask32x4 = Mask<i32, 4>;
 
 /// Vector of eight 32-bit masks
-pub type mask32x8 = Mask32<8>;
+pub type mask32x8 = Mask<i32, 8>;
 
 /// Vector of 16 32-bit masks
-pub type mask32x16 = Mask32<16>;
+pub type mask32x16 = Mask<i32, 16>;
 
 /// Vector of two 64-bit masks
-pub type mask64x2 = Mask64<2>;
+pub type mask64x2 = Mask<i64, 2>;
 
 /// Vector of four 64-bit masks
-pub type mask64x4 = Mask64<4>;
+pub type mask64x4 = Mask<i64, 4>;
 
 /// Vector of eight 64-bit masks
-pub type mask64x8 = Mask64<8>;
+pub type mask64x8 = Mask<i64, 8>;
 
 /// Vector of two pointer-width masks
-pub type masksizex2 = MaskSize<2>;
+pub type masksizex2 = Mask<isize, 2>;
 
 /// Vector of four pointer-width masks
-pub type masksizex4 = MaskSize<4>;
+pub type masksizex4 = Mask<isize, 4>;
 
 /// Vector of eight pointer-width masks
-pub type masksizex8 = MaskSize<8>;
+pub type masksizex8 = Mask<isize, 8>;
 
 macro_rules! impl_from {
     { $from:ty  => $($to:ty),* } => {
diff --git a/crates/core_simd/tests/masks.rs b/crates/core_simd/tests/masks.rs
index 61d8e449744b..cf8039d153d5 100644
--- a/crates/core_simd/tests/masks.rs
+++ b/crates/core_simd/tests/masks.rs
@@ -7,9 +7,9 @@ use wasm_bindgen_test::*;
 wasm_bindgen_test_configure!(run_in_browser);
 
 macro_rules! test_mask_api {
-    { $name:ident } => {
+    { $type:ident } => {
         #[allow(non_snake_case)]
-        mod $name {
+        mod $type {
             #[cfg(target_arch = "wasm32")]
             use wasm_bindgen_test::*;
 
@@ -17,7 +17,7 @@ macro_rules! test_mask_api {
             #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
             fn set_and_test() {
                 let values = [true, false, false, true, false, false, true, false];
-                let mut mask = core_simd::$name::<8>::splat(false);
+                let mut mask = core_simd::Mask::<$type, 8>::splat(false);
                 for (lane, value) in values.iter().copied().enumerate() {
                     mask.set(lane, value);
                 }
@@ -29,7 +29,7 @@ macro_rules! test_mask_api {
             #[test]
             #[should_panic]
             fn set_invalid_lane() {
-                let mut mask = core_simd::$name::<8>::splat(false);
+                let mut mask = core_simd::Mask::<$type, 8>::splat(false);
                 mask.set(8, true);
                 let _ = mask;
             }
@@ -37,24 +37,24 @@ macro_rules! test_mask_api {
             #[test]
             #[should_panic]
             fn test_invalid_lane() {
-                let mask = core_simd::$name::<8>::splat(false);
+                let mask = core_simd::Mask::<$type, 8>::splat(false);
                 let _ = mask.test(8);
             }
 
             #[test]
             fn any() {
-                assert!(!core_simd::$name::<8>::splat(false).any());
-                assert!(core_simd::$name::<8>::splat(true).any());
-                let mut v = core_simd::$name::<8>::splat(false);
+                assert!(!core_simd::Mask::<$type, 8>::splat(false).any());
+                assert!(core_simd::Mask::<$type, 8>::splat(true).any());
+                let mut v = core_simd::Mask::<$type, 8>::splat(false);
                 v.set(2, true);
                 assert!(v.any());
             }
 
             #[test]
             fn all() {
-                assert!(!core_simd::$name::<8>::splat(false).all());
-                assert!(core_simd::$name::<8>::splat(true).all());
-                let mut v = core_simd::$name::<8>::splat(false);
+                assert!(!core_simd::Mask::<$type, 8>::splat(false).all());
+                assert!(core_simd::Mask::<$type, 8>::splat(true).all());
+                let mut v = core_simd::Mask::<$type, 8>::splat(false);
                 v.set(2, true);
                 assert!(!v.all());
             }
@@ -62,10 +62,10 @@ macro_rules! test_mask_api {
             #[test]
             fn roundtrip_int_conversion() {
                 let values = [true, false, false, true, false, false, true, false];
-                let mask = core_simd::$name::<8>::from_array(values);
+                let mask = core_simd::Mask::<$type, 8>::from_array(values);
                 let int = mask.to_int();
                 assert_eq!(int.to_array(), [-1, 0, 0, -1, 0, 0, -1, 0]);
-                assert_eq!(core_simd::$name::<8>::from_int(int), mask);
+                assert_eq!(core_simd::Mask::<$type, 8>::from_int(int), mask);
             }
 
             #[test]
@@ -74,24 +74,24 @@ macro_rules! test_mask_api {
                     true, false, false, true, false, false, true, false,
                     true, true, false, false, false, false, false, true,
                 ];
-                let mask = core_simd::$name::<16>::from_array(values);
+                let mask = core_simd::Mask::<$type, 16>::from_array(values);
                 let bitmask = mask.to_bitmask();
                 assert_eq!(bitmask, [0b01001001, 0b10000011]);
-                assert_eq!(core_simd::$name::<16>::from_bitmask(bitmask), mask);
+                assert_eq!(core_simd::Mask::<$type, 16>::from_bitmask(bitmask), mask);
             }
         }
     }
 }
 
 mod mask_api {
-    test_mask_api! { Mask8 }
+    test_mask_api! { i8 }
 }
 
 #[test]
 fn convert() {
     let values = [true, false, false, true, false, false, true, false];
     assert_eq!(
-        core_simd::Mask8::from_array(values),
-        core_simd::Mask32::from_array(values).into()
+        core_simd::Mask::<i8, 8>::from_array(values),
+        core_simd::Mask::<i32, 8>::from_array(values).into()
     );
 }

From cf653c7b9376eb170d0c8db634f5559e3708a87d Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Fri, 13 Aug 2021 20:40:05 -0400
Subject: [PATCH 218/249] Update crates/core_simd/src/vector.rs

Co-authored-by: Jubilee <46493976+workingjubilee@users.noreply.github.com>
---
 crates/core_simd/src/vector.rs | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/crates/core_simd/src/vector.rs b/crates/core_simd/src/vector.rs
index 26c9bc0af6e1..9b6d0a20ed9d 100644
--- a/crates/core_simd/src/vector.rs
+++ b/crates/core_simd/src/vector.rs
@@ -336,6 +336,11 @@ mod sealed {
 use sealed::Sealed;
 
 /// Marker trait for types that may be used as SIMD vector elements.
+/// SAFETY: This trait, when implemented, asserts the compiler can monomorphize
+/// `#[repr(simd)]` structs with the marked type as an element.
+/// Strictly, it is valid to impl if the vector will not be miscompiled.
+/// Practically, it is user-unfriendly to impl it if the vector won't compile,
+/// even when no soundness guarantees are broken by allowing the user to try.
 pub unsafe trait SimdElement: Sealed + Copy {
     /// The mask element type corresponding to this element type.
     type Mask: MaskElement;

From 4aafd8e779bac1122d8218e75872c6b14007f33d Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Mon, 16 Aug 2021 16:38:30 -0400
Subject: [PATCH 219/249] Rename element type variable

---
 crates/core_simd/src/comparisons.rs      |  20 ++--
 crates/core_simd/src/fmt.rs              |   4 +-
 crates/core_simd/src/masks.rs            | 128 +++++++++++------------
 crates/core_simd/src/masks/bitmask.rs    |  62 +++++------
 crates/core_simd/src/masks/full_masks.rs |  72 ++++++-------
 crates/core_simd/src/ops.rs              |  12 +--
 crates/core_simd/src/permute.rs          |   4 +-
 crates/core_simd/src/select.rs           |  22 ++--
 crates/core_simd/src/vector.rs           | 104 +++++++++---------
 9 files changed, 212 insertions(+), 216 deletions(-)

diff --git a/crates/core_simd/src/comparisons.rs b/crates/core_simd/src/comparisons.rs
index c094f680a59d..601576e094fa 100644
--- a/crates/core_simd/src/comparisons.rs
+++ b/crates/core_simd/src/comparisons.rs
@@ -1,49 +1,49 @@
 use crate::{LaneCount, Mask, Simd, SimdElement, SupportedLaneCount};
 
-impl<Element, const LANES: usize> Simd<Element, LANES>
+impl<T, const LANES: usize> Simd<T, LANES>
 where
-    Element: SimdElement + PartialEq,
+    T: SimdElement + PartialEq,
     LaneCount<LANES>: SupportedLaneCount,
 {
     /// Test if each lane is equal to the corresponding lane in `other`.
     #[inline]
-    pub fn lanes_eq(self, other: Self) -> Mask<Element::Mask, LANES> {
+    pub fn lanes_eq(self, other: Self) -> Mask<T::Mask, LANES> {
         unsafe { Mask::from_int_unchecked(crate::intrinsics::simd_eq(self, other)) }
     }
 
     /// Test if each lane is not equal to the corresponding lane in `other`.
     #[inline]
-    pub fn lanes_ne(self, other: Self) -> Mask<Element::Mask, LANES> {
+    pub fn lanes_ne(self, other: Self) -> Mask<T::Mask, LANES> {
         unsafe { Mask::from_int_unchecked(crate::intrinsics::simd_ne(self, other)) }
     }
 }
 
-impl<Element, const LANES: usize> Simd<Element, LANES>
+impl<T, const LANES: usize> Simd<T, LANES>
 where
-    Element: SimdElement + PartialOrd,
+    T: SimdElement + PartialOrd,
     LaneCount<LANES>: SupportedLaneCount,
 {
     /// Test if each lane is less than the corresponding lane in `other`.
     #[inline]
-    pub fn lanes_lt(self, other: Self) -> Mask<Element::Mask, LANES> {
+    pub fn lanes_lt(self, other: Self) -> Mask<T::Mask, LANES> {
         unsafe { Mask::from_int_unchecked(crate::intrinsics::simd_lt(self, other)) }
     }
 
     /// Test if each lane is greater than the corresponding lane in `other`.
     #[inline]
-    pub fn lanes_gt(self, other: Self) -> Mask<Element::Mask, LANES> {
+    pub fn lanes_gt(self, other: Self) -> Mask<T::Mask, LANES> {
         unsafe { Mask::from_int_unchecked(crate::intrinsics::simd_gt(self, other)) }
     }
 
     /// Test if each lane is less than or equal to the corresponding lane in `other`.
     #[inline]
-    pub fn lanes_le(self, other: Self) -> Mask<Element::Mask, LANES> {
+    pub fn lanes_le(self, other: Self) -> Mask<T::Mask, LANES> {
         unsafe { Mask::from_int_unchecked(crate::intrinsics::simd_le(self, other)) }
     }
 
     /// Test if each lane is greater than or equal to the corresponding lane in `other`.
     #[inline]
-    pub fn lanes_ge(self, other: Self) -> Mask<Element::Mask, LANES> {
+    pub fn lanes_ge(self, other: Self) -> Mask<T::Mask, LANES> {
         unsafe { Mask::from_int_unchecked(crate::intrinsics::simd_ge(self, other)) }
     }
 }
diff --git a/crates/core_simd/src/fmt.rs b/crates/core_simd/src/fmt.rs
index 9ad3a6c100ea..c3947c92f2a9 100644
--- a/crates/core_simd/src/fmt.rs
+++ b/crates/core_simd/src/fmt.rs
@@ -1,10 +1,10 @@
 macro_rules! impl_fmt_trait {
     { $($trait:ident,)* } => {
         $(
-            impl<Element, const LANES: usize> core::fmt::$trait for crate::Simd<Element, LANES>
+            impl<T, const LANES: usize> core::fmt::$trait for crate::Simd<T, LANES>
             where
                 crate::LaneCount<LANES>: crate::SupportedLaneCount,
-                Element: crate::SimdElement + core::fmt::$trait,
+                T: crate::SimdElement + core::fmt::$trait,
             {
                 fn fmt(&self, f: &mut core::fmt::Formatter) -> core::fmt::Result {
                     #[repr(transparent)]
diff --git a/crates/core_simd/src/masks.rs b/crates/core_simd/src/masks.rs
index b433712a3291..14b1fe08ffbd 100644
--- a/crates/core_simd/src/masks.rs
+++ b/crates/core_simd/src/masks.rs
@@ -59,21 +59,21 @@ impl_element! { isize }
 ///
 /// The layout of this type is unspecified.
 #[repr(transparent)]
-pub struct Mask<Element, const LANES: usize>(mask_impl::Mask<Element, LANES>)
+pub struct Mask<T, const LANES: usize>(mask_impl::Mask<T, LANES>)
 where
-    Element: MaskElement,
+    T: MaskElement,
     LaneCount<LANES>: SupportedLaneCount;
 
-impl<Element, const LANES: usize> Copy for Mask<Element, LANES>
+impl<T, const LANES: usize> Copy for Mask<T, LANES>
 where
-    Element: MaskElement,
+    T: MaskElement,
     LaneCount<LANES>: SupportedLaneCount,
 {
 }
 
-impl<Element, const LANES: usize> Clone for Mask<Element, LANES>
+impl<T, const LANES: usize> Clone for Mask<T, LANES>
 where
-    Element: MaskElement,
+    T: MaskElement,
     LaneCount<LANES>: SupportedLaneCount,
 {
     fn clone(&self) -> Self {
@@ -81,9 +81,9 @@ where
     }
 }
 
-impl<Element, const LANES: usize> Mask<Element, LANES>
+impl<T, const LANES: usize> Mask<T, LANES>
 where
-    Element: MaskElement,
+    T: MaskElement,
     LaneCount<LANES>: SupportedLaneCount,
 {
     /// Construct a mask by setting all lanes to the given value.
@@ -115,7 +115,7 @@ where
     /// # Safety
     /// All lanes must be either 0 or -1.
     #[inline]
-    pub unsafe fn from_int_unchecked(value: Simd<Element, LANES>) -> Self {
+    pub unsafe fn from_int_unchecked(value: Simd<T, LANES>) -> Self {
         Self(mask_impl::Mask::from_int_unchecked(value))
     }
 
@@ -125,15 +125,15 @@ where
     /// # Panics
     /// Panics if any lane is not 0 or -1.
     #[inline]
-    pub fn from_int(value: Simd<Element, LANES>) -> Self {
-        assert!(Element::valid(value), "all values must be either 0 or -1",);
+    pub fn from_int(value: Simd<T, LANES>) -> Self {
+        assert!(T::valid(value), "all values must be either 0 or -1",);
         unsafe { Self::from_int_unchecked(value) }
     }
 
     /// Converts the mask to a vector of integers, where 0 represents `false` and -1
     /// represents `true`.
     #[inline]
-    pub fn to_int(self) -> Simd<Element, LANES> {
+    pub fn to_int(self) -> Simd<T, LANES> {
         self.0.to_int()
     }
 
@@ -201,9 +201,9 @@ where
 }
 
 // vector/array conversion
-impl<Element, const LANES: usize> From<[bool; LANES]> for Mask<Element, LANES>
+impl<T, const LANES: usize> From<[bool; LANES]> for Mask<T, LANES>
 where
-    Element: MaskElement,
+    T: MaskElement,
     LaneCount<LANES>: SupportedLaneCount,
 {
     fn from(array: [bool; LANES]) -> Self {
@@ -211,19 +211,19 @@ where
     }
 }
 
-impl<Element, const LANES: usize> From<Mask<Element, LANES>> for [bool; LANES]
+impl<T, const LANES: usize> From<Mask<T, LANES>> for [bool; LANES]
 where
-    Element: MaskElement,
+    T: MaskElement,
     LaneCount<LANES>: SupportedLaneCount,
 {
-    fn from(vector: Mask<Element, LANES>) -> Self {
+    fn from(vector: Mask<T, LANES>) -> Self {
         vector.to_array()
     }
 }
 
-impl<Element, const LANES: usize> Default for Mask<Element, LANES>
+impl<T, const LANES: usize> Default for Mask<T, LANES>
 where
-    Element: MaskElement,
+    T: MaskElement,
     LaneCount<LANES>: SupportedLaneCount,
 {
     #[inline]
@@ -232,9 +232,9 @@ where
     }
 }
 
-impl<Element, const LANES: usize> PartialEq for Mask<Element, LANES>
+impl<T, const LANES: usize> PartialEq for Mask<T, LANES>
 where
-    Element: MaskElement + PartialEq,
+    T: MaskElement + PartialEq,
     LaneCount<LANES>: SupportedLaneCount,
 {
     #[inline]
@@ -243,9 +243,9 @@ where
     }
 }
 
-impl<Element, const LANES: usize> PartialOrd for Mask<Element, LANES>
+impl<T, const LANES: usize> PartialOrd for Mask<T, LANES>
 where
-    Element: MaskElement + PartialOrd,
+    T: MaskElement + PartialOrd,
     LaneCount<LANES>: SupportedLaneCount,
 {
     #[inline]
@@ -254,9 +254,9 @@ where
     }
 }
 
-impl<Element, const LANES: usize> core::fmt::Debug for Mask<Element, LANES>
+impl<T, const LANES: usize> core::fmt::Debug for Mask<T, LANES>
 where
-    Element: MaskElement + core::fmt::Debug,
+    T: MaskElement + core::fmt::Debug,
     LaneCount<LANES>: SupportedLaneCount,
 {
     fn fmt(&self, f: &mut core::fmt::Formatter) -> core::fmt::Result {
@@ -266,9 +266,9 @@ where
     }
 }
 
-impl<Element, const LANES: usize> core::ops::BitAnd for Mask<Element, LANES>
+impl<T, const LANES: usize> core::ops::BitAnd for Mask<T, LANES>
 where
-    Element: MaskElement,
+    T: MaskElement,
     LaneCount<LANES>: SupportedLaneCount,
 {
     type Output = Self;
@@ -278,9 +278,9 @@ where
     }
 }
 
-impl<Element, const LANES: usize> core::ops::BitAnd<bool> for Mask<Element, LANES>
+impl<T, const LANES: usize> core::ops::BitAnd<bool> for Mask<T, LANES>
 where
-    Element: MaskElement,
+    T: MaskElement,
     LaneCount<LANES>: SupportedLaneCount,
 {
     type Output = Self;
@@ -290,21 +290,21 @@ where
     }
 }
 
-impl<Element, const LANES: usize> core::ops::BitAnd<Mask<Element, LANES>> for bool
+impl<T, const LANES: usize> core::ops::BitAnd<Mask<T, LANES>> for bool
 where
-    Element: MaskElement,
+    T: MaskElement,
     LaneCount<LANES>: SupportedLaneCount,
 {
-    type Output = Mask<Element, LANES>;
+    type Output = Mask<T, LANES>;
     #[inline]
-    fn bitand(self, rhs: Mask<Element, LANES>) -> Mask<Element, LANES> {
+    fn bitand(self, rhs: Mask<T, LANES>) -> Mask<T, LANES> {
         Mask::splat(self) & rhs
     }
 }
 
-impl<Element, const LANES: usize> core::ops::BitOr for Mask<Element, LANES>
+impl<T, const LANES: usize> core::ops::BitOr for Mask<T, LANES>
 where
-    Element: MaskElement,
+    T: MaskElement,
     LaneCount<LANES>: SupportedLaneCount,
 {
     type Output = Self;
@@ -314,9 +314,9 @@ where
     }
 }
 
-impl<Element, const LANES: usize> core::ops::BitOr<bool> for Mask<Element, LANES>
+impl<T, const LANES: usize> core::ops::BitOr<bool> for Mask<T, LANES>
 where
-    Element: MaskElement,
+    T: MaskElement,
     LaneCount<LANES>: SupportedLaneCount,
 {
     type Output = Self;
@@ -326,21 +326,21 @@ where
     }
 }
 
-impl<Element, const LANES: usize> core::ops::BitOr<Mask<Element, LANES>> for bool
+impl<T, const LANES: usize> core::ops::BitOr<Mask<T, LANES>> for bool
 where
-    Element: MaskElement,
+    T: MaskElement,
     LaneCount<LANES>: SupportedLaneCount,
 {
-    type Output = Mask<Element, LANES>;
+    type Output = Mask<T, LANES>;
     #[inline]
-    fn bitor(self, rhs: Mask<Element, LANES>) -> Mask<Element, LANES> {
+    fn bitor(self, rhs: Mask<T, LANES>) -> Mask<T, LANES> {
         Mask::splat(self) | rhs
     }
 }
 
-impl<Element, const LANES: usize> core::ops::BitXor for Mask<Element, LANES>
+impl<T, const LANES: usize> core::ops::BitXor for Mask<T, LANES>
 where
-    Element: MaskElement,
+    T: MaskElement,
     LaneCount<LANES>: SupportedLaneCount,
 {
     type Output = Self;
@@ -350,9 +350,9 @@ where
     }
 }
 
-impl<Element, const LANES: usize> core::ops::BitXor<bool> for Mask<Element, LANES>
+impl<T, const LANES: usize> core::ops::BitXor<bool> for Mask<T, LANES>
 where
-    Element: MaskElement,
+    T: MaskElement,
     LaneCount<LANES>: SupportedLaneCount,
 {
     type Output = Self;
@@ -362,33 +362,33 @@ where
     }
 }
 
-impl<Element, const LANES: usize> core::ops::BitXor<Mask<Element, LANES>> for bool
+impl<T, const LANES: usize> core::ops::BitXor<Mask<T, LANES>> for bool
 where
-    Element: MaskElement,
+    T: MaskElement,
     LaneCount<LANES>: SupportedLaneCount,
 {
-    type Output = Mask<Element, LANES>;
+    type Output = Mask<T, LANES>;
     #[inline]
-    fn bitxor(self, rhs: Mask<Element, LANES>) -> Self::Output {
+    fn bitxor(self, rhs: Mask<T, LANES>) -> Self::Output {
         Mask::splat(self) ^ rhs
     }
 }
 
-impl<Element, const LANES: usize> core::ops::Not for Mask<Element, LANES>
+impl<T, const LANES: usize> core::ops::Not for Mask<T, LANES>
 where
-    Element: MaskElement,
+    T: MaskElement,
     LaneCount<LANES>: SupportedLaneCount,
 {
-    type Output = Mask<Element, LANES>;
+    type Output = Mask<T, LANES>;
     #[inline]
     fn not(self) -> Self::Output {
         Self(!self.0)
     }
 }
 
-impl<Element, const LANES: usize> core::ops::BitAndAssign for Mask<Element, LANES>
+impl<T, const LANES: usize> core::ops::BitAndAssign for Mask<T, LANES>
 where
-    Element: MaskElement,
+    T: MaskElement,
     LaneCount<LANES>: SupportedLaneCount,
 {
     #[inline]
@@ -397,9 +397,9 @@ where
     }
 }
 
-impl<Element, const LANES: usize> core::ops::BitAndAssign<bool> for Mask<Element, LANES>
+impl<T, const LANES: usize> core::ops::BitAndAssign<bool> for Mask<T, LANES>
 where
-    Element: MaskElement,
+    T: MaskElement,
     LaneCount<LANES>: SupportedLaneCount,
 {
     #[inline]
@@ -408,9 +408,9 @@ where
     }
 }
 
-impl<Element, const LANES: usize> core::ops::BitOrAssign for Mask<Element, LANES>
+impl<T, const LANES: usize> core::ops::BitOrAssign for Mask<T, LANES>
 where
-    Element: MaskElement,
+    T: MaskElement,
     LaneCount<LANES>: SupportedLaneCount,
 {
     #[inline]
@@ -419,9 +419,9 @@ where
     }
 }
 
-impl<Element, const LANES: usize> core::ops::BitOrAssign<bool> for Mask<Element, LANES>
+impl<T, const LANES: usize> core::ops::BitOrAssign<bool> for Mask<T, LANES>
 where
-    Element: MaskElement,
+    T: MaskElement,
     LaneCount<LANES>: SupportedLaneCount,
 {
     #[inline]
@@ -430,9 +430,9 @@ where
     }
 }
 
-impl<Element, const LANES: usize> core::ops::BitXorAssign for Mask<Element, LANES>
+impl<T, const LANES: usize> core::ops::BitXorAssign for Mask<T, LANES>
 where
-    Element: MaskElement,
+    T: MaskElement,
     LaneCount<LANES>: SupportedLaneCount,
 {
     #[inline]
@@ -441,9 +441,9 @@ where
     }
 }
 
-impl<Element, const LANES: usize> core::ops::BitXorAssign<bool> for Mask<Element, LANES>
+impl<T, const LANES: usize> core::ops::BitXorAssign<bool> for Mask<T, LANES>
 where
-    Element: MaskElement,
+    T: MaskElement,
     LaneCount<LANES>: SupportedLaneCount,
 {
     #[inline]
diff --git a/crates/core_simd/src/masks/bitmask.rs b/crates/core_simd/src/masks/bitmask.rs
index 2b8309494510..0b5b3a5c595e 100644
--- a/crates/core_simd/src/masks/bitmask.rs
+++ b/crates/core_simd/src/masks/bitmask.rs
@@ -3,24 +3,24 @@ use core::marker::PhantomData;
 
 /// A mask where each lane is represented by a single bit.
 #[repr(transparent)]
-pub struct Mask<Element, const LANES: usize>(
+pub struct Mask<T, const LANES: usize>(
     <LaneCount<LANES> as SupportedLaneCount>::BitMask,
-    PhantomData<Element>,
+    PhantomData<T>,
 )
 where
-    Element: MaskElement,
+    T: MaskElement,
     LaneCount<LANES>: SupportedLaneCount;
 
-impl<Element, const LANES: usize> Copy for Mask<Element, LANES>
+impl<T, const LANES: usize> Copy for Mask<T, LANES>
 where
-    Element: MaskElement,
+    T: MaskElement,
     LaneCount<LANES>: SupportedLaneCount,
 {
 }
 
-impl<Element, const LANES: usize> Clone for Mask<Element, LANES>
+impl<T, const LANES: usize> Clone for Mask<T, LANES>
 where
-    Element: MaskElement,
+    T: MaskElement,
     LaneCount<LANES>: SupportedLaneCount,
 {
     fn clone(&self) -> Self {
@@ -28,9 +28,9 @@ where
     }
 }
 
-impl<Element, const LANES: usize> PartialEq for Mask<Element, LANES>
+impl<T, const LANES: usize> PartialEq for Mask<T, LANES>
 where
-    Element: MaskElement,
+    T: MaskElement,
     LaneCount<LANES>: SupportedLaneCount,
 {
     fn eq(&self, other: &Self) -> bool {
@@ -38,9 +38,9 @@ where
     }
 }
 
-impl<Element, const LANES: usize> PartialOrd for Mask<Element, LANES>
+impl<T, const LANES: usize> PartialOrd for Mask<T, LANES>
 where
-    Element: MaskElement,
+    T: MaskElement,
     LaneCount<LANES>: SupportedLaneCount,
 {
     fn partial_cmp(&self, other: &Self) -> Option<core::cmp::Ordering> {
@@ -48,16 +48,16 @@ where
     }
 }
 
-impl<Element, const LANES: usize> Eq for Mask<Element, LANES>
+impl<T, const LANES: usize> Eq for Mask<T, LANES>
 where
-    Element: MaskElement,
+    T: MaskElement,
     LaneCount<LANES>: SupportedLaneCount,
 {
 }
 
-impl<Element, const LANES: usize> Ord for Mask<Element, LANES>
+impl<T, const LANES: usize> Ord for Mask<T, LANES>
 where
-    Element: MaskElement,
+    T: MaskElement,
     LaneCount<LANES>: SupportedLaneCount,
 {
     fn cmp(&self, other: &Self) -> core::cmp::Ordering {
@@ -65,9 +65,9 @@ where
     }
 }
 
-impl<Element, const LANES: usize> Mask<Element, LANES>
+impl<T, const LANES: usize> Mask<T, LANES>
 where
-    Element: MaskElement,
+    T: MaskElement,
     LaneCount<LANES>: SupportedLaneCount,
 {
     #[inline]
@@ -95,20 +95,20 @@ where
     }
 
     #[inline]
-    pub fn to_int(self) -> Simd<Element, LANES> {
+    pub fn to_int(self) -> Simd<T, LANES> {
         unsafe {
             let mask: <LaneCount<LANES> as SupportedLaneCount>::IntBitMask =
                 core::mem::transmute_copy(&self);
             crate::intrinsics::simd_select_bitmask(
                 mask,
-                Simd::splat(Element::TRUE),
-                Simd::splat(Element::FALSE),
+                Simd::splat(T::TRUE),
+                Simd::splat(T::FALSE),
             )
         }
     }
 
     #[inline]
-    pub unsafe fn from_int_unchecked(value: Simd<Element, LANES>) -> Self {
+    pub unsafe fn from_int_unchecked(value: Simd<T, LANES>) -> Self {
         // TODO remove the transmute when rustc is more flexible
         assert_eq!(
             core::mem::size_of::<<LaneCount::<LANES> as SupportedLaneCount>::BitMask>(),
@@ -132,9 +132,9 @@ where
     }
 
     #[inline]
-    pub fn convert<T>(self) -> Mask<T, LANES>
+    pub fn convert<U>(self) -> Mask<U, LANES>
     where
-        T: MaskElement,
+        U: MaskElement,
     {
         unsafe { core::mem::transmute_copy(&self) }
     }
@@ -150,9 +150,9 @@ where
     }
 }
 
-impl<Element, const LANES: usize> core::ops::BitAnd for Mask<Element, LANES>
+impl<T, const LANES: usize> core::ops::BitAnd for Mask<T, LANES>
 where
-    Element: MaskElement,
+    T: MaskElement,
     LaneCount<LANES>: SupportedLaneCount,
     <LaneCount<LANES> as SupportedLaneCount>::BitMask: AsRef<[u8]> + AsMut<[u8]>,
 {
@@ -166,9 +166,9 @@ where
     }
 }
 
-impl<Element, const LANES: usize> core::ops::BitOr for Mask<Element, LANES>
+impl<T, const LANES: usize> core::ops::BitOr for Mask<T, LANES>
 where
-    Element: MaskElement,
+    T: MaskElement,
     LaneCount<LANES>: SupportedLaneCount,
     <LaneCount<LANES> as SupportedLaneCount>::BitMask: AsRef<[u8]> + AsMut<[u8]>,
 {
@@ -182,9 +182,9 @@ where
     }
 }
 
-impl<Element, const LANES: usize> core::ops::BitXor for Mask<Element, LANES>
+impl<T, const LANES: usize> core::ops::BitXor for Mask<T, LANES>
 where
-    Element: MaskElement,
+    T: MaskElement,
     LaneCount<LANES>: SupportedLaneCount,
 {
     type Output = Self;
@@ -197,9 +197,9 @@ where
     }
 }
 
-impl<Element, const LANES: usize> core::ops::Not for Mask<Element, LANES>
+impl<T, const LANES: usize> core::ops::Not for Mask<T, LANES>
 where
-    Element: MaskElement,
+    T: MaskElement,
     LaneCount<LANES>: SupportedLaneCount,
 {
     type Output = Self;
diff --git a/crates/core_simd/src/masks/full_masks.rs b/crates/core_simd/src/masks/full_masks.rs
index b45ace3791d3..9c1cc4623f9a 100644
--- a/crates/core_simd/src/masks/full_masks.rs
+++ b/crates/core_simd/src/masks/full_masks.rs
@@ -4,21 +4,21 @@ use super::MaskElement;
 use crate::{LaneCount, Simd, SupportedLaneCount};
 
 #[repr(transparent)]
-pub struct Mask<Element, const LANES: usize>(Simd<Element, LANES>)
+pub struct Mask<T, const LANES: usize>(Simd<T, LANES>)
 where
-    Element: MaskElement,
+    T: MaskElement,
     LaneCount<LANES>: SupportedLaneCount;
 
-impl<Element, const LANES: usize> Copy for Mask<Element, LANES>
+impl<T, const LANES: usize> Copy for Mask<T, LANES>
 where
-    Element: MaskElement,
+    T: MaskElement,
     LaneCount<LANES>: SupportedLaneCount,
 {
 }
 
-impl<Element, const LANES: usize> Clone for Mask<Element, LANES>
+impl<T, const LANES: usize> Clone for Mask<T, LANES>
 where
-    Element: MaskElement,
+    T: MaskElement,
     LaneCount<LANES>: SupportedLaneCount,
 {
     #[inline]
@@ -27,9 +27,9 @@ where
     }
 }
 
-impl<Element, const LANES: usize> PartialEq for Mask<Element, LANES>
+impl<T, const LANES: usize> PartialEq for Mask<T, LANES>
 where
-    Element: MaskElement + PartialEq,
+    T: MaskElement + PartialEq,
     LaneCount<LANES>: SupportedLaneCount,
 {
     fn eq(&self, other: &Self) -> bool {
@@ -37,9 +37,9 @@ where
     }
 }
 
-impl<Element, const LANES: usize> PartialOrd for Mask<Element, LANES>
+impl<T, const LANES: usize> PartialOrd for Mask<T, LANES>
 where
-    Element: MaskElement + PartialOrd,
+    T: MaskElement + PartialOrd,
     LaneCount<LANES>: SupportedLaneCount,
 {
     fn partial_cmp(&self, other: &Self) -> Option<core::cmp::Ordering> {
@@ -47,16 +47,16 @@ where
     }
 }
 
-impl<Element, const LANES: usize> Eq for Mask<Element, LANES>
+impl<T, const LANES: usize> Eq for Mask<T, LANES>
 where
-    Element: MaskElement + Eq,
+    T: MaskElement + Eq,
     LaneCount<LANES>: SupportedLaneCount,
 {
 }
 
-impl<Element, const LANES: usize> Ord for Mask<Element, LANES>
+impl<T, const LANES: usize> Ord for Mask<T, LANES>
 where
-    Element: MaskElement + Ord,
+    T: MaskElement + Ord,
     LaneCount<LANES>: SupportedLaneCount,
 {
     fn cmp(&self, other: &Self) -> core::cmp::Ordering {
@@ -64,43 +64,39 @@ where
     }
 }
 
-impl<Element, const LANES: usize> Mask<Element, LANES>
+impl<T, const LANES: usize> Mask<T, LANES>
 where
-    Element: MaskElement,
+    T: MaskElement,
     LaneCount<LANES>: SupportedLaneCount,
 {
     pub fn splat(value: bool) -> Self {
-        Self(Simd::splat(if value {
-            Element::TRUE
-        } else {
-            Element::FALSE
-        }))
+        Self(Simd::splat(if value { T::TRUE } else { T::FALSE }))
     }
 
     #[inline]
     pub unsafe fn test_unchecked(&self, lane: usize) -> bool {
-        Element::eq(self.0[lane], Element::TRUE)
+        T::eq(self.0[lane], T::TRUE)
     }
 
     #[inline]
     pub unsafe fn set_unchecked(&mut self, lane: usize, value: bool) {
-        self.0[lane] = if value { Element::TRUE } else { Element::FALSE }
+        self.0[lane] = if value { T::TRUE } else { T::FALSE }
     }
 
     #[inline]
-    pub fn to_int(self) -> Simd<Element, LANES> {
+    pub fn to_int(self) -> Simd<T, LANES> {
         self.0
     }
 
     #[inline]
-    pub unsafe fn from_int_unchecked(value: Simd<Element, LANES>) -> Self {
+    pub unsafe fn from_int_unchecked(value: Simd<T, LANES>) -> Self {
         Self(value)
     }
 
     #[inline]
-    pub fn convert<T>(self) -> Mask<T, LANES>
+    pub fn convert<U>(self) -> Mask<U, LANES>
     where
-        T: MaskElement,
+        U: MaskElement,
     {
         unsafe { Mask(crate::intrinsics::simd_cast(self.0)) }
     }
@@ -170,19 +166,19 @@ where
     }
 }
 
-impl<Element, const LANES: usize> core::convert::From<Mask<Element, LANES>> for Simd<Element, LANES>
+impl<T, const LANES: usize> core::convert::From<Mask<T, LANES>> for Simd<T, LANES>
 where
-    Element: MaskElement,
+    T: MaskElement,
     LaneCount<LANES>: SupportedLaneCount,
 {
-    fn from(value: Mask<Element, LANES>) -> Self {
+    fn from(value: Mask<T, LANES>) -> Self {
         value.0
     }
 }
 
-impl<Element, const LANES: usize> core::ops::BitAnd for Mask<Element, LANES>
+impl<T, const LANES: usize> core::ops::BitAnd for Mask<T, LANES>
 where
-    Element: MaskElement,
+    T: MaskElement,
     LaneCount<LANES>: SupportedLaneCount,
 {
     type Output = Self;
@@ -192,9 +188,9 @@ where
     }
 }
 
-impl<Element, const LANES: usize> core::ops::BitOr for Mask<Element, LANES>
+impl<T, const LANES: usize> core::ops::BitOr for Mask<T, LANES>
 where
-    Element: MaskElement,
+    T: MaskElement,
     LaneCount<LANES>: SupportedLaneCount,
 {
     type Output = Self;
@@ -204,9 +200,9 @@ where
     }
 }
 
-impl<Element, const LANES: usize> core::ops::BitXor for Mask<Element, LANES>
+impl<T, const LANES: usize> core::ops::BitXor for Mask<T, LANES>
 where
-    Element: MaskElement,
+    T: MaskElement,
     LaneCount<LANES>: SupportedLaneCount,
 {
     type Output = Self;
@@ -216,9 +212,9 @@ where
     }
 }
 
-impl<Element, const LANES: usize> core::ops::Not for Mask<Element, LANES>
+impl<T, const LANES: usize> core::ops::Not for Mask<T, LANES>
 where
-    Element: MaskElement,
+    T: MaskElement,
     LaneCount<LANES>: SupportedLaneCount,
 {
     type Output = Self;
diff --git a/crates/core_simd/src/ops.rs b/crates/core_simd/src/ops.rs
index 651498817c38..900315660005 100644
--- a/crates/core_simd/src/ops.rs
+++ b/crates/core_simd/src/ops.rs
@@ -1,10 +1,10 @@
 use crate::{LaneCount, Simd, SimdElement, SupportedLaneCount};
 
-impl<I, Element, const LANES: usize> core::ops::Index<I> for Simd<Element, LANES>
+impl<I, T, const LANES: usize> core::ops::Index<I> for Simd<T, LANES>
 where
-    Element: SimdElement,
+    T: SimdElement,
     LaneCount<LANES>: SupportedLaneCount,
-    I: core::slice::SliceIndex<[Element]>,
+    I: core::slice::SliceIndex<[T]>,
 {
     type Output = I::Output;
     fn index(&self, index: I) -> &Self::Output {
@@ -12,11 +12,11 @@ where
     }
 }
 
-impl<I, Element, const LANES: usize> core::ops::IndexMut<I> for Simd<Element, LANES>
+impl<I, T, const LANES: usize> core::ops::IndexMut<I> for Simd<T, LANES>
 where
-    Element: SimdElement,
+    T: SimdElement,
     LaneCount<LANES>: SupportedLaneCount,
-    I: core::slice::SliceIndex<[Element]>,
+    I: core::slice::SliceIndex<[T]>,
 {
     fn index_mut(&mut self, index: I) -> &mut Self::Output {
         &mut self.as_mut_array()[index]
diff --git a/crates/core_simd/src/permute.rs b/crates/core_simd/src/permute.rs
index e1a085fd76db..cc58778b6b4b 100644
--- a/crates/core_simd/src/permute.rs
+++ b/crates/core_simd/src/permute.rs
@@ -1,8 +1,8 @@
 macro_rules! impl_shuffle_lane {
     { $fn:ident, $n:literal } => {
-        impl<Element> crate::Simd<Element, $n>
+        impl<T> crate::Simd<T, $n>
         where
-            Element: crate::SimdElement,
+            T: crate::SimdElement,
         {
             /// A const SIMD shuffle that takes 2 SIMD vectors and produces another vector, using
             /// the indices in the const parameter. The first or "self" vector will have its lanes
diff --git a/crates/core_simd/src/select.rs b/crates/core_simd/src/select.rs
index 710d23a71d03..0951639c9426 100644
--- a/crates/core_simd/src/select.rs
+++ b/crates/core_simd/src/select.rs
@@ -11,34 +11,34 @@ pub trait Select<Mask>: Sealed {
     fn select(mask: Mask, true_values: Self, false_values: Self) -> Self;
 }
 
-impl<Element, const LANES: usize> Sealed for Simd<Element, LANES>
+impl<T, const LANES: usize> Sealed for Simd<T, LANES>
 where
-    Element: SimdElement,
+    T: SimdElement,
     LaneCount<LANES>: SupportedLaneCount,
 {
 }
 
-impl<Element, const LANES: usize> Select<Mask<Element::Mask, LANES>> for Simd<Element, LANES>
+impl<T, const LANES: usize> Select<Mask<T::Mask, LANES>> for Simd<T, LANES>
 where
-    Element: SimdElement,
+    T: SimdElement,
     LaneCount<LANES>: SupportedLaneCount,
 {
     #[inline]
-    fn select(mask: Mask<Element::Mask, LANES>, true_values: Self, false_values: Self) -> Self {
+    fn select(mask: Mask<T::Mask, LANES>, true_values: Self, false_values: Self) -> Self {
         unsafe { crate::intrinsics::simd_select(mask.to_int(), true_values, false_values) }
     }
 }
 
-impl<Element, const LANES: usize> Sealed for Mask<Element, LANES>
+impl<T, const LANES: usize> Sealed for Mask<T, LANES>
 where
-    Element: MaskElement,
+    T: MaskElement,
     LaneCount<LANES>: SupportedLaneCount,
 {
 }
 
-impl<Element, const LANES: usize> Select<Self> for Mask<Element, LANES>
+impl<T, const LANES: usize> Select<Self> for Mask<T, LANES>
 where
-    Element: MaskElement,
+    T: MaskElement,
     LaneCount<LANES>: SupportedLaneCount,
 {
     #[doc(hidden)]
@@ -48,9 +48,9 @@ where
     }
 }
 
-impl<Element, const LANES: usize> Mask<Element, LANES>
+impl<T, const LANES: usize> Mask<T, LANES>
 where
-    Element: MaskElement,
+    T: MaskElement,
     LaneCount<LANES>: SupportedLaneCount,
 {
     /// Choose lanes from two vectors.
diff --git a/crates/core_simd/src/vector.rs b/crates/core_simd/src/vector.rs
index 9b6d0a20ed9d..07e8a6c5926c 100644
--- a/crates/core_simd/src/vector.rs
+++ b/crates/core_simd/src/vector.rs
@@ -11,40 +11,40 @@ pub(crate) mod ptr;
 
 use crate::{LaneCount, Mask, MaskElement, SupportedLaneCount};
 
-/// A SIMD vector of `LANES` elements of type `Element`.
+/// A SIMD vector of `LANES` elements of type `T`.
 #[repr(simd)]
-pub struct Simd<Element, const LANES: usize>([Element; LANES])
+pub struct Simd<T, const LANES: usize>([T; LANES])
 where
-    Element: SimdElement,
+    T: SimdElement,
     LaneCount<LANES>: SupportedLaneCount;
 
-impl<Element, const LANES: usize> Simd<Element, LANES>
+impl<T, const LANES: usize> Simd<T, LANES>
 where
     LaneCount<LANES>: SupportedLaneCount,
-    Element: SimdElement,
+    T: SimdElement,
 {
     /// Construct a SIMD vector by setting all lanes to the given value.
-    pub const fn splat(value: Element) -> Self {
+    pub const fn splat(value: T) -> Self {
         Self([value; LANES])
     }
 
     /// Returns an array reference containing the entire SIMD vector.
-    pub const fn as_array(&self) -> &[Element; LANES] {
+    pub const fn as_array(&self) -> &[T; LANES] {
         &self.0
     }
 
     /// Returns a mutable array reference containing the entire SIMD vector.
-    pub fn as_mut_array(&mut self) -> &mut [Element; LANES] {
+    pub fn as_mut_array(&mut self) -> &mut [T; LANES] {
         &mut self.0
     }
 
     /// Converts an array to a SIMD vector.
-    pub const fn from_array(array: [Element; LANES]) -> Self {
+    pub const fn from_array(array: [T; LANES]) -> Self {
         Self(array)
     }
 
     /// Converts a SIMD vector to an array.
-    pub const fn to_array(self) -> [Element; LANES] {
+    pub const fn to_array(self) -> [T; LANES] {
         self.0
     }
 
@@ -62,7 +62,7 @@ where
     /// ```
     #[must_use]
     #[inline]
-    pub fn gather_or(slice: &[Element], idxs: Simd<usize, LANES>, or: Self) -> Self {
+    pub fn gather_or(slice: &[T], idxs: Simd<usize, LANES>, or: Self) -> Self {
         Self::gather_select(slice, Mask::splat(true), idxs, or)
     }
 
@@ -79,11 +79,11 @@ where
     /// ```
     #[must_use]
     #[inline]
-    pub fn gather_or_default(slice: &[Element], idxs: Simd<usize, LANES>) -> Self
+    pub fn gather_or_default(slice: &[T], idxs: Simd<usize, LANES>) -> Self
     where
-        Element: Default,
+        T: Default,
     {
-        Self::gather_or(slice, idxs, Self::splat(Element::default()))
+        Self::gather_or(slice, idxs, Self::splat(T::default()))
     }
 
     /// SIMD gather: construct a SIMD vector by reading from a slice, using potentially discontiguous indices.
@@ -102,7 +102,7 @@ where
     #[must_use]
     #[inline]
     pub fn gather_select(
-        slice: &[Element],
+        slice: &[T],
         mask: Mask<isize, LANES>,
         idxs: Simd<usize, LANES>,
         or: Self,
@@ -129,7 +129,7 @@ where
     /// assert_eq!(vec, vec![124, 11, 12, 82, 14, 15, 16, 17, 18]);
     /// ```
     #[inline]
-    pub fn scatter(self, slice: &mut [Element], idxs: Simd<usize, LANES>) {
+    pub fn scatter(self, slice: &mut [T], idxs: Simd<usize, LANES>) {
         self.scatter_select(slice, Mask::splat(true), idxs)
     }
 
@@ -150,7 +150,7 @@ where
     #[inline]
     pub fn scatter_select(
         self,
-        slice: &mut [Element],
+        slice: &mut [T],
         mask: Mask<isize, LANES>,
         idxs: Simd<usize, LANES>,
     ) {
@@ -178,16 +178,16 @@ where
     }
 }
 
-impl<Element, const LANES: usize> Copy for Simd<Element, LANES>
+impl<T, const LANES: usize> Copy for Simd<T, LANES>
 where
-    Element: SimdElement,
+    T: SimdElement,
     LaneCount<LANES>: SupportedLaneCount,
 {
 }
 
-impl<Element, const LANES: usize> Clone for Simd<Element, LANES>
+impl<T, const LANES: usize> Clone for Simd<T, LANES>
 where
-    Element: SimdElement,
+    T: SimdElement,
     LaneCount<LANES>: SupportedLaneCount,
 {
     fn clone(&self) -> Self {
@@ -195,21 +195,21 @@ where
     }
 }
 
-impl<Element, const LANES: usize> Default for Simd<Element, LANES>
+impl<T, const LANES: usize> Default for Simd<T, LANES>
 where
     LaneCount<LANES>: SupportedLaneCount,
-    Element: SimdElement + Default,
+    T: SimdElement + Default,
 {
     #[inline]
     fn default() -> Self {
-        Self::splat(Element::default())
+        Self::splat(T::default())
     }
 }
 
-impl<Element, const LANES: usize> PartialEq for Simd<Element, LANES>
+impl<T, const LANES: usize> PartialEq for Simd<T, LANES>
 where
     LaneCount<LANES>: SupportedLaneCount,
-    Element: SimdElement + PartialEq,
+    T: SimdElement + PartialEq,
 {
     #[inline]
     fn eq(&self, other: &Self) -> bool {
@@ -218,10 +218,10 @@ where
     }
 }
 
-impl<Element, const LANES: usize> PartialOrd for Simd<Element, LANES>
+impl<T, const LANES: usize> PartialOrd for Simd<T, LANES>
 where
     LaneCount<LANES>: SupportedLaneCount,
-    Element: SimdElement + PartialOrd,
+    T: SimdElement + PartialOrd,
 {
     #[inline]
     fn partial_cmp(&self, other: &Self) -> Option<core::cmp::Ordering> {
@@ -230,17 +230,17 @@ where
     }
 }
 
-impl<Element, const LANES: usize> Eq for Simd<Element, LANES>
+impl<T, const LANES: usize> Eq for Simd<T, LANES>
 where
     LaneCount<LANES>: SupportedLaneCount,
-    Element: SimdElement + Eq,
+    T: SimdElement + Eq,
 {
 }
 
-impl<Element, const LANES: usize> Ord for Simd<Element, LANES>
+impl<T, const LANES: usize> Ord for Simd<T, LANES>
 where
     LaneCount<LANES>: SupportedLaneCount,
-    Element: SimdElement + Ord,
+    T: SimdElement + Ord,
 {
     #[inline]
     fn cmp(&self, other: &Self) -> core::cmp::Ordering {
@@ -249,10 +249,10 @@ where
     }
 }
 
-impl<Element, const LANES: usize> core::hash::Hash for Simd<Element, LANES>
+impl<T, const LANES: usize> core::hash::Hash for Simd<T, LANES>
 where
     LaneCount<LANES>: SupportedLaneCount,
-    Element: SimdElement + core::hash::Hash,
+    T: SimdElement + core::hash::Hash,
 {
     #[inline]
     fn hash<H>(&self, state: &mut H)
@@ -264,68 +264,68 @@ where
 }
 
 // array references
-impl<Element, const LANES: usize> AsRef<[Element; LANES]> for Simd<Element, LANES>
+impl<T, const LANES: usize> AsRef<[T; LANES]> for Simd<T, LANES>
 where
     LaneCount<LANES>: SupportedLaneCount,
-    Element: SimdElement,
+    T: SimdElement,
 {
     #[inline]
-    fn as_ref(&self) -> &[Element; LANES] {
+    fn as_ref(&self) -> &[T; LANES] {
         &self.0
     }
 }
 
-impl<Element, const LANES: usize> AsMut<[Element; LANES]> for Simd<Element, LANES>
+impl<T, const LANES: usize> AsMut<[T; LANES]> for Simd<T, LANES>
 where
     LaneCount<LANES>: SupportedLaneCount,
-    Element: SimdElement,
+    T: SimdElement,
 {
     #[inline]
-    fn as_mut(&mut self) -> &mut [Element; LANES] {
+    fn as_mut(&mut self) -> &mut [T; LANES] {
         &mut self.0
     }
 }
 
 // slice references
-impl<Element, const LANES: usize> AsRef<[Element]> for Simd<Element, LANES>
+impl<T, const LANES: usize> AsRef<[T]> for Simd<T, LANES>
 where
     LaneCount<LANES>: SupportedLaneCount,
-    Element: SimdElement,
+    T: SimdElement,
 {
     #[inline]
-    fn as_ref(&self) -> &[Element] {
+    fn as_ref(&self) -> &[T] {
         &self.0
     }
 }
 
-impl<Element, const LANES: usize> AsMut<[Element]> for Simd<Element, LANES>
+impl<T, const LANES: usize> AsMut<[T]> for Simd<T, LANES>
 where
     LaneCount<LANES>: SupportedLaneCount,
-    Element: SimdElement,
+    T: SimdElement,
 {
     #[inline]
-    fn as_mut(&mut self) -> &mut [Element] {
+    fn as_mut(&mut self) -> &mut [T] {
         &mut self.0
     }
 }
 
 // vector/array conversion
-impl<Element, const LANES: usize> From<[Element; LANES]> for Simd<Element, LANES>
+impl<T, const LANES: usize> From<[T; LANES]> for Simd<T, LANES>
 where
     LaneCount<LANES>: SupportedLaneCount,
-    Element: SimdElement,
+    T: SimdElement,
 {
-    fn from(array: [Element; LANES]) -> Self {
+    fn from(array: [T; LANES]) -> Self {
         Self(array)
     }
 }
 
-impl<Element, const LANES: usize> From<Simd<Element, LANES>> for [Element; LANES]
+impl<T, const LANES: usize> From<Simd<T, LANES>> for [T; LANES]
 where
     LaneCount<LANES>: SupportedLaneCount,
-    Element: SimdElement,
+    T: SimdElement,
 {
-    fn from(vector: Simd<Element, LANES>) -> Self {
+    fn from(vector: Simd<T, LANES>) -> Self {
         vector.to_array()
     }
 }

From 8cf7a62e5d2552961df51e5200aaa5b7c890a4bf Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Wed, 8 Sep 2021 20:01:16 -0400
Subject: [PATCH 220/249] Fix cargo features for nightly (#155)

* Fix cargo features for nightly
---
 .github/workflows/ci.yml                 | 4 ++--
 crates/core_simd/Cargo.toml              | 4 ++--
 crates/core_simd/src/lib.rs              | 8 ++++----
 crates/core_simd/src/masks.rs            | 2 ++
 crates/core_simd/src/masks/bitmask.rs    | 2 ++
 crates/core_simd/src/masks/full_masks.rs | 2 ++
 crates/core_simd/tests/masks.rs          | 1 +
 crates/core_simd/tests/to_bytes.rs       | 4 ++--
 8 files changed, 17 insertions(+), 10 deletions(-)

diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
index 454bc3154751..a9768f53852c 100644
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -208,8 +208,8 @@ jobs:
         features:
           - ""
           - "--features std"
-          - "--features const_evaluatable_checked"
-          - "--features std --features const_evaluatable_checked"
+          - "--features generic_const_exprs"
+          - "--features std --features generic_const_exprs"
 
     steps:
       - uses: actions/checkout@v2
diff --git a/crates/core_simd/Cargo.toml b/crates/core_simd/Cargo.toml
index c45dde2dbd21..9e8d742d83c7 100644
--- a/crates/core_simd/Cargo.toml
+++ b/crates/core_simd/Cargo.toml
@@ -9,9 +9,9 @@ categories = ["hardware-support", "no-std"]
 license = "MIT OR Apache-2.0"
 
 [features]
-default = ["std", "const_evaluatable_checked"]
+default = ["std", "generic_const_exprs"]
 std = []
-const_evaluatable_checked = []
+generic_const_exprs = []
 
 [target.'cfg(target_arch = "wasm32")'.dev-dependencies.wasm-bindgen]
 version = "0.2"
diff --git a/crates/core_simd/src/lib.rs b/crates/core_simd/src/lib.rs
index fc0df1813b94..7f07aa6393e2 100644
--- a/crates/core_simd/src/lib.rs
+++ b/crates/core_simd/src/lib.rs
@@ -1,15 +1,15 @@
-#![no_std]
+#![cfg_attr(not(feature = "std"), no_std)]
 #![allow(incomplete_features)]
 #![feature(
-    const_evaluatable_checked,
+    adt_const_params,
     const_fn_trait_bound,
-    const_generics,
     platform_intrinsics,
     repr_simd,
     simd_ffi,
     staged_api,
     stdsimd
 )]
+#![cfg_attr(feature = "generic_const_exprs", feature(generic_const_exprs))]
 #![warn(missing_docs)]
 #![unstable(feature = "portable_simd", issue = "86656")]
 //! Portable SIMD module.
@@ -22,7 +22,7 @@ mod reduction;
 mod select;
 pub use select::Select;
 
-#[cfg(feature = "const_evaluatable_checked")]
+#[cfg(feature = "generic_const_exprs")]
 mod to_bytes;
 
 mod comparisons;
diff --git a/crates/core_simd/src/masks.rs b/crates/core_simd/src/masks.rs
index 14b1fe08ffbd..ebd394cd0408 100644
--- a/crates/core_simd/src/masks.rs
+++ b/crates/core_simd/src/masks.rs
@@ -178,11 +178,13 @@ where
     }
 
     /// Convert this mask to a bitmask, with one bit set per lane.
+    #[cfg(feature = "generic_const_exprs")]
     pub fn to_bitmask(self) -> [u8; LaneCount::<LANES>::BITMASK_LEN] {
         self.0.to_bitmask()
     }
 
     /// Convert a bitmask to a mask.
+    #[cfg(feature = "generic_const_exprs")]
     pub fn from_bitmask(bitmask: [u8; LaneCount::<LANES>::BITMASK_LEN]) -> Self {
         Self(mask_impl::Mask::from_bitmask(bitmask))
     }
diff --git a/crates/core_simd/src/masks/bitmask.rs b/crates/core_simd/src/masks/bitmask.rs
index 0b5b3a5c595e..bc68b5076748 100644
--- a/crates/core_simd/src/masks/bitmask.rs
+++ b/crates/core_simd/src/masks/bitmask.rs
@@ -119,12 +119,14 @@ where
         Self(core::mem::transmute_copy(&mask), PhantomData)
     }
 
+    #[cfg(feature = "generic_const_exprs")]
     #[inline]
     pub fn to_bitmask(self) -> [u8; LaneCount::<LANES>::BITMASK_LEN] {
         // Safety: these are the same type and we are laundering the generic
         unsafe { core::mem::transmute_copy(&self.0) }
     }
 
+    #[cfg(feature = "generic_const_exprs")]
     #[inline]
     pub fn from_bitmask(bitmask: [u8; LaneCount::<LANES>::BITMASK_LEN]) -> Self {
         // Safety: these are the same type and we are laundering the generic
diff --git a/crates/core_simd/src/masks/full_masks.rs b/crates/core_simd/src/masks/full_masks.rs
index 9c1cc4623f9a..5b783a7b6a12 100644
--- a/crates/core_simd/src/masks/full_masks.rs
+++ b/crates/core_simd/src/masks/full_masks.rs
@@ -101,6 +101,7 @@ where
         unsafe { Mask(crate::intrinsics::simd_cast(self.0)) }
     }
 
+    #[cfg(feature = "generic_const_exprs")]
     #[inline]
     pub fn to_bitmask(self) -> [u8; LaneCount::<LANES>::BITMASK_LEN] {
         unsafe {
@@ -127,6 +128,7 @@ where
         }
     }
 
+    #[cfg(feature = "generic_const_exprs")]
     #[inline]
     pub fn from_bitmask(mut bitmask: [u8; LaneCount::<LANES>::BITMASK_LEN]) -> Self {
         unsafe {
diff --git a/crates/core_simd/tests/masks.rs b/crates/core_simd/tests/masks.rs
index cf8039d153d5..c2d400d79d49 100644
--- a/crates/core_simd/tests/masks.rs
+++ b/crates/core_simd/tests/masks.rs
@@ -68,6 +68,7 @@ macro_rules! test_mask_api {
                 assert_eq!(core_simd::Mask::<$type, 8>::from_int(int), mask);
             }
 
+            #[cfg(feature = "generic_const_exprs")]
             #[test]
             fn roundtrip_bitmask_conversion() {
                 let values = [
diff --git a/crates/core_simd/tests/to_bytes.rs b/crates/core_simd/tests/to_bytes.rs
index c66c9d5bd36f..debb4335e2c9 100644
--- a/crates/core_simd/tests/to_bytes.rs
+++ b/crates/core_simd/tests/to_bytes.rs
@@ -1,6 +1,6 @@
-#![feature(portable_simd, const_generics, const_evaluatable_checked)]
+#![feature(portable_simd, generic_const_exprs, adt_const_params)]
 #![allow(incomplete_features)]
-#![cfg(feature = "const_evaluatable_checked")]
+#![cfg(feature = "generic_const_exprs")]
 
 use core_simd::Simd;
 

From b25ed7f86d457d64194740730136bf72e9b92aaf Mon Sep 17 00:00:00 2001
From: Jubilee Young <workingjubilee@gmail.com>
Date: Sat, 18 Sep 2021 18:31:49 -0700
Subject: [PATCH 221/249] Restructure crate as core module

Aligns module with rust-lang/library/core, creating an... unusual
architecture that is easier to pull in as a module, as core itself can
have no dependencies (as we haven't built core yet).
---
 crates/core_simd/src/comparisons.rs      | 15 ++++++-----
 crates/core_simd/src/core_simd_docs.md   |  4 +++
 crates/core_simd/src/fmt.rs              | 17 +++++++-----
 crates/core_simd/src/intrinsics.rs       |  6 ++---
 crates/core_simd/src/iter.rs             |  4 +--
 crates/core_simd/src/lib.rs              | 31 +++-------------------
 crates/core_simd/src/masks.rs            | 12 +++++----
 crates/core_simd/src/masks/bitmask.rs    | 11 +++-----
 crates/core_simd/src/masks/full_masks.rs | 19 +++++++-------
 crates/core_simd/src/math.rs             | 11 ++++----
 crates/core_simd/src/mod.rs              | 33 ++++++++++++++++++++++++
 crates/core_simd/src/ops.rs              | 25 +++++++++---------
 crates/core_simd/src/permute.rs          |  9 ++++---
 crates/core_simd/src/reduction.rs        | 28 +++++++++++---------
 crates/core_simd/src/round.rs            | 15 ++++++-----
 crates/core_simd/src/select.rs           |  5 ++--
 crates/core_simd/src/to_bytes.rs         | 10 +++----
 crates/core_simd/src/vector.rs           | 11 ++++----
 crates/core_simd/src/vector/float.rs     |  9 ++++---
 crates/core_simd/src/vector/int.rs       |  2 +-
 crates/core_simd/src/vector/ptr.rs       |  2 +-
 crates/core_simd/src/vector/uint.rs      |  2 +-
 crates/core_simd/src/vendor/arm.rs       |  2 +-
 crates/core_simd/src/vendor/powerpc.rs   |  2 +-
 crates/core_simd/src/vendor/wasm32.rs    |  2 +-
 crates/core_simd/src/vendor/x86.rs       |  2 +-
 26 files changed, 159 insertions(+), 130 deletions(-)
 create mode 100644 crates/core_simd/src/core_simd_docs.md
 create mode 100644 crates/core_simd/src/mod.rs

diff --git a/crates/core_simd/src/comparisons.rs b/crates/core_simd/src/comparisons.rs
index 601576e094fa..8c51baca8ede 100644
--- a/crates/core_simd/src/comparisons.rs
+++ b/crates/core_simd/src/comparisons.rs
@@ -1,4 +1,5 @@
-use crate::{LaneCount, Mask, Simd, SimdElement, SupportedLaneCount};
+use crate::simd::intrinsics;
+use crate::simd::{LaneCount, Mask, Simd, SimdElement, SupportedLaneCount};
 
 impl<T, const LANES: usize> Simd<T, LANES>
 where
@@ -8,13 +9,13 @@ where
     /// Test if each lane is equal to the corresponding lane in `other`.
     #[inline]
     pub fn lanes_eq(self, other: Self) -> Mask<T::Mask, LANES> {
-        unsafe { Mask::from_int_unchecked(crate::intrinsics::simd_eq(self, other)) }
+        unsafe { Mask::from_int_unchecked(intrinsics::simd_eq(self, other)) }
     }
 
     /// Test if each lane is not equal to the corresponding lane in `other`.
     #[inline]
     pub fn lanes_ne(self, other: Self) -> Mask<T::Mask, LANES> {
-        unsafe { Mask::from_int_unchecked(crate::intrinsics::simd_ne(self, other)) }
+        unsafe { Mask::from_int_unchecked(intrinsics::simd_ne(self, other)) }
     }
 }
 
@@ -26,24 +27,24 @@ where
     /// Test if each lane is less than the corresponding lane in `other`.
     #[inline]
     pub fn lanes_lt(self, other: Self) -> Mask<T::Mask, LANES> {
-        unsafe { Mask::from_int_unchecked(crate::intrinsics::simd_lt(self, other)) }
+        unsafe { Mask::from_int_unchecked(intrinsics::simd_lt(self, other)) }
     }
 
     /// Test if each lane is greater than the corresponding lane in `other`.
     #[inline]
     pub fn lanes_gt(self, other: Self) -> Mask<T::Mask, LANES> {
-        unsafe { Mask::from_int_unchecked(crate::intrinsics::simd_gt(self, other)) }
+        unsafe { Mask::from_int_unchecked(intrinsics::simd_gt(self, other)) }
     }
 
     /// Test if each lane is less than or equal to the corresponding lane in `other`.
     #[inline]
     pub fn lanes_le(self, other: Self) -> Mask<T::Mask, LANES> {
-        unsafe { Mask::from_int_unchecked(crate::intrinsics::simd_le(self, other)) }
+        unsafe { Mask::from_int_unchecked(intrinsics::simd_le(self, other)) }
     }
 
     /// Test if each lane is greater than or equal to the corresponding lane in `other`.
     #[inline]
     pub fn lanes_ge(self, other: Self) -> Mask<T::Mask, LANES> {
-        unsafe { Mask::from_int_unchecked(crate::intrinsics::simd_ge(self, other)) }
+        unsafe { Mask::from_int_unchecked(intrinsics::simd_ge(self, other)) }
     }
 }
diff --git a/crates/core_simd/src/core_simd_docs.md b/crates/core_simd/src/core_simd_docs.md
new file mode 100644
index 000000000000..15e8ed0253e1
--- /dev/null
+++ b/crates/core_simd/src/core_simd_docs.md
@@ -0,0 +1,4 @@
+Portable SIMD module.
+
+This module offers a portable abstraction for SIMD operations
+that is not bound to any particular hardware architecture.
diff --git a/crates/core_simd/src/fmt.rs b/crates/core_simd/src/fmt.rs
index c3947c92f2a9..dbd9839c4bfe 100644
--- a/crates/core_simd/src/fmt.rs
+++ b/crates/core_simd/src/fmt.rs
@@ -1,17 +1,20 @@
+use crate::simd::{LaneCount, Simd, SimdElement, SupportedLaneCount};
+use core::fmt;
+
 macro_rules! impl_fmt_trait {
     { $($trait:ident,)* } => {
         $(
-            impl<T, const LANES: usize> core::fmt::$trait for crate::Simd<T, LANES>
+            impl<T, const LANES: usize> fmt::$trait for Simd<T, LANES>
             where
-                crate::LaneCount<LANES>: crate::SupportedLaneCount,
-                T: crate::SimdElement + core::fmt::$trait,
+                LaneCount<LANES>: SupportedLaneCount,
+                T: SimdElement + fmt::$trait,
             {
-                fn fmt(&self, f: &mut core::fmt::Formatter) -> core::fmt::Result {
+                fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
                     #[repr(transparent)]
-                    struct Wrapper<'a, T: core::fmt::$trait>(&'a T);
+                    struct Wrapper<'a, T: fmt::$trait>(&'a T);
 
-                    impl<T: core::fmt::$trait> core::fmt::Debug for Wrapper<'_, T> {
-                        fn fmt(&self, f: &mut core::fmt::Formatter) -> core::fmt::Result {
+                    impl<T: fmt::$trait> fmt::Debug for Wrapper<'_, T> {
+                        fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
                             self.0.fmt(f)
                         }
                     }
diff --git a/crates/core_simd/src/intrinsics.rs b/crates/core_simd/src/intrinsics.rs
index 916c0dadf752..3ed9845d608e 100644
--- a/crates/core_simd/src/intrinsics.rs
+++ b/crates/core_simd/src/intrinsics.rs
@@ -91,9 +91,9 @@ extern "platform-intrinsic" {
     pub(crate) fn simd_bitmask<T, U>(x: T) -> U;
 
     // select
-    pub(crate) fn simd_select<T, U>(m: T, a: U, b: U) -> U;
+    pub(crate) fn simd_select<M, T>(m: M, a: T, b: T) -> T;
     #[allow(unused)]
-    pub(crate) fn simd_select_bitmask<T, U>(m: T, a: U, b: U) -> U;
+    pub(crate) fn simd_select_bitmask<M, T>(m: M, a: T, b: T) -> T;
 }
 
 #[cfg(feature = "std")]
@@ -114,4 +114,4 @@ mod std {
 }
 
 #[cfg(feature = "std")]
-pub(crate) use crate::intrinsics::std::*;
+pub(crate) use crate::simd::intrinsics::std::*;
diff --git a/crates/core_simd/src/iter.rs b/crates/core_simd/src/iter.rs
index f403f4d90473..3275b4db8e49 100644
--- a/crates/core_simd/src/iter.rs
+++ b/crates/core_simd/src/iter.rs
@@ -1,4 +1,4 @@
-use crate::{LaneCount, Simd, SupportedLaneCount};
+use crate::simd::{LaneCount, Simd, SupportedLaneCount};
 use core::{
     iter::{Product, Sum},
     ops::{Add, Mul},
@@ -15,7 +15,7 @@ macro_rules! impl_traits {
             }
         }
 
-        impl<const LANES: usize> core::iter::Product<Self> for Simd<$type, LANES>
+        impl<const LANES: usize> Product<Self> for Simd<$type, LANES>
         where
             LaneCount<LANES>: SupportedLaneCount,
         {
diff --git a/crates/core_simd/src/lib.rs b/crates/core_simd/src/lib.rs
index 7f07aa6393e2..279999b09e2f 100644
--- a/crates/core_simd/src/lib.rs
+++ b/crates/core_simd/src/lib.rs
@@ -14,31 +14,6 @@
 #![unstable(feature = "portable_simd", issue = "86656")]
 //! Portable SIMD module.
 
-#[macro_use]
-mod permute;
-#[macro_use]
-mod reduction;
-
-mod select;
-pub use select::Select;
-
-#[cfg(feature = "generic_const_exprs")]
-mod to_bytes;
-
-mod comparisons;
-mod fmt;
-mod intrinsics;
-mod iter;
-mod math;
-mod ops;
-mod round;
-mod vendor;
-
-mod lane_count;
-pub use lane_count::*;
-
-mod masks;
-pub use masks::*;
-
-mod vector;
-pub use vector::*;
+#[path = "mod.rs"]
+mod core_simd;
+pub use self::core_simd::simd::*;
diff --git a/crates/core_simd/src/masks.rs b/crates/core_simd/src/masks.rs
index ebd394cd0408..c4d6e1883483 100644
--- a/crates/core_simd/src/masks.rs
+++ b/crates/core_simd/src/masks.rs
@@ -12,7 +12,9 @@
 )]
 mod mask_impl;
 
-use crate::{LaneCount, Simd, SimdElement, SupportedLaneCount};
+use crate::simd::{LaneCount, Simd, SimdElement, SupportedLaneCount};
+use core::cmp::Ordering;
+use core::fmt;
 
 /// Marker trait for types that may be used as SIMD mask elements.
 pub unsafe trait MaskElement: SimdElement {
@@ -251,17 +253,17 @@ where
     LaneCount<LANES>: SupportedLaneCount,
 {
     #[inline]
-    fn partial_cmp(&self, other: &Self) -> Option<core::cmp::Ordering> {
+    fn partial_cmp(&self, other: &Self) -> Option<Ordering> {
         self.0.partial_cmp(&other.0)
     }
 }
 
-impl<T, const LANES: usize> core::fmt::Debug for Mask<T, LANES>
+impl<T, const LANES: usize> fmt::Debug for Mask<T, LANES>
 where
-    T: MaskElement + core::fmt::Debug,
+    T: MaskElement + fmt::Debug,
     LaneCount<LANES>: SupportedLaneCount,
 {
-    fn fmt(&self, f: &mut core::fmt::Formatter) -> core::fmt::Result {
+    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
         f.debug_list()
             .entries((0..LANES).map(|lane| self.test(lane)))
             .finish()
diff --git a/crates/core_simd/src/masks/bitmask.rs b/crates/core_simd/src/masks/bitmask.rs
index bc68b5076748..0691c6ecd218 100644
--- a/crates/core_simd/src/masks/bitmask.rs
+++ b/crates/core_simd/src/masks/bitmask.rs
@@ -1,4 +1,5 @@
-use crate::{LaneCount, MaskElement, Simd, SupportedLaneCount};
+use crate::simd::intrinsics;
+use crate::simd::{LaneCount, Simd, SupportedLaneCount};
 use core::marker::PhantomData;
 
 /// A mask where each lane is represented by a single bit.
@@ -99,11 +100,7 @@ where
         unsafe {
             let mask: <LaneCount<LANES> as SupportedLaneCount>::IntBitMask =
                 core::mem::transmute_copy(&self);
-            crate::intrinsics::simd_select_bitmask(
-                mask,
-                Simd::splat(T::TRUE),
-                Simd::splat(T::FALSE),
-            )
+            intrinsics::simd_select_bitmask(mask, Simd::splat(T::TRUE), Simd::splat(T::FALSE))
         }
     }
 
@@ -115,7 +112,7 @@ where
             core::mem::size_of::<<LaneCount::<LANES> as SupportedLaneCount>::IntBitMask>(),
         );
         let mask: <LaneCount<LANES> as SupportedLaneCount>::IntBitMask =
-            crate::intrinsics::simd_bitmask(value);
+            intrinsics::simd_bitmask(value);
         Self(core::mem::transmute_copy(&mask), PhantomData)
     }
 
diff --git a/crates/core_simd/src/masks/full_masks.rs b/crates/core_simd/src/masks/full_masks.rs
index 5b783a7b6a12..b653bce05b9e 100644
--- a/crates/core_simd/src/masks/full_masks.rs
+++ b/crates/core_simd/src/masks/full_masks.rs
@@ -1,7 +1,8 @@
 //! Masks that take up full SIMD vector registers.
 
 use super::MaskElement;
-use crate::{LaneCount, Simd, SupportedLaneCount};
+use crate::simd::intrinsics;
+use crate::simd::{LaneCount, Simd, SupportedLaneCount};
 
 #[repr(transparent)]
 pub struct Mask<T, const LANES: usize>(Simd<T, LANES>)
@@ -98,7 +99,7 @@ where
     where
         U: MaskElement,
     {
-        unsafe { Mask(crate::intrinsics::simd_cast(self.0)) }
+        unsafe { Mask(intrinsics::simd_cast(self.0)) }
     }
 
     #[cfg(feature = "generic_const_exprs")]
@@ -111,7 +112,7 @@ where
                 LaneCount::<LANES>::BITMASK_LEN,
             );
             let bitmask: <LaneCount<LANES> as SupportedLaneCount>::IntBitMask =
-                crate::intrinsics::simd_bitmask(self.0);
+                intrinsics::simd_bitmask(self.0);
             let mut bitmask: [u8; LaneCount::<LANES>::BITMASK_LEN] =
                 core::mem::transmute_copy(&bitmask);
 
@@ -149,7 +150,7 @@ where
             let bitmask: <LaneCount<LANES> as SupportedLaneCount>::IntBitMask =
                 core::mem::transmute_copy(&bitmask);
 
-            Self::from_int_unchecked(crate::intrinsics::simd_select_bitmask(
+            Self::from_int_unchecked(intrinsics::simd_select_bitmask(
                 bitmask,
                 Self::splat(true).to_int(),
                 Self::splat(false).to_int(),
@@ -159,12 +160,12 @@ where
 
     #[inline]
     pub fn any(self) -> bool {
-        unsafe { crate::intrinsics::simd_reduce_any(self.to_int()) }
+        unsafe { intrinsics::simd_reduce_any(self.to_int()) }
     }
 
     #[inline]
     pub fn all(self) -> bool {
-        unsafe { crate::intrinsics::simd_reduce_all(self.to_int()) }
+        unsafe { intrinsics::simd_reduce_all(self.to_int()) }
     }
 }
 
@@ -186,7 +187,7 @@ where
     type Output = Self;
     #[inline]
     fn bitand(self, rhs: Self) -> Self {
-        unsafe { Self(crate::intrinsics::simd_and(self.0, rhs.0)) }
+        unsafe { Self(intrinsics::simd_and(self.0, rhs.0)) }
     }
 }
 
@@ -198,7 +199,7 @@ where
     type Output = Self;
     #[inline]
     fn bitor(self, rhs: Self) -> Self {
-        unsafe { Self(crate::intrinsics::simd_or(self.0, rhs.0)) }
+        unsafe { Self(intrinsics::simd_or(self.0, rhs.0)) }
     }
 }
 
@@ -210,7 +211,7 @@ where
     type Output = Self;
     #[inline]
     fn bitxor(self, rhs: Self) -> Self {
-        unsafe { Self(crate::intrinsics::simd_xor(self.0, rhs.0)) }
+        unsafe { Self(intrinsics::simd_xor(self.0, rhs.0)) }
     }
 }
 
diff --git a/crates/core_simd/src/math.rs b/crates/core_simd/src/math.rs
index 7affecbafd68..6ee5efdb981c 100644
--- a/crates/core_simd/src/math.rs
+++ b/crates/core_simd/src/math.rs
@@ -1,4 +1,5 @@
-use crate::{LaneCount, Simd, SupportedLaneCount};
+use crate::simd::intrinsics::{simd_saturating_add, simd_saturating_sub};
+use crate::simd::{LaneCount, Simd, SupportedLaneCount};
 
 macro_rules! impl_uint_arith {
     ($($ty:ty),+) => {
@@ -20,7 +21,7 @@ macro_rules! impl_uint_arith {
             /// ```
             #[inline]
             pub fn saturating_add(self, second: Self) -> Self {
-                unsafe { crate::intrinsics::simd_saturating_add(self, second) }
+                unsafe { simd_saturating_add(self, second) }
             }
 
             /// Lanewise saturating subtract.
@@ -38,7 +39,7 @@ macro_rules! impl_uint_arith {
             /// assert_eq!(sat, Simd::splat(0));
             #[inline]
             pub fn saturating_sub(self, second: Self) -> Self {
-                unsafe { crate::intrinsics::simd_saturating_sub(self, second) }
+                unsafe { simd_saturating_sub(self, second) }
             }
         })+
     }
@@ -64,7 +65,7 @@ macro_rules! impl_int_arith {
             /// ```
             #[inline]
             pub fn saturating_add(self, second: Self) -> Self {
-                unsafe { crate::intrinsics::simd_saturating_add(self, second) }
+                unsafe { simd_saturating_add(self, second) }
             }
 
             /// Lanewise saturating subtract.
@@ -82,7 +83,7 @@ macro_rules! impl_int_arith {
             /// assert_eq!(sat, Simd::from_array([MIN, MIN, MIN, 0]));
             #[inline]
             pub fn saturating_sub(self, second: Self) -> Self {
-                unsafe { crate::intrinsics::simd_saturating_sub(self, second) }
+                unsafe { simd_saturating_sub(self, second) }
             }
 
             /// Lanewise absolute value, implemented in Rust.
diff --git a/crates/core_simd/src/mod.rs b/crates/core_simd/src/mod.rs
new file mode 100644
index 000000000000..251091c1dc3e
--- /dev/null
+++ b/crates/core_simd/src/mod.rs
@@ -0,0 +1,33 @@
+#[macro_use]
+mod permute;
+#[macro_use]
+mod reduction;
+
+mod select;
+
+#[cfg(feature = "generic_const_exprs")]
+mod to_bytes;
+
+mod comparisons;
+mod fmt;
+mod intrinsics;
+mod iter;
+mod math;
+mod ops;
+mod round;
+mod vendor;
+
+mod lane_count;
+
+mod masks;
+
+mod vector;
+
+#[doc = include_str!("core_simd_docs.md")]
+pub mod simd {
+    pub use crate::core_simd::lane_count::*;
+    pub use crate::core_simd::masks::*;
+    pub use crate::core_simd::select::Select;
+    pub use crate::core_simd::vector::*;
+    pub(crate) use crate::core_simd::*;
+}
diff --git a/crates/core_simd/src/ops.rs b/crates/core_simd/src/ops.rs
index 900315660005..5d7af474caf7 100644
--- a/crates/core_simd/src/ops.rs
+++ b/crates/core_simd/src/ops.rs
@@ -1,4 +1,5 @@
-use crate::{LaneCount, Simd, SimdElement, SupportedLaneCount};
+use crate::simd::intrinsics;
+use crate::simd::{LaneCount, Simd, SimdElement, SupportedLaneCount};
 
 impl<I, T, const LANES: usize> core::ops::Index<I> for Simd<T, LANES>
 where
@@ -208,7 +209,7 @@ macro_rules! impl_op {
             {
                 type Output = Self;
                 fn neg(self) -> Self::Output {
-                    unsafe { crate::intrinsics::simd_neg(self) }
+                    unsafe { intrinsics::simd_neg(self) }
                 }
             }
         }
@@ -226,7 +227,7 @@ macro_rules! impl_op {
                 #[inline]
                 fn $trait_fn(self, rhs: Self) -> Self::Output {
                     unsafe {
-                        crate::intrinsics::$intrinsic(self, rhs)
+                        intrinsics::$intrinsic(self, rhs)
                     }
                 }
             }
@@ -268,7 +269,7 @@ macro_rules! impl_op {
                 #[inline]
                 fn $assign_trait_fn(&mut self, rhs: Self) {
                     unsafe {
-                        *self = crate::intrinsics::$intrinsic(*self, rhs);
+                        *self = intrinsics::$intrinsic(*self, rhs);
                     }
                 }
             }
@@ -338,7 +339,7 @@ macro_rules! impl_unsigned_int_ops {
                                 .any(|(x,y)| *x == <$scalar>::MIN && *y == -1 as _) {
                             panic!("attempt to divide with overflow");
                         }
-                        unsafe { crate::intrinsics::simd_div(self, rhs) }
+                        unsafe { intrinsics::simd_div(self, rhs) }
                     }
                 }
             }
@@ -361,7 +362,7 @@ macro_rules! impl_unsigned_int_ops {
                                 panic!("attempt to divide with overflow");
                         }
                         let rhs = Self::splat(rhs);
-                        unsafe { crate::intrinsics::simd_div(self, rhs) }
+                        unsafe { intrinsics::simd_div(self, rhs) }
                     }
                 }
             }
@@ -428,7 +429,7 @@ macro_rules! impl_unsigned_int_ops {
                                 .any(|(x,y)| *x == <$scalar>::MIN && *y == -1 as _) {
                             panic!("attempt to calculate the remainder with overflow");
                         }
-                        unsafe { crate::intrinsics::simd_rem(self, rhs) }
+                        unsafe { intrinsics::simd_rem(self, rhs) }
                     }
                 }
             }
@@ -451,7 +452,7 @@ macro_rules! impl_unsigned_int_ops {
                                 panic!("attempt to calculate the remainder with overflow");
                         }
                         let rhs = Self::splat(rhs);
-                        unsafe { crate::intrinsics::simd_rem(self, rhs) }
+                        unsafe { intrinsics::simd_rem(self, rhs) }
                     }
                 }
             }
@@ -512,7 +513,7 @@ macro_rules! impl_unsigned_int_ops {
                         {
                             panic!("attempt to shift left with overflow");
                         }
-                        unsafe { crate::intrinsics::simd_shl(self, rhs) }
+                        unsafe { intrinsics::simd_shl(self, rhs) }
                     }
                 }
             }
@@ -530,7 +531,7 @@ macro_rules! impl_unsigned_int_ops {
                             panic!("attempt to shift left with overflow");
                         }
                         let rhs = Self::splat(rhs);
-                        unsafe { crate::intrinsics::simd_shl(self, rhs) }
+                        unsafe { intrinsics::simd_shl(self, rhs) }
                     }
                 }
             }
@@ -577,7 +578,7 @@ macro_rules! impl_unsigned_int_ops {
                         {
                             panic!("attempt to shift with overflow");
                         }
-                        unsafe { crate::intrinsics::simd_shr(self, rhs) }
+                        unsafe { intrinsics::simd_shr(self, rhs) }
                     }
                 }
             }
@@ -595,7 +596,7 @@ macro_rules! impl_unsigned_int_ops {
                             panic!("attempt to shift with overflow");
                         }
                         let rhs = Self::splat(rhs);
-                        unsafe { crate::intrinsics::simd_shr(self, rhs) }
+                        unsafe { intrinsics::simd_shr(self, rhs) }
                     }
                 }
             }
diff --git a/crates/core_simd/src/permute.rs b/crates/core_simd/src/permute.rs
index cc58778b6b4b..206519340b3a 100644
--- a/crates/core_simd/src/permute.rs
+++ b/crates/core_simd/src/permute.rs
@@ -1,8 +1,11 @@
+use crate::simd::intrinsics;
+use crate::simd::{Simd, SimdElement};
+
 macro_rules! impl_shuffle_lane {
     { $fn:ident, $n:literal } => {
-        impl<T> crate::Simd<T, $n>
+        impl<T> Simd<T, $n>
         where
-            T: crate::SimdElement,
+            T: SimdElement,
         {
             /// A const SIMD shuffle that takes 2 SIMD vectors and produces another vector, using
             /// the indices in the const parameter. The first or "self" vector will have its lanes
@@ -24,7 +27,7 @@ macro_rules! impl_shuffle_lane {
             /// ```
             #[inline]
             pub fn shuffle<const IDX: [u32; $n]>(self, second: Self) -> Self {
-                unsafe { crate::intrinsics::$fn(self, second, IDX) }
+                unsafe { intrinsics::$fn(self, second, IDX) }
             }
 
             /// Reverse the order of the lanes in the vector.
diff --git a/crates/core_simd/src/reduction.rs b/crates/core_simd/src/reduction.rs
index 943d2856e359..db0640aae790 100644
--- a/crates/core_simd/src/reduction.rs
+++ b/crates/core_simd/src/reduction.rs
@@ -1,4 +1,8 @@
-use crate::{LaneCount, Simd, SupportedLaneCount};
+use crate::simd::intrinsics::{
+    simd_reduce_add_ordered, simd_reduce_and, simd_reduce_max, simd_reduce_min,
+    simd_reduce_mul_ordered, simd_reduce_or, simd_reduce_xor,
+};
+use crate::simd::{LaneCount, Simd, SupportedLaneCount};
 
 macro_rules! impl_integer_reductions {
     { $scalar:ty } => {
@@ -9,46 +13,46 @@ macro_rules! impl_integer_reductions {
             /// Horizontal wrapping add.  Returns the sum of the lanes of the vector, with wrapping addition.
             #[inline]
             pub fn horizontal_sum(self) -> $scalar {
-                unsafe { crate::intrinsics::simd_reduce_add_ordered(self, 0) }
+                unsafe { simd_reduce_add_ordered(self, 0) }
             }
 
             /// Horizontal wrapping multiply.  Returns the product of the lanes of the vector, with wrapping multiplication.
             #[inline]
             pub fn horizontal_product(self) -> $scalar {
-                unsafe { crate::intrinsics::simd_reduce_mul_ordered(self, 1) }
+                unsafe { simd_reduce_mul_ordered(self, 1) }
             }
 
             /// Horizontal bitwise "and".  Returns the cumulative bitwise "and" across the lanes of
             /// the vector.
             #[inline]
             pub fn horizontal_and(self) -> $scalar {
-                unsafe { crate::intrinsics::simd_reduce_and(self) }
+                unsafe { simd_reduce_and(self) }
             }
 
             /// Horizontal bitwise "or".  Returns the cumulative bitwise "or" across the lanes of
             /// the vector.
             #[inline]
             pub fn horizontal_or(self) -> $scalar {
-                unsafe { crate::intrinsics::simd_reduce_or(self) }
+                unsafe { simd_reduce_or(self) }
             }
 
             /// Horizontal bitwise "xor".  Returns the cumulative bitwise "xor" across the lanes of
             /// the vector.
             #[inline]
             pub fn horizontal_xor(self) -> $scalar {
-                unsafe { crate::intrinsics::simd_reduce_xor(self) }
+                unsafe { simd_reduce_xor(self) }
             }
 
             /// Horizontal maximum.  Returns the maximum lane in the vector.
             #[inline]
             pub fn horizontal_max(self) -> $scalar {
-                unsafe { crate::intrinsics::simd_reduce_max(self) }
+                unsafe { simd_reduce_max(self) }
             }
 
             /// Horizontal minimum.  Returns the minimum lane in the vector.
             #[inline]
             pub fn horizontal_min(self) -> $scalar {
-                unsafe { crate::intrinsics::simd_reduce_min(self) }
+                unsafe { simd_reduce_min(self) }
             }
         }
     }
@@ -79,7 +83,7 @@ macro_rules! impl_float_reductions {
                 if cfg!(all(target_arch = "x86", not(target_feature = "sse2"))) {
                     self.as_array().iter().sum()
                 } else {
-                    unsafe { crate::intrinsics::simd_reduce_add_ordered(self, 0.) }
+                    unsafe { simd_reduce_add_ordered(self, 0.) }
                 }
             }
 
@@ -90,7 +94,7 @@ macro_rules! impl_float_reductions {
                 if cfg!(all(target_arch = "x86", not(target_feature = "sse2"))) {
                     self.as_array().iter().product()
                 } else {
-                    unsafe { crate::intrinsics::simd_reduce_mul_ordered(self, 1.) }
+                    unsafe { simd_reduce_mul_ordered(self, 1.) }
                 }
             }
 
@@ -100,7 +104,7 @@ macro_rules! impl_float_reductions {
             /// return either.  This function will not return `NaN` unless all lanes are `NaN`.
             #[inline]
             pub fn horizontal_max(self) -> $scalar {
-                unsafe { crate::intrinsics::simd_reduce_max(self) }
+                unsafe { simd_reduce_max(self) }
             }
 
             /// Horizontal minimum.  Returns the minimum lane in the vector.
@@ -109,7 +113,7 @@ macro_rules! impl_float_reductions {
             /// return either.  This function will not return `NaN` unless all lanes are `NaN`.
             #[inline]
             pub fn horizontal_min(self) -> $scalar {
-                unsafe { crate::intrinsics::simd_reduce_min(self) }
+                unsafe { simd_reduce_min(self) }
             }
         }
     }
diff --git a/crates/core_simd/src/round.rs b/crates/core_simd/src/round.rs
index 96d46b9a1232..3bb10d0ed0bf 100644
--- a/crates/core_simd/src/round.rs
+++ b/crates/core_simd/src/round.rs
@@ -1,4 +1,5 @@
-use crate::{LaneCount, Simd, SupportedLaneCount};
+use crate::simd::intrinsics;
+use crate::simd::{LaneCount, Simd, SupportedLaneCount};
 
 macro_rules! implement {
     {
@@ -13,28 +14,28 @@ macro_rules! implement {
             #[must_use = "method returns a new vector and does not mutate the original value"]
             #[inline]
             pub fn ceil(self) -> Self {
-                unsafe { crate::intrinsics::simd_ceil(self) }
+                unsafe { intrinsics::simd_ceil(self) }
             }
 
             /// Returns the largest integer value less than or equal to each lane.
             #[must_use = "method returns a new vector and does not mutate the original value"]
             #[inline]
             pub fn floor(self) -> Self {
-                unsafe { crate::intrinsics::simd_floor(self) }
+                unsafe { intrinsics::simd_floor(self) }
             }
 
             /// Rounds to the nearest integer value. Ties round toward zero.
             #[must_use = "method returns a new vector and does not mutate the original value"]
             #[inline]
             pub fn round(self) -> Self {
-                unsafe { crate::intrinsics::simd_round(self) }
+                unsafe { intrinsics::simd_round(self) }
             }
 
             /// Returns the floating point's integer value, with its fractional part removed.
             #[must_use = "method returns a new vector and does not mutate the original value"]
             #[inline]
             pub fn trunc(self) -> Self {
-                unsafe { crate::intrinsics::simd_trunc(self) }
+                unsafe { intrinsics::simd_trunc(self) }
             }
 
             /// Returns the floating point's fractional value, with its integer part removed.
@@ -60,14 +61,14 @@ macro_rules! implement {
             /// * Be representable in the return type, after truncating off its fractional part
             #[inline]
             pub unsafe fn to_int_unchecked(self) -> Simd<$int_type, LANES> {
-                crate::intrinsics::simd_cast(self)
+                intrinsics::simd_cast(self)
             }
 
             /// Creates a floating-point vector from an integer vector.  Rounds values that are
             /// not exactly representable.
             #[inline]
             pub fn round_from_int(value: Simd<$int_type, LANES>) -> Self {
-                unsafe { crate::intrinsics::simd_cast(value) }
+                unsafe { intrinsics::simd_cast(value) }
             }
         }
     }
diff --git a/crates/core_simd/src/select.rs b/crates/core_simd/src/select.rs
index 0951639c9426..596621c5676b 100644
--- a/crates/core_simd/src/select.rs
+++ b/crates/core_simd/src/select.rs
@@ -1,4 +1,5 @@
-use crate::{LaneCount, Mask, MaskElement, Simd, SimdElement, SupportedLaneCount};
+use crate::simd::intrinsics;
+use crate::simd::{LaneCount, Mask, MaskElement, Simd, SimdElement, SupportedLaneCount};
 
 mod sealed {
     pub trait Sealed {}
@@ -25,7 +26,7 @@ where
 {
     #[inline]
     fn select(mask: Mask<T::Mask, LANES>, true_values: Self, false_values: Self) -> Self {
-        unsafe { crate::intrinsics::simd_select(mask.to_int(), true_values, false_values) }
+        unsafe { intrinsics::simd_select(mask.to_int(), true_values, false_values) }
     }
 }
 
diff --git a/crates/core_simd/src/to_bytes.rs b/crates/core_simd/src/to_bytes.rs
index bd818f532118..8d9b3e8ff85e 100644
--- a/crates/core_simd/src/to_bytes.rs
+++ b/crates/core_simd/src/to_bytes.rs
@@ -1,19 +1,19 @@
 macro_rules! impl_to_bytes {
     { $ty:ty, $size:literal } => {
-        impl<const LANES: usize> crate::Simd<$ty, LANES>
+        impl<const LANES: usize> crate::simd::Simd<$ty, LANES>
         where
-            crate::LaneCount<LANES>: crate::SupportedLaneCount,
-            crate::LaneCount<{{ $size * LANES }}>: crate::SupportedLaneCount,
+            crate::simd::LaneCount<LANES>: crate::simd::SupportedLaneCount,
+            crate::simd::LaneCount<{{ $size * LANES }}>: crate::simd::SupportedLaneCount,
         {
             /// Return the memory representation of this integer as a byte array in native byte
             /// order.
-            pub fn to_ne_bytes(self) -> crate::Simd<u8, {{ $size * LANES }}> {
+            pub fn to_ne_bytes(self) -> crate::simd::Simd<u8, {{ $size * LANES }}> {
                 unsafe { core::mem::transmute_copy(&self) }
             }
 
             /// Create a native endian integer value from its memory representation as a byte array
             /// in native endianness.
-            pub fn from_ne_bytes(bytes: crate::Simd<u8, {{ $size * LANES }}>) -> Self {
+            pub fn from_ne_bytes(bytes: crate::simd::Simd<u8, {{ $size * LANES }}>) -> Self {
                 unsafe { core::mem::transmute_copy(&bytes) }
             }
         }
diff --git a/crates/core_simd/src/vector.rs b/crates/core_simd/src/vector.rs
index 07e8a6c5926c..fb3518fa13d7 100644
--- a/crates/core_simd/src/vector.rs
+++ b/crates/core_simd/src/vector.rs
@@ -9,7 +9,8 @@ pub use uint::*;
 // Vectors of pointers are not for public use at the current time.
 pub(crate) mod ptr;
 
-use crate::{LaneCount, Mask, MaskElement, SupportedLaneCount};
+use crate::simd::intrinsics;
+use crate::simd::{LaneCount, Mask, MaskElement, SupportedLaneCount};
 
 /// A SIMD vector of `LANES` elements of type `T`.
 #[repr(simd)]
@@ -108,11 +109,11 @@ where
         or: Self,
     ) -> Self {
         let mask = (mask & idxs.lanes_lt(Simd::splat(slice.len()))).to_int();
-        let base_ptr = crate::vector::ptr::SimdConstPtr::splat(slice.as_ptr());
+        let base_ptr = crate::simd::ptr::SimdConstPtr::splat(slice.as_ptr());
         // Ferris forgive me, I have done pointer arithmetic here.
         let ptrs = base_ptr.wrapping_add(idxs);
         // SAFETY: The ptrs have been bounds-masked to prevent memory-unsafe reads insha'allah
-        unsafe { crate::intrinsics::simd_gather(or, ptrs, mask) }
+        unsafe { intrinsics::simd_gather(or, ptrs, mask) }
     }
 
     /// SIMD scatter: write a SIMD vector's values into a slice, using potentially discontiguous indices.
@@ -168,11 +169,11 @@ where
         // 3. &mut [T] which will become our base ptr.
         unsafe {
             // Now Entering ☢️ *mut T Zone
-            let base_ptr = crate::vector::ptr::SimdMutPtr::splat(slice.as_mut_ptr());
+            let base_ptr = crate::simd::ptr::SimdMutPtr::splat(slice.as_mut_ptr());
             // Ferris forgive me, I have done pointer arithmetic here.
             let ptrs = base_ptr.wrapping_add(idxs);
             // The ptrs have been bounds-masked to prevent memory-unsafe writes insha'allah
-            crate::intrinsics::simd_scatter(self, ptrs, mask)
+            intrinsics::simd_scatter(self, ptrs, mask)
             // Cleared ☢️ *mut T Zone
         }
     }
diff --git a/crates/core_simd/src/vector/float.rs b/crates/core_simd/src/vector/float.rs
index 6ef88ddebc68..21a6c43e153a 100644
--- a/crates/core_simd/src/vector/float.rs
+++ b/crates/core_simd/src/vector/float.rs
@@ -1,6 +1,7 @@
 #![allow(non_camel_case_types)]
 
-use crate::{LaneCount, Mask, Simd, SupportedLaneCount};
+use crate::simd::intrinsics;
+use crate::simd::{LaneCount, Mask, Simd, SupportedLaneCount};
 
 /// Implements inherent methods for a float vector containing multiple
 /// `$lanes` of float `$type`, which uses `$bits_ty` as its binary
@@ -31,7 +32,7 @@ macro_rules! impl_float_vector {
             /// equivalently-indexed lane in `self`.
             #[inline]
             pub fn abs(self) -> Self {
-                unsafe { crate::intrinsics::simd_fabs(self) }
+                unsafe { intrinsics::simd_fabs(self) }
             }
 
             /// Fused multiply-add.  Computes `(self * a) + b` with only one rounding error,
@@ -43,7 +44,7 @@ macro_rules! impl_float_vector {
             /// hardware in mind.
             #[inline]
             pub fn mul_add(self, a: Self, b: Self) -> Self {
-                unsafe { crate::intrinsics::simd_fma(self, a, b) }
+                unsafe { intrinsics::simd_fma(self, a, b) }
             }
 
             /// Produces a vector where every lane has the square root value
@@ -51,7 +52,7 @@ macro_rules! impl_float_vector {
             #[inline]
             #[cfg(feature = "std")]
             pub fn sqrt(self) -> Self {
-                unsafe { crate::intrinsics::simd_fsqrt(self) }
+                unsafe { intrinsics::simd_fsqrt(self) }
             }
 
             /// Takes the reciprocal (inverse) of each lane, `1/x`.
diff --git a/crates/core_simd/src/vector/int.rs b/crates/core_simd/src/vector/int.rs
index 5f435e16b687..3eac02a27618 100644
--- a/crates/core_simd/src/vector/int.rs
+++ b/crates/core_simd/src/vector/int.rs
@@ -1,6 +1,6 @@
 #![allow(non_camel_case_types)]
 
-use crate::{LaneCount, Mask, Simd, SupportedLaneCount};
+use crate::simd::{LaneCount, Mask, Simd, SupportedLaneCount};
 
 /// Implements additional integer traits (Eq, Ord, Hash) on the specified vector `$name`, holding multiple `$lanes` of `$type`.
 macro_rules! impl_integer_vector {
diff --git a/crates/core_simd/src/vector/ptr.rs b/crates/core_simd/src/vector/ptr.rs
index fc4082a4b555..ac9b98ca031a 100644
--- a/crates/core_simd/src/vector/ptr.rs
+++ b/crates/core_simd/src/vector/ptr.rs
@@ -1,5 +1,5 @@
 //! Private implementation details of public gather/scatter APIs.
-use crate::{LaneCount, Simd, SupportedLaneCount};
+use crate::simd::{LaneCount, Simd, SupportedLaneCount};
 use core::mem;
 
 /// A vector of *const T.
diff --git a/crates/core_simd/src/vector/uint.rs b/crates/core_simd/src/vector/uint.rs
index b3dd199a5463..ed91fc3640e7 100644
--- a/crates/core_simd/src/vector/uint.rs
+++ b/crates/core_simd/src/vector/uint.rs
@@ -1,6 +1,6 @@
 #![allow(non_camel_case_types)]
 
-use crate::Simd;
+use crate::simd::Simd;
 
 /// Vector of two `usize` values
 pub type usizex2 = Simd<usize, 2>;
diff --git a/crates/core_simd/src/vendor/arm.rs b/crates/core_simd/src/vendor/arm.rs
index e39173a9c3c4..3e9487dfb33d 100644
--- a/crates/core_simd/src/vendor/arm.rs
+++ b/crates/core_simd/src/vendor/arm.rs
@@ -1,4 +1,4 @@
-use crate::*;
+use crate::simd::*;
 
 #[cfg(target_arch = "arm")]
 use core::arch::arm::*;
diff --git a/crates/core_simd/src/vendor/powerpc.rs b/crates/core_simd/src/vendor/powerpc.rs
index 248764efd516..92f97d471b6e 100644
--- a/crates/core_simd/src/vendor/powerpc.rs
+++ b/crates/core_simd/src/vendor/powerpc.rs
@@ -1,4 +1,4 @@
-use crate::*;
+use crate::simd::*;
 
 #[cfg(target_arch = "powerpc")]
 use core::arch::powerpc::*;
diff --git a/crates/core_simd/src/vendor/wasm32.rs b/crates/core_simd/src/vendor/wasm32.rs
index ef67572b534d..ef3baf885b0f 100644
--- a/crates/core_simd/src/vendor/wasm32.rs
+++ b/crates/core_simd/src/vendor/wasm32.rs
@@ -1,4 +1,4 @@
-use crate::*;
+use crate::simd::*;
 use core::arch::wasm32::v128;
 
 from_transmute! { unsafe u8x16 => v128 }
diff --git a/crates/core_simd/src/vendor/x86.rs b/crates/core_simd/src/vendor/x86.rs
index 0090c3756481..d3c19ccc539a 100644
--- a/crates/core_simd/src/vendor/x86.rs
+++ b/crates/core_simd/src/vendor/x86.rs
@@ -1,4 +1,4 @@
-use crate::*;
+use crate::simd::*;
 
 #[cfg(any(target_arch = "x86"))]
 use core::arch::x86::*;

From 8342fe75f2925cda4787561272eec87efe596303 Mon Sep 17 00:00:00 2001
From: Jubilee Young <workingjubilee@gmail.com>
Date: Mon, 20 Sep 2021 22:34:02 -0700
Subject: [PATCH 222/249] Cleanup more for std::simd also

---
 crates/core_simd/src/lib.rs |  3 ++-
 crates/core_simd/src/mod.rs | 18 ++++++++----------
 2 files changed, 10 insertions(+), 11 deletions(-)

diff --git a/crates/core_simd/src/lib.rs b/crates/core_simd/src/lib.rs
index 279999b09e2f..486905e4d811 100644
--- a/crates/core_simd/src/lib.rs
+++ b/crates/core_simd/src/lib.rs
@@ -16,4 +16,5 @@
 
 #[path = "mod.rs"]
 mod core_simd;
-pub use self::core_simd::simd::*;
+use self::core_simd::simd;
+pub use simd::*;
diff --git a/crates/core_simd/src/mod.rs b/crates/core_simd/src/mod.rs
index 251091c1dc3e..5696570d23e6 100644
--- a/crates/core_simd/src/mod.rs
+++ b/crates/core_simd/src/mod.rs
@@ -3,31 +3,29 @@ mod permute;
 #[macro_use]
 mod reduction;
 
-mod select;
+pub(crate) mod intrinsics;
 
 #[cfg(feature = "generic_const_exprs")]
 mod to_bytes;
 
 mod comparisons;
 mod fmt;
-mod intrinsics;
 mod iter;
+mod lane_count;
+mod masks;
 mod math;
 mod ops;
 mod round;
-mod vendor;
-
-mod lane_count;
-
-mod masks;
-
+mod select;
 mod vector;
+mod vendor;
 
 #[doc = include_str!("core_simd_docs.md")]
 pub mod simd {
-    pub use crate::core_simd::lane_count::*;
+    pub(crate) use crate::core_simd::intrinsics;
+
+    pub use crate::core_simd::lane_count::{LaneCount, SupportedLaneCount};
     pub use crate::core_simd::masks::*;
     pub use crate::core_simd::select::Select;
     pub use crate::core_simd::vector::*;
-    pub(crate) use crate::core_simd::*;
 }

From 6d3d07abfede1f885230ecefa2a076919ef6290c Mon Sep 17 00:00:00 2001
From: Jubilee Young <workingjubilee@gmail.com>
Date: Tue, 21 Sep 2021 17:03:25 -0700
Subject: [PATCH 223/249] Feature-flag doc tests so they run for core

---
 crates/core_simd/src/math.rs    | 21 ++++++++++++++-------
 crates/core_simd/src/permute.rs |  9 ++++++---
 crates/core_simd/src/select.rs  |  6 ++++--
 crates/core_simd/src/vector.rs  | 15 ++++++++++-----
 4 files changed, 34 insertions(+), 17 deletions(-)

diff --git a/crates/core_simd/src/math.rs b/crates/core_simd/src/math.rs
index 6ee5efdb981c..2bae414ebfb5 100644
--- a/crates/core_simd/src/math.rs
+++ b/crates/core_simd/src/math.rs
@@ -10,7 +10,8 @@ macro_rules! impl_uint_arith {
             /// # Examples
             /// ```
             /// # #![feature(portable_simd)]
-            /// # use core_simd::*;
+            /// # #[cfg(feature = "std")] use core_simd::Simd;
+            /// # #[cfg(not(feature = "std"))] use core::simd::Simd;
             #[doc = concat!("# use core::", stringify!($ty), "::MAX;")]
             /// let x = Simd::from_array([2, 1, 0, MAX]);
             /// let max = Simd::splat(MAX);
@@ -29,7 +30,8 @@ macro_rules! impl_uint_arith {
             /// # Examples
             /// ```
             /// # #![feature(portable_simd)]
-            /// # use core_simd::*;
+            /// # #[cfg(feature = "std")] use core_simd::Simd;
+            /// # #[cfg(not(feature = "std"))] use core::simd::Simd;
             #[doc = concat!("# use core::", stringify!($ty), "::MAX;")]
             /// let x = Simd::from_array([2, 1, 0, MAX]);
             /// let max = Simd::splat(MAX);
@@ -54,7 +56,8 @@ macro_rules! impl_int_arith {
             /// # Examples
             /// ```
             /// # #![feature(portable_simd)]
-            /// # use core_simd::*;
+            /// # #[cfg(feature = "std")] use core_simd::Simd;
+            /// # #[cfg(not(feature = "std"))] use core::simd::Simd;
             #[doc = concat!("# use core::", stringify!($ty), "::{MIN, MAX};")]
             /// let x = Simd::from_array([MIN, 0, 1, MAX]);
             /// let max = Simd::splat(MAX);
@@ -73,7 +76,8 @@ macro_rules! impl_int_arith {
             /// # Examples
             /// ```
             /// # #![feature(portable_simd)]
-            /// # use core_simd::*;
+            /// # #[cfg(feature = "std")] use core_simd::Simd;
+            /// # #[cfg(not(feature = "std"))] use core::simd::Simd;
             #[doc = concat!("# use core::", stringify!($ty), "::{MIN, MAX};")]
             /// let x = Simd::from_array([MIN, -2, -1, MAX]);
             /// let max = Simd::splat(MAX);
@@ -92,7 +96,8 @@ macro_rules! impl_int_arith {
             /// # Examples
             /// ```
             /// # #![feature(portable_simd)]
-            /// # use core_simd::*;
+            /// # #[cfg(feature = "std")] use core_simd::Simd;
+            /// # #[cfg(not(feature = "std"))] use core::simd::Simd;
             #[doc = concat!("# use core::", stringify!($ty), "::{MIN, MAX};")]
             /// let xs = Simd::from_array([MIN, MIN +1, -5, 0]);
             /// assert_eq!(xs.abs(), Simd::from_array([MIN, MAX, 5, 0]));
@@ -110,7 +115,8 @@ macro_rules! impl_int_arith {
             /// # Examples
             /// ```
             /// # #![feature(portable_simd)]
-            /// # use core_simd::*;
+            /// # #[cfg(feature = "std")] use core_simd::Simd;
+            /// # #[cfg(not(feature = "std"))] use core::simd::Simd;
             #[doc = concat!("# use core::", stringify!($ty), "::{MIN, MAX};")]
             /// let xs = Simd::from_array([MIN, -2, 0, 3]);
             /// let unsat = xs.abs();
@@ -132,7 +138,8 @@ macro_rules! impl_int_arith {
             /// # Examples
             /// ```
             /// # #![feature(portable_simd)]
-            /// # use core_simd::*;
+            /// # #[cfg(feature = "std")] use core_simd::Simd;
+            /// # #[cfg(not(feature = "std"))] use core::simd::Simd;
             #[doc = concat!("# use core::", stringify!($ty), "::{MIN, MAX};")]
             /// let x = Simd::from_array([MIN, -2, 3, MAX]);
             /// let unsat = -x;
diff --git a/crates/core_simd/src/permute.rs b/crates/core_simd/src/permute.rs
index 206519340b3a..3e31c3365e88 100644
--- a/crates/core_simd/src/permute.rs
+++ b/crates/core_simd/src/permute.rs
@@ -18,7 +18,8 @@ macro_rules! impl_shuffle_lane {
             ///
             /// ```
             /// #![feature(portable_simd)]
-            /// # use core_simd::Simd;
+            /// # #[cfg(feature = "std")] use core_simd::Simd;
+            /// # #[cfg(not(feature = "std"))] use core::simd::Simd;
             /// let a = Simd::from_array([1.0, 2.0, 3.0, 4.0]);
             /// let b = Simd::from_array([5.0, 6.0, 7.0, 8.0]);
             /// const IDXS: [u32; 4] = [4,0,3,7];
@@ -59,7 +60,8 @@ macro_rules! impl_shuffle_lane {
             ///
             /// ```
             /// #![feature(portable_simd)]
-            /// # use core_simd::Simd;
+            /// # #[cfg(feature = "std")] use core_simd::Simd;
+            /// # #[cfg(not(feature = "std"))] use core::simd::Simd;
             /// let a = Simd::from_array([0, 1, 2, 3]);
             /// let b = Simd::from_array([4, 5, 6, 7]);
             /// let (x, y) = a.interleave(b);
@@ -111,7 +113,8 @@ macro_rules! impl_shuffle_lane {
             ///
             /// ```
             /// #![feature(portable_simd)]
-            /// # use core_simd::Simd;
+            /// # #[cfg(feature = "std")] use core_simd::Simd;
+            /// # #[cfg(not(feature = "std"))] use core::simd::Simd;
             /// let a = Simd::from_array([0, 4, 1, 5]);
             /// let b = Simd::from_array([2, 6, 3, 7]);
             /// let (x, y) = a.deinterleave(b);
diff --git a/crates/core_simd/src/select.rs b/crates/core_simd/src/select.rs
index 596621c5676b..c3d69a83088b 100644
--- a/crates/core_simd/src/select.rs
+++ b/crates/core_simd/src/select.rs
@@ -61,7 +61,8 @@ where
     ///
     /// ```
     /// # #![feature(portable_simd)]
-    /// # use core_simd::{Mask, Simd};
+    /// # #[cfg(feature = "std")] use core_simd::{Simd, Mask};
+    /// # #[cfg(not(feature = "std"))] use core::simd::{Simd, Mask};
     /// let a = Simd::from_array([0, 1, 2, 3]);
     /// let b = Simd::from_array([4, 5, 6, 7]);
     /// let mask = Mask::from_array([true, false, false, true]);
@@ -72,7 +73,8 @@ where
     /// `select` can also be used on masks:
     /// ```
     /// # #![feature(portable_simd)]
-    /// # use core_simd::Mask;
+    /// # #[cfg(feature = "std")] use core_simd::Mask;
+    /// # #[cfg(not(feature = "std"))] use core::simd::Mask;
     /// let a = Mask::<i32, 4>::from_array([true, true, false, false]);
     /// let b = Mask::<i32, 4>::from_array([false, false, true, true]);
     /// let mask = Mask::<i32, 4>::from_array([true, false, false, true]);
diff --git a/crates/core_simd/src/vector.rs b/crates/core_simd/src/vector.rs
index fb3518fa13d7..eee53385c15e 100644
--- a/crates/core_simd/src/vector.rs
+++ b/crates/core_simd/src/vector.rs
@@ -53,7 +53,8 @@ where
     /// If an index is out of bounds, that lane instead selects the value from the "or" vector.
     /// ```
     /// # #![feature(portable_simd)]
-    /// # use core_simd::*;
+    /// # #[cfg(feature = "std")] use core_simd::Simd;
+    /// # #[cfg(not(feature = "std"))] use core::simd::Simd;
     /// let vec: Vec<i32> = vec![10, 11, 12, 13, 14, 15, 16, 17, 18];
     /// let idxs = Simd::from_array([9, 3, 0, 5]);
     /// let alt = Simd::from_array([-5, -4, -3, -2]);
@@ -71,7 +72,8 @@ where
     /// Out-of-bounds indices instead use the default value for that lane (0).
     /// ```
     /// # #![feature(portable_simd)]
-    /// # use core_simd::*;
+    /// # #[cfg(feature = "std")] use core_simd::Simd;
+    /// # #[cfg(not(feature = "std"))] use core::simd::Simd;
     /// let vec: Vec<i32> = vec![10, 11, 12, 13, 14, 15, 16, 17, 18];
     /// let idxs = Simd::from_array([9, 3, 0, 5]);
     ///
@@ -91,7 +93,8 @@ where
     /// Out-of-bounds or masked indices instead select the value from the "or" vector.
     /// ```
     /// # #![feature(portable_simd)]
-    /// # use core_simd::*;
+    /// # #[cfg(feature = "std")] use core_simd::{Simd, Mask};
+    /// # #[cfg(not(feature = "std"))] use core::simd::{Simd, Mask};
     /// let vec: Vec<i32> = vec![10, 11, 12, 13, 14, 15, 16, 17, 18];
     /// let idxs = Simd::from_array([9, 3, 0, 5]);
     /// let alt = Simd::from_array([-5, -4, -3, -2]);
@@ -121,7 +124,8 @@ where
     /// `scatter` writes "in order", so if an index receives two writes, only the last is guaranteed.
     /// ```
     /// # #![feature(portable_simd)]
-    /// # use core_simd::*;
+    /// # #[cfg(feature = "std")] use core_simd::Simd;
+    /// # #[cfg(not(feature = "std"))] use core::simd::Simd;
     /// let mut vec: Vec<i32> = vec![10, 11, 12, 13, 14, 15, 16, 17, 18];
     /// let idxs = Simd::from_array([9, 3, 0, 0]);
     /// let vals = Simd::from_array([-27, 82, -41, 124]);
@@ -139,7 +143,8 @@ where
     /// `scatter_select` writes "in order", so if an index receives two writes, only the last is guaranteed.
     /// ```
     /// # #![feature(portable_simd)]
-    /// # use core_simd::*;
+    /// # #[cfg(feature = "std")] use core_simd::{Simd, Mask};
+    /// # #[cfg(not(feature = "std"))] use core::simd::{Simd, Mask};
     /// let mut vec: Vec<i32> = vec![10, 11, 12, 13, 14, 15, 16, 17, 18];
     /// let idxs = Simd::from_array([9, 3, 0, 0]);
     /// let vals = Simd::from_array([-27, 82, -41, 124]);

From c2f59483f96cf1ab1e92cf10e0f9094432a8374c Mon Sep 17 00:00:00 2001
From: Jubilee Young <workingjubilee@gmail.com>
Date: Tue, 21 Sep 2021 18:55:05 -0700
Subject: [PATCH 224/249] Feature-flag fused mul-add to block libcalls

---
 crates/core_simd/src/intrinsics.rs   | 13 +++++------
 crates/core_simd/src/vector/float.rs |  1 +
 crates/core_simd/tests/ops_macros.rs | 33 +++++++++++++++-------------
 3 files changed, 25 insertions(+), 22 deletions(-)

diff --git a/crates/core_simd/src/intrinsics.rs b/crates/core_simd/src/intrinsics.rs
index 3ed9845d608e..5783950f353e 100644
--- a/crates/core_simd/src/intrinsics.rs
+++ b/crates/core_simd/src/intrinsics.rs
@@ -46,13 +46,6 @@ extern "platform-intrinsic" {
     /// fabs
     pub(crate) fn simd_fabs<T>(x: T) -> T;
 
-    /// fsqrt
-    #[cfg(feature = "std")]
-    pub(crate) fn simd_fsqrt<T>(x: T) -> T;
-
-    /// fma
-    pub(crate) fn simd_fma<T>(x: T, y: T, z: T) -> T;
-
     pub(crate) fn simd_eq<T, U>(x: T, y: T) -> U;
     pub(crate) fn simd_ne<T, U>(x: T, y: T) -> U;
     pub(crate) fn simd_lt<T, U>(x: T, y: T) -> U;
@@ -110,6 +103,12 @@ mod std {
 
         // trunc
         pub(crate) fn simd_trunc<T>(x: T) -> T;
+
+        // fsqrt
+        pub(crate) fn simd_fsqrt<T>(x: T) -> T;
+
+        // fma
+        pub(crate) fn simd_fma<T>(x: T, y: T, z: T) -> T;
     }
 }
 
diff --git a/crates/core_simd/src/vector/float.rs b/crates/core_simd/src/vector/float.rs
index 21a6c43e153a..c09d0ac84d24 100644
--- a/crates/core_simd/src/vector/float.rs
+++ b/crates/core_simd/src/vector/float.rs
@@ -42,6 +42,7 @@ macro_rules! impl_float_vector {
             /// architecture has a dedicated `fma` CPU instruction.  However, this is not always
             /// true, and will be heavily dependent on designing algorithms with specific target
             /// hardware in mind.
+            #[cfg(feature = "std")]
             #[inline]
             pub fn mul_add(self, a: Self, b: Self) -> Self {
                 unsafe { intrinsics::simd_fma(self, a, b) }
diff --git a/crates/core_simd/tests/ops_macros.rs b/crates/core_simd/tests/ops_macros.rs
index 0c45ea2367c9..31b7ee206959 100644
--- a/crates/core_simd/tests/ops_macros.rs
+++ b/crates/core_simd/tests/ops_macros.rs
@@ -437,14 +437,6 @@ macro_rules! impl_float_tests {
                     )
                 }
 
-                fn mul_add<const LANES: usize>() {
-                    test_helpers::test_ternary_elementwise(
-                        &Vector::<LANES>::mul_add,
-                        &Scalar::mul_add,
-                        &|_, _, _| true,
-                    )
-                }
-
                 fn recip<const LANES: usize>() {
                     test_helpers::test_unary_elementwise(
                         &Vector::<LANES>::recip,
@@ -601,13 +593,24 @@ macro_rules! impl_float_tests {
             }
 
             #[cfg(feature = "std")]
-            test_helpers::test_lanes! {
-                fn sqrt<const LANES: usize>() {
-                    test_helpers::test_unary_elementwise(
-                        &Vector::<LANES>::sqrt,
-                        &Scalar::sqrt,
-                        &|_| true,
-                    )
+            mod std {
+                use super::*;
+                test_helpers::test_lanes! {
+                    fn sqrt<const LANES: usize>() {
+                        test_helpers::test_unary_elementwise(
+                            &Vector::<LANES>::sqrt,
+                            &Scalar::sqrt,
+                            &|_| true,
+                        )
+                    }
+
+                    fn mul_add<const LANES: usize>() {
+                        test_helpers::test_ternary_elementwise(
+                            &Vector::<LANES>::mul_add,
+                            &Scalar::mul_add,
+                            &|_, _, _| true,
+                        )
+                    }
                 }
             }
         }

From afd7c5a5eec9d885388044dcbcd5880d20eddfcb Mon Sep 17 00:00:00 2001
From: Jubilee Young <workingjubilee@gmail.com>
Date: Mon, 27 Sep 2021 14:44:31 -0700
Subject: [PATCH 225/249] Make sure MaskElement is in bitmasks.rs

---
 crates/core_simd/src/masks/bitmask.rs | 1 +
 1 file changed, 1 insertion(+)

diff --git a/crates/core_simd/src/masks/bitmask.rs b/crates/core_simd/src/masks/bitmask.rs
index 0691c6ecd218..04fb86595d6f 100644
--- a/crates/core_simd/src/masks/bitmask.rs
+++ b/crates/core_simd/src/masks/bitmask.rs
@@ -1,3 +1,4 @@
+use super::MaskElement;
 use crate::simd::intrinsics;
 use crate::simd::{LaneCount, Simd, SupportedLaneCount};
 use core::marker::PhantomData;

From 4fbccafc66fcec3f36d40e4993fc4567c7a89c29 Mon Sep 17 00:00:00 2001
From: Magnus Ulimoen <magnus@ulimoen.dev>
Date: Wed, 29 Sep 2021 16:30:42 +0000
Subject: [PATCH 226/249] Add lanes()

---
 crates/core_simd/src/vector.rs | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/crates/core_simd/src/vector.rs b/crates/core_simd/src/vector.rs
index eee53385c15e..4ccdb9c5a5da 100644
--- a/crates/core_simd/src/vector.rs
+++ b/crates/core_simd/src/vector.rs
@@ -24,6 +24,11 @@ where
     LaneCount<LANES>: SupportedLaneCount,
     T: SimdElement,
 {
+    /// Get the number of lanes in this vector.
+    pub const fn lanes(&self) -> usize {
+        LANES
+    }
+
     /// Construct a SIMD vector by setting all lanes to the given value.
     pub const fn splat(value: T) -> Self {
         Self([value; LANES])

From ec05dfbbf9f77f706feb4858aa6b38e4d84bf12f Mon Sep 17 00:00:00 2001
From: Magnus Ulimoen <magnus@ulimoen.dev>
Date: Wed, 29 Sep 2021 17:01:27 +0000
Subject: [PATCH 227/249] Add associated LANES const

---
 crates/core_simd/src/vector.rs | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/crates/core_simd/src/vector.rs b/crates/core_simd/src/vector.rs
index 4ccdb9c5a5da..9cefe3842641 100644
--- a/crates/core_simd/src/vector.rs
+++ b/crates/core_simd/src/vector.rs
@@ -24,6 +24,9 @@ where
     LaneCount<LANES>: SupportedLaneCount,
     T: SimdElement,
 {
+    /// Number of lanes in this vector.
+    pub const LANES: usize = LANES;
+
     /// Get the number of lanes in this vector.
     pub const fn lanes(&self) -> usize {
         LANES

From b506e3e28e7a476119cead381e54094d6fe0257e Mon Sep 17 00:00:00 2001
From: Jubilee Young <workingjubilee@gmail.com>
Date: Wed, 29 Sep 2021 13:07:27 -0700
Subject: [PATCH 228/249] Renovate for Edition 2021

In a still-future edition, `unsafe_op_in_unsafe_fn` may error.
Let's get ahead of that.
---
 crates/core_simd/Cargo.toml           |  2 +-
 crates/core_simd/src/lib.rs           |  1 +
 crates/core_simd/src/masks.rs         |  8 +++++---
 crates/core_simd/src/masks/bitmask.rs | 12 ++++++++----
 crates/core_simd/src/round.rs         |  2 +-
 crates/test_helpers/Cargo.toml        |  2 +-
 6 files changed, 17 insertions(+), 10 deletions(-)

diff --git a/crates/core_simd/Cargo.toml b/crates/core_simd/Cargo.toml
index 9e8d742d83c7..a103ef115a58 100644
--- a/crates/core_simd/Cargo.toml
+++ b/crates/core_simd/Cargo.toml
@@ -1,7 +1,7 @@
 [package]
 name = "core_simd"
 version = "0.1.0"
-edition = "2018"
+edition = "2021"
 homepage = "https://github.com/rust-lang/portable-simd"
 repository = "https://github.com/rust-lang/portable-simd"
 keywords = ["core", "simd", "intrinsics"]
diff --git a/crates/core_simd/src/lib.rs b/crates/core_simd/src/lib.rs
index 486905e4d811..2d5949f8e791 100644
--- a/crates/core_simd/src/lib.rs
+++ b/crates/core_simd/src/lib.rs
@@ -11,6 +11,7 @@
 )]
 #![cfg_attr(feature = "generic_const_exprs", feature(generic_const_exprs))]
 #![warn(missing_docs)]
+#![deny(unsafe_op_in_unsafe_fn)]
 #![unstable(feature = "portable_simd", issue = "86656")]
 //! Portable SIMD module.
 
diff --git a/crates/core_simd/src/masks.rs b/crates/core_simd/src/masks.rs
index c4d6e1883483..1b1677330fb6 100644
--- a/crates/core_simd/src/masks.rs
+++ b/crates/core_simd/src/masks.rs
@@ -118,7 +118,7 @@ where
     /// All lanes must be either 0 or -1.
     #[inline]
     pub unsafe fn from_int_unchecked(value: Simd<T, LANES>) -> Self {
-        Self(mask_impl::Mask::from_int_unchecked(value))
+        unsafe { Self(mask_impl::Mask::from_int_unchecked(value)) }
     }
 
     /// Converts a vector of integers to a mask, where 0 represents `false` and -1
@@ -145,7 +145,7 @@ where
     /// `lane` must be less than `LANES`.
     #[inline]
     pub unsafe fn test_unchecked(&self, lane: usize) -> bool {
-        self.0.test_unchecked(lane)
+        unsafe { self.0.test_unchecked(lane) }
     }
 
     /// Tests the value of the specified lane.
@@ -164,7 +164,9 @@ where
     /// `lane` must be less than `LANES`.
     #[inline]
     pub unsafe fn set_unchecked(&mut self, lane: usize, value: bool) {
-        self.0.set_unchecked(lane, value);
+        unsafe {
+            self.0.set_unchecked(lane, value);
+        }
     }
 
     /// Sets the value of the specified lane.
diff --git a/crates/core_simd/src/masks/bitmask.rs b/crates/core_simd/src/masks/bitmask.rs
index 04fb86595d6f..2689e1a88a8c 100644
--- a/crates/core_simd/src/masks/bitmask.rs
+++ b/crates/core_simd/src/masks/bitmask.rs
@@ -93,7 +93,9 @@ where
 
     #[inline]
     pub unsafe fn set_unchecked(&mut self, lane: usize, value: bool) {
-        self.0.as_mut()[lane / 8] ^= ((value ^ self.test_unchecked(lane)) as u8) << (lane % 8)
+        unsafe {
+            self.0.as_mut()[lane / 8] ^= ((value ^ self.test_unchecked(lane)) as u8) << (lane % 8)
+        }
     }
 
     #[inline]
@@ -112,9 +114,11 @@ where
             core::mem::size_of::<<LaneCount::<LANES> as SupportedLaneCount>::BitMask>(),
             core::mem::size_of::<<LaneCount::<LANES> as SupportedLaneCount>::IntBitMask>(),
         );
-        let mask: <LaneCount<LANES> as SupportedLaneCount>::IntBitMask =
-            intrinsics::simd_bitmask(value);
-        Self(core::mem::transmute_copy(&mask), PhantomData)
+        unsafe {
+            let mask: <LaneCount<LANES> as SupportedLaneCount>::IntBitMask =
+                intrinsics::simd_bitmask(value);
+            Self(core::mem::transmute_copy(&mask), PhantomData)
+        }
     }
 
     #[cfg(feature = "generic_const_exprs")]
diff --git a/crates/core_simd/src/round.rs b/crates/core_simd/src/round.rs
index 3bb10d0ed0bf..09789e114920 100644
--- a/crates/core_simd/src/round.rs
+++ b/crates/core_simd/src/round.rs
@@ -61,7 +61,7 @@ macro_rules! implement {
             /// * Be representable in the return type, after truncating off its fractional part
             #[inline]
             pub unsafe fn to_int_unchecked(self) -> Simd<$int_type, LANES> {
-                intrinsics::simd_cast(self)
+                unsafe { intrinsics::simd_cast(self) }
             }
 
             /// Creates a floating-point vector from an integer vector.  Rounds values that are
diff --git a/crates/test_helpers/Cargo.toml b/crates/test_helpers/Cargo.toml
index e38b223d6c92..a04b0961d7f7 100644
--- a/crates/test_helpers/Cargo.toml
+++ b/crates/test_helpers/Cargo.toml
@@ -1,7 +1,7 @@
 [package]
 name = "test_helpers"
 version = "0.1.0"
-edition = "2018"
+edition = "2021"
 publish = false
 
 [dependencies.proptest]

From 6d236626891f206d5974e2fac03e3fa816279b74 Mon Sep 17 00:00:00 2001
From: Jubilee Young <workingjubilee@gmail.com>
Date: Fri, 24 Sep 2021 12:46:50 -0700
Subject: [PATCH 229/249] Add {gather,scatter}_select_unchecked

This unsafe variant allows the thinnest API, in case LLVM cannot
perform loop-invariant code motion on a hot loop when the safe
form is used.

An unchecked variant could be added to other forms, but doesn't
seem likely to improve anything, since it would just add heavier
codegen.
---
 crates/core_simd/src/vector.rs | 72 +++++++++++++++++++++++++++++++---
 1 file changed, 66 insertions(+), 6 deletions(-)

diff --git a/crates/core_simd/src/vector.rs b/crates/core_simd/src/vector.rs
index 9cefe3842641..82b440896f0d 100644
--- a/crates/core_simd/src/vector.rs
+++ b/crates/core_simd/src/vector.rs
@@ -119,12 +119,42 @@ where
         idxs: Simd<usize, LANES>,
         or: Self,
     ) -> Self {
-        let mask = (mask & idxs.lanes_lt(Simd::splat(slice.len()))).to_int();
+        let mask: Mask<isize, LANES> = mask & idxs.lanes_lt(Simd::splat(slice.len()));
+        // SAFETY: We have masked-off out-of-bounds lanes.
+        unsafe { Self::gather_select_unchecked(slice, mask, idxs, or) }
+    }
+
+    /// Unsafe SIMD gather: construct a SIMD vector by reading from a slice, using potentially discontiguous indices.
+    /// Masked indices instead select the value from the "or" vector.
+    /// `gather_select_unchecked` is unsound if any unmasked index is out-of-bounds of the slice.
+    /// ```
+    /// # #![feature(portable_simd)]
+    /// # #[cfg(feature = "std")] use core_simd::{Simd, Mask};
+    /// # #[cfg(not(feature = "std"))] use core::simd::{Simd, Mask};
+    /// let vec: Vec<i32> = vec![10, 11, 12, 13, 14, 15, 16, 17, 18];
+    /// let idxs = Simd::from_array([9, 3, 0, 5]);
+    /// let alt = Simd::from_array([-5, -4, -3, -2]);
+    /// let mask = Mask::from_array([true, true, true, false]); // Note the final mask lane.
+    /// // If this mask was used to gather, it would be unsound. Let's fix that.
+    /// let mask = mask & idxs.lanes_lt(Simd::splat(vec.len()));
+    ///
+    /// // We have masked the OOB lane, so it's safe to gather now.
+    /// let result = unsafe { Simd::gather_select_unchecked(&vec, mask, idxs, alt) };
+    /// assert_eq!(result, Simd::from_array([-5, 13, 10, -2]));
+    /// ```
+    #[must_use]
+    #[inline]
+    pub unsafe fn gather_select_unchecked(
+        slice: &[T],
+        mask: Mask<isize, LANES>,
+        idxs: Simd<usize, LANES>,
+        or: Self,
+    ) -> Self {
         let base_ptr = crate::simd::ptr::SimdConstPtr::splat(slice.as_ptr());
         // Ferris forgive me, I have done pointer arithmetic here.
         let ptrs = base_ptr.wrapping_add(idxs);
         // SAFETY: The ptrs have been bounds-masked to prevent memory-unsafe reads insha'allah
-        unsafe { intrinsics::simd_gather(or, ptrs, mask) }
+        unsafe { intrinsics::simd_gather(or, ptrs, mask.to_int()) }
     }
 
     /// SIMD scatter: write a SIMD vector's values into a slice, using potentially discontiguous indices.
@@ -168,12 +198,42 @@ where
         mask: Mask<isize, LANES>,
         idxs: Simd<usize, LANES>,
     ) {
-        // We must construct our scatter mask before we derive a pointer!
-        let mask = (mask & idxs.lanes_lt(Simd::splat(slice.len()))).to_int();
+        let mask: Mask<isize, LANES> = mask & idxs.lanes_lt(Simd::splat(slice.len()));
+        // SAFETY: We have masked-off out-of-bounds lanes.
+        unsafe { self.scatter_select_unchecked(slice, mask, idxs) }
+    }
+
+    /// Unsafe SIMD scatter: write a SIMD vector's values into a slice, using potentially discontiguous indices.
+    /// Out-of-bounds or masked indices are not written.
+    /// `scatter_select_unchecked` is unsound if any unmasked index is out of bounds of the slice.
+    /// `scatter_select_unchecked` writes "in order", so if the same index receives two writes, only the last is guaranteed.
+    /// ```
+    /// # #![feature(portable_simd)]
+    /// # #[cfg(feature = "std")] use core_simd::{Simd, Mask};
+    /// # #[cfg(not(feature = "std"))] use core::simd::{Simd, Mask};
+    /// let mut vec: Vec<i32> = vec![10, 11, 12, 13, 14, 15, 16, 17, 18];
+    /// let idxs = Simd::from_array([9, 3, 0, 0]);
+    /// let vals = Simd::from_array([-27, 82, -41, 124]);
+    /// let mask = Mask::from_array([true, true, true, false]); // Note the mask of the last lane.
+    /// // If this mask was used to scatter, it would be unsound. Let's fix that.
+    /// let mask = mask & idxs.lanes_lt(Simd::splat(vec.len()));
+    ///
+    /// // We have masked the OOB lane, so it's safe to gather now.
+    /// unsafe { vals.scatter_select_unchecked(&mut vec, mask, idxs); }
+    /// // index 0's second write is masked, thus was omitted.
+    /// assert_eq!(vec, vec![-41, 11, 12, 82, 14, 15, 16, 17, 18]);
+    /// ```
+    #[inline]
+    pub unsafe fn scatter_select_unchecked(
+        self,
+        slice: &mut [T],
+        mask: Mask<isize, LANES>,
+        idxs: Simd<usize, LANES>,
+    ) {
         // SAFETY: This block works with *mut T derived from &mut 'a [T],
         // which means it is delicate in Rust's borrowing model, circa 2021:
         // &mut 'a [T] asserts uniqueness, so deriving &'a [T] invalidates live *mut Ts!
-        // Even though this block is largely safe methods, it must be almost exactly this way
+        // Even though this block is largely safe methods, it must be exactly this way
         // to prevent invalidating the raw ptrs while they're live.
         // Thus, entering this block requires all values to use being already ready:
         // 0. idxs we want to write to, which are used to construct the mask.
@@ -186,7 +246,7 @@ where
             // Ferris forgive me, I have done pointer arithmetic here.
             let ptrs = base_ptr.wrapping_add(idxs);
             // The ptrs have been bounds-masked to prevent memory-unsafe writes insha'allah
-            intrinsics::simd_scatter(self, ptrs, mask)
+            intrinsics::simd_scatter(self, ptrs, mask.to_int())
             // Cleared ☢️ *mut T Zone
         }
     }

From 01e9816ace1b30935a076395e23a239bd08ea1a4 Mon Sep 17 00:00:00 2001
From: Jubilee <46493976+workingjubilee@users.noreply.github.com>
Date: Tue, 28 Sep 2021 12:47:44 -0700
Subject: [PATCH 230/249] docs: fix typo gather -> scatter

Co-authored-by: Jacob Lifshay <programmerjake@gmail.com>
---
 crates/core_simd/src/vector.rs | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/crates/core_simd/src/vector.rs b/crates/core_simd/src/vector.rs
index 82b440896f0d..695fed430b7c 100644
--- a/crates/core_simd/src/vector.rs
+++ b/crates/core_simd/src/vector.rs
@@ -218,7 +218,7 @@ where
     /// // If this mask was used to scatter, it would be unsound. Let's fix that.
     /// let mask = mask & idxs.lanes_lt(Simd::splat(vec.len()));
     ///
-    /// // We have masked the OOB lane, so it's safe to gather now.
+    /// // We have masked the OOB lane, so it's safe to scatter now.
     /// unsafe { vals.scatter_select_unchecked(&mut vec, mask, idxs); }
     /// // index 0's second write is masked, thus was omitted.
     /// assert_eq!(vec, vec![-41, 11, 12, 82, 14, 15, 16, 17, 18]);

From 9be26656d2647e06da4a1a1c5615f93322607a5d Mon Sep 17 00:00:00 2001
From: Jubilee Young <workingjubilee@gmail.com>
Date: Tue, 28 Sep 2021 22:43:28 -0700
Subject: [PATCH 231/249] Rewrite gather/scatter docs

Headings with # Safety and # Examples are more "std style".
Use terms like "enable" and "disable", rather than "mask" jargon.
---
 crates/core_simd/src/vector.rs | 109 +++++++++++++++++++++------------
 1 file changed, 69 insertions(+), 40 deletions(-)

diff --git a/crates/core_simd/src/vector.rs b/crates/core_simd/src/vector.rs
index 695fed430b7c..a043ef2074e8 100644
--- a/crates/core_simd/src/vector.rs
+++ b/crates/core_simd/src/vector.rs
@@ -57,8 +57,10 @@ where
         self.0
     }
 
-    /// SIMD gather: construct a SIMD vector by reading from a slice, using potentially discontiguous indices.
-    /// If an index is out of bounds, that lane instead selects the value from the "or" vector.
+    /// Reads from potentially discontiguous indices in `slice` to construct a SIMD vector.
+    /// Lanes given an out-of-bounds index instead select values from the `or` vector.
+    ///
+    /// # Examples
     /// ```
     /// # #![feature(portable_simd)]
     /// # #[cfg(feature = "std")] use core_simd::Simd;
@@ -76,8 +78,10 @@ where
         Self::gather_select(slice, Mask::splat(true), idxs, or)
     }
 
-    /// SIMD gather: construct a SIMD vector by reading from a slice, using potentially discontiguous indices.
-    /// Out-of-bounds indices instead use the default value for that lane (0).
+    /// Reads from potentially discontiguous indices in `slice` to construct a SIMD vector.
+    /// Lanes given an out-of-bounds index instead are set the default value for the type.
+    ///
+    /// # Examples
     /// ```
     /// # #![feature(portable_simd)]
     /// # #[cfg(feature = "std")] use core_simd::Simd;
@@ -97,8 +101,11 @@ where
         Self::gather_or(slice, idxs, Self::splat(T::default()))
     }
 
-    /// SIMD gather: construct a SIMD vector by reading from a slice, using potentially discontiguous indices.
-    /// Out-of-bounds or masked indices instead select the value from the "or" vector.
+    /// Reads from potentially discontiguous indices in `slice` to construct a SIMD vector.
+    /// The mask `enable`s all `true` lanes and disables all `false` lanes.
+    /// If an index is disabled or is out-of-bounds, the lane is selected from the `or` vector.
+    ///
+    /// # Examples
     /// ```
     /// # #![feature(portable_simd)]
     /// # #[cfg(feature = "std")] use core_simd::{Simd, Mask};
@@ -106,27 +113,34 @@ where
     /// let vec: Vec<i32> = vec![10, 11, 12, 13, 14, 15, 16, 17, 18];
     /// let idxs = Simd::from_array([9, 3, 0, 5]);
     /// let alt = Simd::from_array([-5, -4, -3, -2]);
-    /// let mask = Mask::from_array([true, true, true, false]); // Note the mask of the last lane.
+    /// let enable = Mask::from_array([true, true, true, false]); // Note the mask of the last lane.
     ///
-    /// let result = Simd::gather_select(&vec, mask, idxs, alt); // Note the lane that is out-of-bounds.
+    /// let result = Simd::gather_select(&vec, enable, idxs, alt); // Note the lane that is out-of-bounds.
     /// assert_eq!(result, Simd::from_array([-5, 13, 10, -2]));
     /// ```
     #[must_use]
     #[inline]
     pub fn gather_select(
         slice: &[T],
-        mask: Mask<isize, LANES>,
+        enable: Mask<isize, LANES>,
         idxs: Simd<usize, LANES>,
         or: Self,
     ) -> Self {
-        let mask: Mask<isize, LANES> = mask & idxs.lanes_lt(Simd::splat(slice.len()));
+        let enable: Mask<isize, LANES> = enable & idxs.lanes_lt(Simd::splat(slice.len()));
         // SAFETY: We have masked-off out-of-bounds lanes.
-        unsafe { Self::gather_select_unchecked(slice, mask, idxs, or) }
+        unsafe { Self::gather_select_unchecked(slice, enable, idxs, or) }
     }
 
-    /// Unsafe SIMD gather: construct a SIMD vector by reading from a slice, using potentially discontiguous indices.
-    /// Masked indices instead select the value from the "or" vector.
-    /// `gather_select_unchecked` is unsound if any unmasked index is out-of-bounds of the slice.
+    /// Reads from potentially discontiguous indices in `slice` to construct a SIMD vector.
+    /// The mask `enable`s all `true` lanes and disables all `false` lanes.
+    /// If an index is disabled, the lane is selected from the `or` vector.
+    ///
+    /// # Safety
+    ///
+    /// Calling this function with an `enable`d out-of-bounds index is *[undefined behavior]*
+    /// even if the resulting value is not used.
+    ///
+    /// # Examples
     /// ```
     /// # #![feature(portable_simd)]
     /// # #[cfg(feature = "std")] use core_simd::{Simd, Mask};
@@ -134,19 +148,20 @@ where
     /// let vec: Vec<i32> = vec![10, 11, 12, 13, 14, 15, 16, 17, 18];
     /// let idxs = Simd::from_array([9, 3, 0, 5]);
     /// let alt = Simd::from_array([-5, -4, -3, -2]);
-    /// let mask = Mask::from_array([true, true, true, false]); // Note the final mask lane.
+    /// let enable = Mask::from_array([true, true, true, false]); // Note the final mask lane.
     /// // If this mask was used to gather, it would be unsound. Let's fix that.
-    /// let mask = mask & idxs.lanes_lt(Simd::splat(vec.len()));
+    /// let enable = enable & idxs.lanes_lt(Simd::splat(vec.len()));
     ///
     /// // We have masked the OOB lane, so it's safe to gather now.
-    /// let result = unsafe { Simd::gather_select_unchecked(&vec, mask, idxs, alt) };
+    /// let result = unsafe { Simd::gather_select_unchecked(&vec, enable, idxs, alt) };
     /// assert_eq!(result, Simd::from_array([-5, 13, 10, -2]));
     /// ```
+    /// [undefined behavior]: https://doc.rust-lang.org/reference/behavior-considered-undefined.html
     #[must_use]
     #[inline]
     pub unsafe fn gather_select_unchecked(
         slice: &[T],
-        mask: Mask<isize, LANES>,
+        enable: Mask<isize, LANES>,
         idxs: Simd<usize, LANES>,
         or: Self,
     ) -> Self {
@@ -154,12 +169,14 @@ where
         // Ferris forgive me, I have done pointer arithmetic here.
         let ptrs = base_ptr.wrapping_add(idxs);
         // SAFETY: The ptrs have been bounds-masked to prevent memory-unsafe reads insha'allah
-        unsafe { intrinsics::simd_gather(or, ptrs, mask.to_int()) }
+        unsafe { intrinsics::simd_gather(or, ptrs, enable.to_int()) }
     }
 
-    /// SIMD scatter: write a SIMD vector's values into a slice, using potentially discontiguous indices.
-    /// Out-of-bounds indices are not written.
-    /// `scatter` writes "in order", so if an index receives two writes, only the last is guaranteed.
+    /// Writes the values in a SIMD vector to potentially discontiguous indices in `slice`.
+    /// If two lanes in the scattered vector would write to the same index
+    /// only the last lane is guaranteed to actually be written.
+    ///
+    /// # Examples
     /// ```
     /// # #![feature(portable_simd)]
     /// # #[cfg(feature = "std")] use core_simd::Simd;
@@ -176,9 +193,13 @@ where
         self.scatter_select(slice, Mask::splat(true), idxs)
     }
 
-    /// SIMD scatter: write a SIMD vector's values into a slice, using potentially discontiguous indices.
-    /// Out-of-bounds or masked indices are not written.
-    /// `scatter_select` writes "in order", so if an index receives two writes, only the last is guaranteed.
+    /// Writes the values in a SIMD vector to multiple potentially discontiguous indices in `slice`.
+    /// The mask `enable`s all `true` lanes and disables all `false` lanes.
+    /// If an enabled index is out-of-bounds, the lane is not written.
+    /// If two enabled lanes in the scattered vector would write to the same index,
+    /// only the last lane is guaranteed to actually be written.
+    ///
+    /// # Examples
     /// ```
     /// # #![feature(portable_simd)]
     /// # #[cfg(feature = "std")] use core_simd::{Simd, Mask};
@@ -186,27 +207,34 @@ where
     /// let mut vec: Vec<i32> = vec![10, 11, 12, 13, 14, 15, 16, 17, 18];
     /// let idxs = Simd::from_array([9, 3, 0, 0]);
     /// let vals = Simd::from_array([-27, 82, -41, 124]);
-    /// let mask = Mask::from_array([true, true, true, false]); // Note the mask of the last lane.
+    /// let enable = Mask::from_array([true, true, true, false]); // Note the mask of the last lane.
     ///
-    /// vals.scatter_select(&mut vec, mask, idxs); // index 0's second write is masked, thus omitted.
+    /// vals.scatter_select(&mut vec, enable, idxs); // index 0's second write is masked, thus omitted.
     /// assert_eq!(vec, vec![-41, 11, 12, 82, 14, 15, 16, 17, 18]);
     /// ```
     #[inline]
     pub fn scatter_select(
         self,
         slice: &mut [T],
-        mask: Mask<isize, LANES>,
+        enable: Mask<isize, LANES>,
         idxs: Simd<usize, LANES>,
     ) {
-        let mask: Mask<isize, LANES> = mask & idxs.lanes_lt(Simd::splat(slice.len()));
+        let enable: Mask<isize, LANES> = enable & idxs.lanes_lt(Simd::splat(slice.len()));
         // SAFETY: We have masked-off out-of-bounds lanes.
-        unsafe { self.scatter_select_unchecked(slice, mask, idxs) }
+        unsafe { self.scatter_select_unchecked(slice, enable, idxs) }
     }
 
-    /// Unsafe SIMD scatter: write a SIMD vector's values into a slice, using potentially discontiguous indices.
-    /// Out-of-bounds or masked indices are not written.
-    /// `scatter_select_unchecked` is unsound if any unmasked index is out of bounds of the slice.
-    /// `scatter_select_unchecked` writes "in order", so if the same index receives two writes, only the last is guaranteed.
+    /// Writes the values in a SIMD vector to multiple potentially discontiguous indices in `slice`.
+    /// The mask `enable`s all `true` lanes and disables all `false` lanes.
+    /// If two enabled lanes in the scattered vector would write to the same index,
+    /// only the last lane is guaranteed to actually be written.
+    ///
+    /// # Safety
+    ///
+    /// Calling this function with an enabled out-of-bounds index is *[undefined behavior]*,
+    /// and may lead to memory corruption.
+    ///
+    /// # Examples
     /// ```
     /// # #![feature(portable_simd)]
     /// # #[cfg(feature = "std")] use core_simd::{Simd, Mask};
@@ -214,20 +242,21 @@ where
     /// let mut vec: Vec<i32> = vec![10, 11, 12, 13, 14, 15, 16, 17, 18];
     /// let idxs = Simd::from_array([9, 3, 0, 0]);
     /// let vals = Simd::from_array([-27, 82, -41, 124]);
-    /// let mask = Mask::from_array([true, true, true, false]); // Note the mask of the last lane.
+    /// let enable = Mask::from_array([true, true, true, false]); // Note the mask of the last lane.
     /// // If this mask was used to scatter, it would be unsound. Let's fix that.
-    /// let mask = mask & idxs.lanes_lt(Simd::splat(vec.len()));
+    /// let enable = enable & idxs.lanes_lt(Simd::splat(vec.len()));
     ///
     /// // We have masked the OOB lane, so it's safe to scatter now.
-    /// unsafe { vals.scatter_select_unchecked(&mut vec, mask, idxs); }
+    /// unsafe { vals.scatter_select_unchecked(&mut vec, enable, idxs); }
     /// // index 0's second write is masked, thus was omitted.
     /// assert_eq!(vec, vec![-41, 11, 12, 82, 14, 15, 16, 17, 18]);
     /// ```
+    /// [undefined behavior]: https://doc.rust-lang.org/reference/behavior-considered-undefined.html
     #[inline]
     pub unsafe fn scatter_select_unchecked(
         self,
         slice: &mut [T],
-        mask: Mask<isize, LANES>,
+        enable: Mask<isize, LANES>,
         idxs: Simd<usize, LANES>,
     ) {
         // SAFETY: This block works with *mut T derived from &mut 'a [T],
@@ -237,7 +266,7 @@ where
         // to prevent invalidating the raw ptrs while they're live.
         // Thus, entering this block requires all values to use being already ready:
         // 0. idxs we want to write to, which are used to construct the mask.
-        // 1. mask, which depends on an initial &'a [T] and the idxs.
+        // 1. enable, which depends on an initial &'a [T] and the idxs.
         // 2. actual values to scatter (self).
         // 3. &mut [T] which will become our base ptr.
         unsafe {
@@ -246,7 +275,7 @@ where
             // Ferris forgive me, I have done pointer arithmetic here.
             let ptrs = base_ptr.wrapping_add(idxs);
             // The ptrs have been bounds-masked to prevent memory-unsafe writes insha'allah
-            intrinsics::simd_scatter(self, ptrs, mask.to_int())
+            intrinsics::simd_scatter(self, ptrs, enable.to_int())
             // Cleared ☢️ *mut T Zone
         }
     }

From a16b481a08a3d7560f9c92370f18f6ee8c006c9e Mon Sep 17 00:00:00 2001
From: Jubilee <46493976+workingjubilee@users.noreply.github.com>
Date: Sat, 2 Oct 2021 15:16:33 -0700
Subject: [PATCH 232/249] Simplify language for scatter/gather

Co-authored-by: Caleb Zulawski <caleb.zulawski@gmail.com>
---
 crates/core_simd/src/vector.rs | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/crates/core_simd/src/vector.rs b/crates/core_simd/src/vector.rs
index a043ef2074e8..893eff674ffb 100644
--- a/crates/core_simd/src/vector.rs
+++ b/crates/core_simd/src/vector.rs
@@ -58,7 +58,7 @@ where
     }
 
     /// Reads from potentially discontiguous indices in `slice` to construct a SIMD vector.
-    /// Lanes given an out-of-bounds index instead select values from the `or` vector.
+    /// If an index is out-of-bounds, the lane is instead selected from the `or` vector.
     ///
     /// # Examples
     /// ```
@@ -79,7 +79,7 @@ where
     }
 
     /// Reads from potentially discontiguous indices in `slice` to construct a SIMD vector.
-    /// Lanes given an out-of-bounds index instead are set the default value for the type.
+    /// If an index is out-of-bounds, the lane is set to the default value for the type.
     ///
     /// # Examples
     /// ```

From 10168fb7c4ab6e8f02627331cc472a7a2b83c11b Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Wed, 15 Sep 2021 04:59:03 +0000
Subject: [PATCH 233/249] Add new swizzle API

Expand swizzle API and migrate existing functions. Add rotate_left, rotate_right.

Hide implementation details

Add simd_shuffle macro
---
 crates/core_simd/examples/matrix_inversion.rs | 122 +++---
 crates/core_simd/src/intrinsics.rs            |   6 +-
 crates/core_simd/src/lib.rs                   |   1 +
 crates/core_simd/src/mod.rs                   |   6 +-
 crates/core_simd/src/permute.rs               | 154 --------
 crates/core_simd/src/swizzle.rs               | 364 ++++++++++++++++++
 crates/core_simd/tests/permute.rs             |  37 --
 crates/core_simd/tests/swizzle.rs             |  62 +++
 8 files changed, 491 insertions(+), 261 deletions(-)
 delete mode 100644 crates/core_simd/src/permute.rs
 create mode 100644 crates/core_simd/src/swizzle.rs
 delete mode 100644 crates/core_simd/tests/permute.rs
 create mode 100644 crates/core_simd/tests/swizzle.rs

diff --git a/crates/core_simd/examples/matrix_inversion.rs b/crates/core_simd/examples/matrix_inversion.rs
index 29bdc512d77d..ee8c477b8386 100644
--- a/crates/core_simd/examples/matrix_inversion.rs
+++ b/crates/core_simd/examples/matrix_inversion.rs
@@ -2,6 +2,7 @@
 // Code ported from the `packed_simd` crate
 // Run this code with `cargo test --example matrix_inversion`
 #![feature(array_chunks, portable_simd)]
+use core_simd::Which::*;
 use core_simd::*;
 
 // Gotta define our own 4x4 matrix since Rust doesn't ship multidim arrays yet :^)
@@ -163,86 +164,81 @@ pub fn simd_inv4x4(m: Matrix4x4) -> Option<Matrix4x4> {
     let m_2 = f32x4::from_array(m[2]);
     let m_3 = f32x4::from_array(m[3]);
 
-    // 2 argument shuffle, returns an f32x4
-    // the first f32x4 is indexes 0..=3
-    // the second f32x4 is indexed 4..=7
-    let tmp1 = f32x4::shuffle::<{ [0, 1, 4, 5] }>(m_0, m_1);
-    let row1 = f32x4::shuffle::<{ [0, 1, 4, 5] }>(m_2, m_3);
+    const SHUFFLE01: [Which; 4] = [First(0), First(1), Second(0), Second(1)];
+    const SHUFFLE02: [Which; 4] = [First(0), First(2), Second(0), Second(2)];
+    const SHUFFLE13: [Which; 4] = [First(1), First(3), Second(1), Second(3)];
+    const SHUFFLE23: [Which; 4] = [First(2), First(3), Second(2), Second(3)];
 
-    let row0 = f32x4::shuffle::<{ [0, 2, 4, 6] }>(tmp1, row1);
-    let row1 = f32x4::shuffle::<{ [1, 3, 5, 7] }>(row1, tmp1);
+    let tmp = simd_shuffle!(m_0, m_1, SHUFFLE01);
+    let row1 = simd_shuffle!(m_2, m_3, SHUFFLE01);
 
-    let tmp1 = f32x4::shuffle::<{ [2, 3, 6, 7] }>(m_0, m_1);
-    let row3 = f32x4::shuffle::<{ [2, 3, 6, 7] }>(m_2, m_3);
-    let row2 = f32x4::shuffle::<{ [0, 2, 4, 6] }>(tmp1, row3);
-    let row3 = f32x4::shuffle::<{ [1, 3, 5, 7] }>(row3, tmp1);
+    let row0 = simd_shuffle!(tmp, row1, SHUFFLE02);
+    let row1 = simd_shuffle!(row1, tmp, SHUFFLE13);
 
-    let tmp1 = row2 * row3;
-    // there's no syntax for a 1 arg shuffle yet,
-    // so we just pass the same f32x4 twice
-    let tmp1 = f32x4::shuffle::<{ [1, 0, 3, 2] }>(tmp1, tmp1);
+    let tmp = simd_shuffle!(m_0, m_1, SHUFFLE23);
+    let row3 = simd_shuffle!(m_2, m_3, SHUFFLE23);
+    let row2 = simd_shuffle!(tmp, row3, SHUFFLE02);
+    let row3 = simd_shuffle!(row3, tmp, SHUFFLE13);
 
-    let minor0 = row1 * tmp1;
-    let minor1 = row0 * tmp1;
-    let tmp1 = f32x4::shuffle::<{ [2, 3, 0, 1] }>(tmp1, tmp1);
-    let minor0 = (row1 * tmp1) - minor0;
-    let minor1 = (row0 * tmp1) - minor1;
-    let minor1 = f32x4::shuffle::<{ [2, 3, 0, 1] }>(minor1, minor1);
+    let tmp = (row2 * row3).reverse().rotate_right::<2>();
+    let minor0 = row1 * tmp;
+    let minor1 = row0 * tmp;
+    let tmp = tmp.rotate_right::<2>();
+    let minor0 = (row1 * tmp) - minor0;
+    let minor1 = (row0 * tmp) - minor1;
+    let minor1 = minor1.rotate_right::<2>();
 
-    let tmp1 = row1 * row2;
-    let tmp1 = f32x4::shuffle::<{ [1, 0, 3, 2] }>(tmp1, tmp1);
-    let minor0 = (row3 * tmp1) + minor0;
-    let minor3 = row0 * tmp1;
-    let tmp1 = f32x4::shuffle::<{ [2, 3, 0, 1] }>(tmp1, tmp1);
+    let tmp = (row1 * row2).reverse().rotate_right::<2>();
+    let minor0 = (row3 * tmp) + minor0;
+    let minor3 = row0 * tmp;
+    let tmp = tmp.rotate_right::<2>();
 
-    let minor0 = minor0 - row3 * tmp1;
-    let minor3 = row0 * tmp1 - minor3;
-    let minor3 = f32x4::shuffle::<{ [2, 3, 0, 1] }>(minor3, minor3);
+    let minor0 = minor0 - row3 * tmp;
+    let minor3 = row0 * tmp - minor3;
+    let minor3 = minor3.rotate_right::<2>();
 
-    let tmp1 = row3 * f32x4::shuffle::<{ [2, 3, 0, 1] }>(row1, row1);
-    let tmp1 = f32x4::shuffle::<{ [1, 0, 3, 2] }>(tmp1, tmp1);
-    let row2 = f32x4::shuffle::<{ [2, 3, 0, 1] }>(row2, row2);
-    let minor0 = row2 * tmp1 + minor0;
-    let minor2 = row0 * tmp1;
-    let tmp1 = f32x4::shuffle::<{ [2, 3, 0, 1] }>(tmp1, tmp1);
-    let minor0 = minor0 - row2 * tmp1;
-    let minor2 = row0 * tmp1 - minor2;
-    let minor2 = f32x4::shuffle::<{ [2, 3, 0, 1] }>(minor2, minor2);
+    let tmp = (row3 * row1.rotate_right::<2>())
+        .reverse()
+        .rotate_right::<2>();
+    let row2 = row2.rotate_right::<2>();
+    let minor0 = row2 * tmp + minor0;
+    let minor2 = row0 * tmp;
+    let tmp = tmp.rotate_right::<2>();
+    let minor0 = minor0 - row2 * tmp;
+    let minor2 = row0 * tmp - minor2;
+    let minor2 = minor2.rotate_right::<2>();
 
-    let tmp1 = row0 * row1;
-    let tmp1 = f32x4::shuffle::<{ [1, 0, 3, 2] }>(tmp1, tmp1);
-    let minor2 = minor2 + row3 * tmp1;
-    let minor3 = row2 * tmp1 - minor3;
-    let tmp1 = f32x4::shuffle::<{ [2, 3, 0, 1] }>(tmp1, tmp1);
-    let minor2 = row3 * tmp1 - minor2;
-    let minor3 = minor3 - row2 * tmp1;
+    let tmp = (row0 * row1).reverse().rotate_right::<2>();
+    let minor2 = minor2 + row3 * tmp;
+    let minor3 = row2 * tmp - minor3;
+    let tmp = tmp.rotate_right::<2>();
+    let minor2 = row3 * tmp - minor2;
+    let minor3 = minor3 - row2 * tmp;
 
-    let tmp1 = row0 * row3;
-    let tmp1 = f32x4::shuffle::<{ [1, 0, 3, 2] }>(tmp1, tmp1);
-    let minor1 = minor1 - row2 * tmp1;
-    let minor2 = row1 * tmp1 + minor2;
-    let tmp1 = f32x4::shuffle::<{ [2, 3, 0, 1] }>(tmp1, tmp1);
-    let minor1 = row2 * tmp1 + minor1;
-    let minor2 = minor2 - row1 * tmp1;
+    let tmp = (row0 * row3).reverse().rotate_right::<2>();
+    let minor1 = minor1 - row2 * tmp;
+    let minor2 = row1 * tmp + minor2;
+    let tmp = tmp.rotate_right::<2>();
+    let minor1 = row2 * tmp + minor1;
+    let minor2 = minor2 - row1 * tmp;
 
-    let tmp1 = row0 * row2;
-    let tmp1 = f32x4::shuffle::<{ [1, 0, 3, 2] }>(tmp1, tmp1);
-    let minor1 = row3 * tmp1 + minor1;
-    let minor3 = minor3 - row1 * tmp1;
-    let tmp1 = f32x4::shuffle::<{ [2, 3, 0, 1] }>(tmp1, tmp1);
-    let minor1 = minor1 - row3 * tmp1;
-    let minor3 = row1 * tmp1 + minor3;
+    let tmp = (row0 * row2).reverse().rotate_right::<2>();
+    let minor1 = row3 * tmp + minor1;
+    let minor3 = minor3 - row1 * tmp;
+    let tmp = tmp.rotate_right::<2>();
+    let minor1 = minor1 - row3 * tmp;
+    let minor3 = row1 * tmp + minor3;
 
     let det = row0 * minor0;
-    let det = f32x4::shuffle::<{ [2, 3, 0, 1] }>(det, det) + det;
-    let det = f32x4::shuffle::<{ [1, 0, 3, 2] }>(det, det) + det;
+    let det = det.rotate_right::<2>() + det;
+    let det = det.reverse().rotate_right::<2>() + det;
 
     if det.horizontal_sum() == 0. {
         return None;
     }
     // calculate the reciprocal
-    let tmp1 = f32x4::splat(1.0) / det;
-    let det = tmp1 + tmp1 - det * tmp1 * tmp1;
+    let tmp = f32x4::splat(1.0) / det;
+    let det = tmp + tmp - det * tmp * tmp;
 
     let res0 = minor0 * det;
     let res1 = minor1 * det;
diff --git a/crates/core_simd/src/intrinsics.rs b/crates/core_simd/src/intrinsics.rs
index 5783950f353e..5f55cdf0399e 100644
--- a/crates/core_simd/src/intrinsics.rs
+++ b/crates/core_simd/src/intrinsics.rs
@@ -54,11 +54,7 @@ extern "platform-intrinsic" {
     pub(crate) fn simd_ge<T, U>(x: T, y: T) -> U;
 
     // shufflevector
-    pub(crate) fn simd_shuffle2<T, U>(x: T, y: T, idx: [u32; 2]) -> U;
-    pub(crate) fn simd_shuffle4<T, U>(x: T, y: T, idx: [u32; 4]) -> U;
-    pub(crate) fn simd_shuffle8<T, U>(x: T, y: T, idx: [u32; 8]) -> U;
-    pub(crate) fn simd_shuffle16<T, U>(x: T, y: T, idx: [u32; 16]) -> U;
-    pub(crate) fn simd_shuffle32<T, U>(x: T, y: T, idx: [u32; 32]) -> U;
+    pub(crate) fn simd_shuffle<T, U, V>(x: T, y: T, idx: U) -> V;
 
     pub(crate) fn simd_gather<T, U, V>(val: T, ptr: U, mask: V) -> T;
     pub(crate) fn simd_scatter<T, U, V>(val: T, ptr: U, mask: V);
diff --git a/crates/core_simd/src/lib.rs b/crates/core_simd/src/lib.rs
index 2d5949f8e791..55b8be97e0ee 100644
--- a/crates/core_simd/src/lib.rs
+++ b/crates/core_simd/src/lib.rs
@@ -3,6 +3,7 @@
 #![feature(
     adt_const_params,
     const_fn_trait_bound,
+    const_panic,
     platform_intrinsics,
     repr_simd,
     simd_ffi,
diff --git a/crates/core_simd/src/mod.rs b/crates/core_simd/src/mod.rs
index 5696570d23e6..ec874a22389d 100644
--- a/crates/core_simd/src/mod.rs
+++ b/crates/core_simd/src/mod.rs
@@ -1,8 +1,9 @@
 #[macro_use]
-mod permute;
-#[macro_use]
 mod reduction;
 
+#[macro_use]
+mod swizzle;
+
 pub(crate) mod intrinsics;
 
 #[cfg(feature = "generic_const_exprs")]
@@ -27,5 +28,6 @@ pub mod simd {
     pub use crate::core_simd::lane_count::{LaneCount, SupportedLaneCount};
     pub use crate::core_simd::masks::*;
     pub use crate::core_simd::select::Select;
+    pub use crate::core_simd::swizzle::*;
     pub use crate::core_simd::vector::*;
 }
diff --git a/crates/core_simd/src/permute.rs b/crates/core_simd/src/permute.rs
deleted file mode 100644
index 3e31c3365e88..000000000000
--- a/crates/core_simd/src/permute.rs
+++ /dev/null
@@ -1,154 +0,0 @@
-use crate::simd::intrinsics;
-use crate::simd::{Simd, SimdElement};
-
-macro_rules! impl_shuffle_lane {
-    { $fn:ident, $n:literal } => {
-        impl<T> Simd<T, $n>
-        where
-            T: SimdElement,
-        {
-            /// A const SIMD shuffle that takes 2 SIMD vectors and produces another vector, using
-            /// the indices in the const parameter. The first or "self" vector will have its lanes
-            /// indexed from 0, and the second vector will have its first lane indexed at $n.
-            /// Indices must be in-bounds of either vector at compile time.
-            ///
-            /// Some SIMD shuffle instructions can be quite slow, so avoiding them by loading data
-            /// into the desired patterns in advance is preferred, but shuffles are still faster
-            /// than storing and reloading from memory.
-            ///
-            /// ```
-            /// #![feature(portable_simd)]
-            /// # #[cfg(feature = "std")] use core_simd::Simd;
-            /// # #[cfg(not(feature = "std"))] use core::simd::Simd;
-            /// let a = Simd::from_array([1.0, 2.0, 3.0, 4.0]);
-            /// let b = Simd::from_array([5.0, 6.0, 7.0, 8.0]);
-            /// const IDXS: [u32; 4] = [4,0,3,7];
-            /// let c = Simd::<_, 4>::shuffle::<IDXS>(a,b);
-            /// assert_eq!(Simd::from_array([5.0, 1.0, 4.0, 8.0]), c);
-            /// ```
-            #[inline]
-            pub fn shuffle<const IDX: [u32; $n]>(self, second: Self) -> Self {
-                unsafe { intrinsics::$fn(self, second, IDX) }
-            }
-
-            /// Reverse the order of the lanes in the vector.
-            #[inline]
-            pub fn reverse(self) -> Self {
-                const fn idx() -> [u32; $n] {
-                    let mut idx = [0u32; $n];
-                    let mut i = 0;
-                    while i < $n {
-                        idx[i] = ($n - i - 1) as u32;
-                        i += 1;
-                    }
-                    idx
-                }
-                self.shuffle::<{ idx() }>(self)
-            }
-
-            /// Interleave two vectors.
-            ///
-            /// Produces two vectors with lanes taken alternately from `self` and `other`.
-            ///
-            /// The first result contains the first `LANES / 2` lanes from `self` and `other`,
-            /// alternating, starting with the first lane of `self`.
-            ///
-            /// The second result contains the last `LANES / 2` lanes from `self` and `other`,
-            /// alternating, starting with the lane `LANES / 2` from the start of `self`.
-            ///
-            /// This particular permutation is efficient on many architectures.
-            ///
-            /// ```
-            /// #![feature(portable_simd)]
-            /// # #[cfg(feature = "std")] use core_simd::Simd;
-            /// # #[cfg(not(feature = "std"))] use core::simd::Simd;
-            /// let a = Simd::from_array([0, 1, 2, 3]);
-            /// let b = Simd::from_array([4, 5, 6, 7]);
-            /// let (x, y) = a.interleave(b);
-            /// assert_eq!(x.to_array(), [0, 4, 1, 5]);
-            /// assert_eq!(y.to_array(), [2, 6, 3, 7]);
-            /// ```
-            #[inline]
-            pub fn interleave(self, other: Self) -> (Self, Self) {
-                const fn lo() -> [u32; $n] {
-                    let mut idx = [0u32; $n];
-                    let mut i = 0;
-                    while i < $n {
-                        let offset = i / 2;
-                        idx[i] = if i % 2 == 0 {
-                            offset
-                        } else {
-                            $n + offset
-                        } as u32;
-                        i += 1;
-                    }
-                    idx
-                }
-                const fn hi() -> [u32; $n] {
-                    let mut idx = [0u32; $n];
-                    let mut i = 0;
-                    while i < $n {
-                        let offset = ($n + i) / 2;
-                        idx[i] = if i % 2 == 0 {
-                            offset
-                        } else {
-                            $n + offset
-                        } as u32;
-                        i += 1;
-                    }
-                    idx
-                }
-                (self.shuffle::<{ lo() }>(other), self.shuffle::<{ hi() }>(other))
-            }
-
-            /// Deinterleave two vectors.
-            ///
-            /// The first result takes every other lane of `self` and then `other`, starting with
-            /// the first lane.
-            ///
-            /// The second result takes every other lane of `self` and then `other`, starting with
-            /// the second lane.
-            ///
-            /// This particular permutation is efficient on many architectures.
-            ///
-            /// ```
-            /// #![feature(portable_simd)]
-            /// # #[cfg(feature = "std")] use core_simd::Simd;
-            /// # #[cfg(not(feature = "std"))] use core::simd::Simd;
-            /// let a = Simd::from_array([0, 4, 1, 5]);
-            /// let b = Simd::from_array([2, 6, 3, 7]);
-            /// let (x, y) = a.deinterleave(b);
-            /// assert_eq!(x.to_array(), [0, 1, 2, 3]);
-            /// assert_eq!(y.to_array(), [4, 5, 6, 7]);
-            /// ```
-            #[inline]
-            pub fn deinterleave(self, other: Self) -> (Self, Self) {
-                const fn even() -> [u32; $n] {
-                    let mut idx = [0u32; $n];
-                    let mut i = 0;
-                    while i < $n {
-                        idx[i] = 2 * i as u32;
-                        i += 1;
-                    }
-                    idx
-                }
-                const fn odd() -> [u32; $n] {
-                    let mut idx = [0u32; $n];
-                    let mut i = 0;
-                    while i < $n {
-                        idx[i] = 1 + 2 * i as u32;
-                        i += 1;
-                    }
-                    idx
-                }
-                (self.shuffle::<{ even() }>(other), self.shuffle::<{ odd() }>(other))
-            }
-        }
-    }
-}
-
-impl_shuffle_lane! { simd_shuffle2, 2 }
-impl_shuffle_lane! { simd_shuffle4, 4 }
-impl_shuffle_lane! { simd_shuffle8, 8 }
-impl_shuffle_lane! { simd_shuffle16, 16 }
-impl_shuffle_lane! { simd_shuffle32, 32 }
diff --git a/crates/core_simd/src/swizzle.rs b/crates/core_simd/src/swizzle.rs
new file mode 100644
index 000000000000..048945ddffa8
--- /dev/null
+++ b/crates/core_simd/src/swizzle.rs
@@ -0,0 +1,364 @@
+use crate::simd::intrinsics;
+use crate::{LaneCount, Simd, SimdElement, SupportedLaneCount};
+
+/// Rearrange vector elements.
+///
+/// A new vector is constructed by specifying the the lanes of the source vector or vectors to use.
+///
+/// When shuffling one vector, the indices of the result vector are indicated by a `const` array
+/// of `usize`, like [`Swizzle`].
+/// When shuffling two vectors, the indices are indicated by a `const` array of [`Which`], like
+/// [`Swizzle2`].
+///
+/// # Examples
+/// ## One source vector
+/// ```
+/// # #![feature(portable_simd)]
+/// # use core_simd::{Simd, simd_shuffle};
+/// let v = Simd::<f32, 4>::from_array([0., 1., 2., 3.]);
+/// let v = simd_shuffle!(v, [3, 0, 1, 2]);
+/// assert_eq!(v.to_array(), [3., 0., 1., 2.]);
+/// ```
+///
+/// ## Two source vectors
+/// ```
+/// # #![feature(portable_simd)]
+/// # use core_simd::{Simd, simd_shuffle, Which};
+/// use Which::*;
+/// let a = Simd::<f32, 4>::from_array([0., 1., 2., 3.]);
+/// let b = Simd::<f32, 4>::from_array([4., 5., 6., 7.]);
+/// let v = simd_shuffle!(a, b, [First(0), First(1), Second(2), Second(3)]);
+/// assert_eq!(v.to_array(), [0., 1., 6., 7.]);
+/// ```
+#[macro_export]
+macro_rules! simd_shuffle {
+    {
+        $vector:expr, $index:expr $(,)?
+    } => {
+        {
+            // FIXME this won't work when we are in `core`!
+            use $crate::Swizzle;
+            struct Shuffle;
+            impl Swizzle<{$index.len()}, {$index.len()}> for Shuffle {
+                const INDEX: [usize; {$index.len()}] = $index;
+            }
+            Shuffle::swizzle($vector)
+        }
+    };
+    {
+        $first:expr, $second:expr, $index:expr $(,)?
+    } => {
+        {
+            // FIXME this won't work when we are in `core`!
+            use $crate::{Which, Swizzle2};
+            struct Shuffle;
+            impl Swizzle2<{$index.len()}, {$index.len()}> for Shuffle {
+                const INDEX: [Which; {$index.len()}] = $index;
+            }
+            Shuffle::swizzle2($first, $second)
+        }
+    }
+}
+
+/// An index into one of two vectors.
+#[derive(Copy, Clone, Debug, PartialEq, Eq, PartialOrd, Ord, Hash)]
+pub enum Which {
+    /// Indexes the first vector.
+    First(usize),
+    /// Indexes the second vector.
+    Second(usize),
+}
+
+/// Create a vector from the elements of another vector.
+pub trait Swizzle<const INPUT_LANES: usize, const OUTPUT_LANES: usize> {
+    /// Map from the lanes of the input vector to the output vector.
+    const INDEX: [usize; OUTPUT_LANES];
+
+    /// Create a new vector from the lanes of `vector`.
+    ///
+    /// Lane `i` of the output is `vector[Self::INDEX[i]]`.
+    fn swizzle<T>(vector: Simd<T, INPUT_LANES>) -> Simd<T, OUTPUT_LANES>
+    where
+        T: SimdElement,
+        LaneCount<INPUT_LANES>: SupportedLaneCount,
+        LaneCount<OUTPUT_LANES>: SupportedLaneCount,
+    {
+        unsafe { intrinsics::simd_shuffle(vector, vector, Self::INDEX_IMPL) }
+    }
+}
+
+/// Create a vector from the elements of two other vectors.
+pub trait Swizzle2<const INPUT_LANES: usize, const OUTPUT_LANES: usize> {
+    /// Map from the lanes of the input vectors to the output vector
+    const INDEX: [Which; OUTPUT_LANES];
+
+    /// Create a new vector from the lanes of `first` and `second`.
+    ///
+    /// Lane `i` is `first[j]` when `Self::INDEX[i]` is `First(j)`, or `second[j]` when it is
+    /// `Second(j)`.
+    fn swizzle2<T>(
+        first: Simd<T, INPUT_LANES>,
+        second: Simd<T, INPUT_LANES>,
+    ) -> Simd<T, OUTPUT_LANES>
+    where
+        T: SimdElement,
+        LaneCount<INPUT_LANES>: SupportedLaneCount,
+        LaneCount<OUTPUT_LANES>: SupportedLaneCount,
+    {
+        unsafe { intrinsics::simd_shuffle(first, second, Self::INDEX_IMPL) }
+    }
+}
+
+/// The `simd_shuffle` intrinsic expects `u32`, so do error checking and conversion here.
+trait SwizzleImpl<const INPUT_LANES: usize, const OUTPUT_LANES: usize> {
+    const INDEX_IMPL: [u32; OUTPUT_LANES];
+}
+
+impl<T, const INPUT_LANES: usize, const OUTPUT_LANES: usize> SwizzleImpl<INPUT_LANES, OUTPUT_LANES>
+    for T
+where
+    T: Swizzle<INPUT_LANES, OUTPUT_LANES> + ?Sized,
+{
+    const INDEX_IMPL: [u32; OUTPUT_LANES] = {
+        let mut output = [0; OUTPUT_LANES];
+        let mut i = 0;
+        while i < OUTPUT_LANES {
+            let index = Self::INDEX[i];
+            assert!(index as u32 as usize == index);
+            assert!(index < INPUT_LANES, "source lane exceeds input lane count",);
+            output[i] = index as u32;
+            i += 1;
+        }
+        output
+    };
+}
+
+/// The `simd_shuffle` intrinsic expects `u32`, so do error checking and conversion here.
+trait Swizzle2Impl<const INPUT_LANES: usize, const OUTPUT_LANES: usize> {
+    const INDEX_IMPL: [u32; OUTPUT_LANES];
+}
+
+impl<T, const INPUT_LANES: usize, const OUTPUT_LANES: usize> Swizzle2Impl<INPUT_LANES, OUTPUT_LANES>
+    for T
+where
+    T: Swizzle2<INPUT_LANES, OUTPUT_LANES> + ?Sized,
+{
+    const INDEX_IMPL: [u32; OUTPUT_LANES] = {
+        let mut output = [0; OUTPUT_LANES];
+        let mut i = 0;
+        while i < OUTPUT_LANES {
+            let (offset, index) = match Self::INDEX[i] {
+                Which::First(index) => (false, index),
+                Which::Second(index) => (true, index),
+            };
+            assert!(index < INPUT_LANES, "source lane exceeds input lane count",);
+
+            // lanes are indexed by the first vector, then second vector
+            let index = if offset { index + INPUT_LANES } else { index };
+            assert!(index as u32 as usize == index);
+            output[i] = index as u32;
+            i += 1;
+        }
+        output
+    };
+}
+
+impl<T, const LANES: usize> Simd<T, LANES>
+where
+    T: SimdElement,
+    LaneCount<LANES>: SupportedLaneCount,
+{
+    /// Reverse the order of the lanes in the vector.
+    #[inline]
+    pub fn reverse(self) -> Self {
+        const fn reverse_index<const LANES: usize>() -> [usize; LANES] {
+            let mut index = [0; LANES];
+            let mut i = 0;
+            while i < LANES {
+                index[i] = LANES - i - 1;
+                i += 1;
+            }
+            index
+        }
+
+        struct Reverse;
+
+        impl<const LANES: usize> Swizzle<LANES, LANES> for Reverse {
+            const INDEX: [usize; LANES] = reverse_index::<LANES>();
+        }
+
+        Reverse::swizzle(self)
+    }
+
+    /// Rotates the vector such that the first `OFFSET` elements of the slice move to the end
+    /// while the last `LANES - OFFSET` elements move to the front. After calling `rotate_left`, the
+    /// element previously in lane `OFFSET` will become the first element in the slice.
+    #[inline]
+    pub fn rotate_left<const OFFSET: usize>(self) -> Self {
+        const fn rotate_index<const OFFSET: usize, const LANES: usize>() -> [usize; LANES] {
+            let offset = OFFSET % LANES;
+            let mut index = [0; LANES];
+            let mut i = 0;
+            while i < LANES {
+                index[i] = (i + offset) % LANES;
+                i += 1;
+            }
+            index
+        }
+
+        struct Rotate<const OFFSET: usize>;
+
+        impl<const OFFSET: usize, const LANES: usize> Swizzle<LANES, LANES> for Rotate<OFFSET> {
+            const INDEX: [usize; LANES] = rotate_index::<OFFSET, LANES>();
+        }
+
+        Rotate::<OFFSET>::swizzle(self)
+    }
+
+    /// Rotates the vector such that the first `LANES - OFFSET` elements of the vector move to
+    /// the end while the last `OFFSET` elements move to the front. After calling `rotate_right`, the
+    /// element previously at index `LANES - OFFSET` will become the first element in the slice.
+    #[inline]
+    pub fn rotate_right<const OFFSET: usize>(self) -> Self {
+        const fn rotate_index<const OFFSET: usize, const LANES: usize>() -> [usize; LANES] {
+            let offset = LANES - OFFSET % LANES;
+            let mut index = [0; LANES];
+            let mut i = 0;
+            while i < LANES {
+                index[i] = (i + offset) % LANES;
+                i += 1;
+            }
+            index
+        }
+
+        struct Rotate<const OFFSET: usize>;
+
+        impl<const OFFSET: usize, const LANES: usize> Swizzle<LANES, LANES> for Rotate<OFFSET> {
+            const INDEX: [usize; LANES] = rotate_index::<OFFSET, LANES>();
+        }
+
+        Rotate::<OFFSET>::swizzle(self)
+    }
+
+    /// Interleave two vectors.
+    ///
+    /// Produces two vectors with lanes taken alternately from `self` and `other`.
+    ///
+    /// The first result contains the first `LANES / 2` lanes from `self` and `other`,
+    /// alternating, starting with the first lane of `self`.
+    ///
+    /// The second result contains the last `LANES / 2` lanes from `self` and `other`,
+    /// alternating, starting with the lane `LANES / 2` from the start of `self`.
+    ///
+    /// This particular permutation is efficient on many architectures.
+    ///
+    /// ```
+    /// #![feature(portable_simd)]
+    /// # use core_simd::Simd;
+    /// let a = Simd::from_array([0, 1, 2, 3]);
+    /// let b = Simd::from_array([4, 5, 6, 7]);
+    /// let (x, y) = a.interleave(b);
+    /// assert_eq!(x.to_array(), [0, 4, 1, 5]);
+    /// assert_eq!(y.to_array(), [2, 6, 3, 7]);
+    /// ```
+    #[inline]
+    pub fn interleave(self, other: Self) -> (Self, Self) {
+        const fn lo<const LANES: usize>() -> [Which; LANES] {
+            let mut idx = [Which::First(0); LANES];
+            let mut i = 0;
+            while i < LANES {
+                let offset = i / 2;
+                idx[i] = if i % 2 == 0 {
+                    Which::First(offset)
+                } else {
+                    Which::Second(offset)
+                };
+                i += 1;
+            }
+            idx
+        }
+        const fn hi<const LANES: usize>() -> [Which; LANES] {
+            let mut idx = [Which::First(0); LANES];
+            let mut i = 0;
+            while i < LANES {
+                let offset = (LANES + i) / 2;
+                idx[i] = if i % 2 == 0 {
+                    Which::First(offset)
+                } else {
+                    Which::Second(offset)
+                };
+                i += 1;
+            }
+            idx
+        }
+
+        struct Lo;
+        struct Hi;
+
+        impl<const LANES: usize> Swizzle2<LANES, LANES> for Lo {
+            const INDEX: [Which; LANES] = lo::<LANES>();
+        }
+
+        impl<const LANES: usize> Swizzle2<LANES, LANES> for Hi {
+            const INDEX: [Which; LANES] = hi::<LANES>();
+        }
+
+        (Lo::swizzle2(self, other), Hi::swizzle2(self, other))
+    }
+
+    /// Deinterleave two vectors.
+    ///
+    /// The first result takes every other lane of `self` and then `other`, starting with
+    /// the first lane.
+    ///
+    /// The second result takes every other lane of `self` and then `other`, starting with
+    /// the second lane.
+    ///
+    /// This particular permutation is efficient on many architectures.
+    ///
+    /// ```
+    /// #![feature(portable_simd)]
+    /// # use core_simd::Simd;
+    /// let a = Simd::from_array([0, 4, 1, 5]);
+    /// let b = Simd::from_array([2, 6, 3, 7]);
+    /// let (x, y) = a.deinterleave(b);
+    /// assert_eq!(x.to_array(), [0, 1, 2, 3]);
+    /// assert_eq!(y.to_array(), [4, 5, 6, 7]);
+    /// ```
+    #[inline]
+    pub fn deinterleave(self, other: Self) -> (Self, Self) {
+        const fn even<const LANES: usize>() -> [Which; LANES] {
+            let mut idx = [Which::First(0); LANES];
+            let mut i = 0;
+            while i < LANES / 2 {
+                idx[i] = Which::First(2 * i);
+                idx[i + LANES / 2] = Which::Second(2 * i);
+                i += 1;
+            }
+            idx
+        }
+        const fn odd<const LANES: usize>() -> [Which; LANES] {
+            let mut idx = [Which::First(0); LANES];
+            let mut i = 0;
+            while i < LANES / 2 {
+                idx[i] = Which::First(2 * i + 1);
+                idx[i + LANES / 2] = Which::Second(2 * i + 1);
+                i += 1;
+            }
+            idx
+        }
+
+        struct Even;
+        struct Odd;
+
+        impl<const LANES: usize> Swizzle2<LANES, LANES> for Even {
+            const INDEX: [Which; LANES] = even::<LANES>();
+        }
+
+        impl<const LANES: usize> Swizzle2<LANES, LANES> for Odd {
+            const INDEX: [Which; LANES] = odd::<LANES>();
+        }
+
+        (Even::swizzle2(self, other), Odd::swizzle2(self, other))
+    }
+}
diff --git a/crates/core_simd/tests/permute.rs b/crates/core_simd/tests/permute.rs
deleted file mode 100644
index ea52e8f5ca73..000000000000
--- a/crates/core_simd/tests/permute.rs
+++ /dev/null
@@ -1,37 +0,0 @@
-#![feature(portable_simd)]
-
-use core_simd::Simd;
-
-#[cfg(target_arch = "wasm32")]
-use wasm_bindgen_test::*;
-
-#[cfg(target_arch = "wasm32")]
-wasm_bindgen_test_configure!(run_in_browser);
-
-#[test]
-#[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
-fn simple_shuffle() {
-    let a = Simd::from_array([2, 4, 1, 9]);
-    let b = a;
-    assert_eq!(a.shuffle::<{ [3, 1, 4, 6] }>(b).to_array(), [9, 4, 2, 1]);
-}
-
-#[test]
-#[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
-fn reverse() {
-    let a = Simd::from_array([0, 1, 2, 3, 4, 5, 6, 7]);
-    assert_eq!(a.reverse().to_array(), [7, 6, 5, 4, 3, 2, 1, 0]);
-}
-
-#[test]
-#[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
-fn interleave() {
-    let a = Simd::from_array([0, 1, 2, 3, 4, 5, 6, 7]);
-    let b = Simd::from_array([8, 9, 10, 11, 12, 13, 14, 15]);
-    let (lo, hi) = a.interleave(b);
-    assert_eq!(lo.to_array(), [0, 8, 1, 9, 2, 10, 3, 11]);
-    assert_eq!(hi.to_array(), [4, 12, 5, 13, 6, 14, 7, 15]);
-    let (even, odd) = lo.deinterleave(hi);
-    assert_eq!(even, a);
-    assert_eq!(odd, b);
-}
diff --git a/crates/core_simd/tests/swizzle.rs b/crates/core_simd/tests/swizzle.rs
new file mode 100644
index 000000000000..d4abc46b9327
--- /dev/null
+++ b/crates/core_simd/tests/swizzle.rs
@@ -0,0 +1,62 @@
+#![feature(portable_simd)]
+use core_simd::{Simd, Swizzle};
+
+#[cfg(target_arch = "wasm32")]
+use wasm_bindgen_test::*;
+
+#[cfg(target_arch = "wasm32")]
+wasm_bindgen_test_configure!(run_in_browser);
+
+#[test]
+#[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
+fn swizzle() {
+    struct Index;
+    impl Swizzle<4, 4> for Index {
+        const INDEX: [usize; 4] = [2, 1, 3, 0];
+    }
+    impl Swizzle<4, 2> for Index {
+        const INDEX: [usize; 2] = [1, 1];
+    }
+
+    let vector = Simd::from_array([2, 4, 1, 9]);
+    assert_eq!(Index::swizzle(vector).to_array(), [1, 4, 9, 2]);
+    assert_eq!(Index::swizzle(vector).to_array(), [4, 4]);
+}
+
+#[test]
+#[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
+fn reverse() {
+    let a = Simd::from_array([1, 2, 3, 4]);
+    assert_eq!(a.reverse().to_array(), [4, 3, 2, 1]);
+}
+
+#[test]
+#[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
+fn rotate() {
+    let a = Simd::from_array([1, 2, 3, 4]);
+    assert_eq!(a.rotate_left::<0>().to_array(), [1, 2, 3, 4]);
+    assert_eq!(a.rotate_left::<1>().to_array(), [2, 3, 4, 1]);
+    assert_eq!(a.rotate_left::<2>().to_array(), [3, 4, 1, 2]);
+    assert_eq!(a.rotate_left::<3>().to_array(), [4, 1, 2, 3]);
+    assert_eq!(a.rotate_left::<4>().to_array(), [1, 2, 3, 4]);
+    assert_eq!(a.rotate_left::<5>().to_array(), [2, 3, 4, 1]);
+    assert_eq!(a.rotate_right::<0>().to_array(), [1, 2, 3, 4]);
+    assert_eq!(a.rotate_right::<1>().to_array(), [4, 1, 2, 3]);
+    assert_eq!(a.rotate_right::<2>().to_array(), [3, 4, 1, 2]);
+    assert_eq!(a.rotate_right::<3>().to_array(), [2, 3, 4, 1]);
+    assert_eq!(a.rotate_right::<4>().to_array(), [1, 2, 3, 4]);
+    assert_eq!(a.rotate_right::<5>().to_array(), [4, 1, 2, 3]);
+}
+
+#[test]
+#[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
+fn interleave() {
+    let a = Simd::from_array([0, 1, 2, 3, 4, 5, 6, 7]);
+    let b = Simd::from_array([8, 9, 10, 11, 12, 13, 14, 15]);
+    let (lo, hi) = a.interleave(b);
+    assert_eq!(lo.to_array(), [0, 8, 1, 9, 2, 10, 3, 11]);
+    assert_eq!(hi.to_array(), [4, 12, 5, 13, 6, 14, 7, 15]);
+    let (even, odd) = lo.deinterleave(hi);
+    assert_eq!(even, a);
+    assert_eq!(odd, b);
+}

From 98e4fcae5aa0080d97901835e8391ec394acfef6 Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Wed, 22 Sep 2021 19:45:09 -0400
Subject: [PATCH 234/249] Fix macro in core

Co-authored-by: Jubilee <46493976+workingjubilee@users.noreply.github.com>
---
 crates/core_simd/src/swizzle.rs | 6 ++----
 1 file changed, 2 insertions(+), 4 deletions(-)

diff --git a/crates/core_simd/src/swizzle.rs b/crates/core_simd/src/swizzle.rs
index 048945ddffa8..5ba9f3dec59d 100644
--- a/crates/core_simd/src/swizzle.rs
+++ b/crates/core_simd/src/swizzle.rs
@@ -36,8 +36,7 @@ macro_rules! simd_shuffle {
         $vector:expr, $index:expr $(,)?
     } => {
         {
-            // FIXME this won't work when we are in `core`!
-            use $crate::Swizzle;
+            use $crate::simd::Swizzle;
             struct Shuffle;
             impl Swizzle<{$index.len()}, {$index.len()}> for Shuffle {
                 const INDEX: [usize; {$index.len()}] = $index;
@@ -49,8 +48,7 @@ macro_rules! simd_shuffle {
         $first:expr, $second:expr, $index:expr $(,)?
     } => {
         {
-            // FIXME this won't work when we are in `core`!
-            use $crate::{Which, Swizzle2};
+            use $crate::simd::{Which, Swizzle2};
             struct Shuffle;
             impl Swizzle2<{$index.len()}, {$index.len()}> for Shuffle {
                 const INDEX: [Which; {$index.len()}] = $index;

From 37797d9c0a7e2f28fec5273399f86edc5cff7ae3 Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Thu, 23 Sep 2021 00:28:07 +0000
Subject: [PATCH 235/249] simd_shuffle -> simd_swizzle

---
 crates/core_simd/examples/matrix_inversion.rs | 16 +++----
 crates/core_simd/src/lib.rs                   |  2 +-
 crates/core_simd/src/swizzle.rs               | 42 ++++++++++++-------
 3 files changed, 36 insertions(+), 24 deletions(-)

diff --git a/crates/core_simd/examples/matrix_inversion.rs b/crates/core_simd/examples/matrix_inversion.rs
index ee8c477b8386..468319325e22 100644
--- a/crates/core_simd/examples/matrix_inversion.rs
+++ b/crates/core_simd/examples/matrix_inversion.rs
@@ -169,16 +169,16 @@ pub fn simd_inv4x4(m: Matrix4x4) -> Option<Matrix4x4> {
     const SHUFFLE13: [Which; 4] = [First(1), First(3), Second(1), Second(3)];
     const SHUFFLE23: [Which; 4] = [First(2), First(3), Second(2), Second(3)];
 
-    let tmp = simd_shuffle!(m_0, m_1, SHUFFLE01);
-    let row1 = simd_shuffle!(m_2, m_3, SHUFFLE01);
+    let tmp = simd_swizzle!(m_0, m_1, SHUFFLE01);
+    let row1 = simd_swizzle!(m_2, m_3, SHUFFLE01);
 
-    let row0 = simd_shuffle!(tmp, row1, SHUFFLE02);
-    let row1 = simd_shuffle!(row1, tmp, SHUFFLE13);
+    let row0 = simd_swizzle!(tmp, row1, SHUFFLE02);
+    let row1 = simd_swizzle!(row1, tmp, SHUFFLE13);
 
-    let tmp = simd_shuffle!(m_0, m_1, SHUFFLE23);
-    let row3 = simd_shuffle!(m_2, m_3, SHUFFLE23);
-    let row2 = simd_shuffle!(tmp, row3, SHUFFLE02);
-    let row3 = simd_shuffle!(row3, tmp, SHUFFLE13);
+    let tmp = simd_swizzle!(m_0, m_1, SHUFFLE23);
+    let row3 = simd_swizzle!(m_2, m_3, SHUFFLE23);
+    let row2 = simd_swizzle!(tmp, row3, SHUFFLE02);
+    let row3 = simd_swizzle!(row3, tmp, SHUFFLE13);
 
     let tmp = (row2 * row3).reverse().rotate_right::<2>();
     let minor0 = row1 * tmp;
diff --git a/crates/core_simd/src/lib.rs b/crates/core_simd/src/lib.rs
index 55b8be97e0ee..1d8cdad8900a 100644
--- a/crates/core_simd/src/lib.rs
+++ b/crates/core_simd/src/lib.rs
@@ -18,5 +18,5 @@
 
 #[path = "mod.rs"]
 mod core_simd;
-use self::core_simd::simd;
+pub use self::core_simd::simd;
 pub use simd::*;
diff --git a/crates/core_simd/src/swizzle.rs b/crates/core_simd/src/swizzle.rs
index 5ba9f3dec59d..d4702784dc5f 100644
--- a/crates/core_simd/src/swizzle.rs
+++ b/crates/core_simd/src/swizzle.rs
@@ -5,43 +5,55 @@ use crate::{LaneCount, Simd, SimdElement, SupportedLaneCount};
 ///
 /// A new vector is constructed by specifying the the lanes of the source vector or vectors to use.
 ///
-/// When shuffling one vector, the indices of the result vector are indicated by a `const` array
+/// When swizzling one vector, the indices of the result vector are indicated by a `const` array
 /// of `usize`, like [`Swizzle`].
-/// When shuffling two vectors, the indices are indicated by a `const` array of [`Which`], like
+/// When swizzling two vectors, the indices are indicated by a `const` array of [`Which`], like
 /// [`Swizzle2`].
 ///
 /// # Examples
 /// ## One source vector
 /// ```
 /// # #![feature(portable_simd)]
-/// # use core_simd::{Simd, simd_shuffle};
+/// # use core_simd::{Simd, simd_swizzle};
 /// let v = Simd::<f32, 4>::from_array([0., 1., 2., 3.]);
-/// let v = simd_shuffle!(v, [3, 0, 1, 2]);
-/// assert_eq!(v.to_array(), [3., 0., 1., 2.]);
+///
+/// // Keeping the same size
+/// let r = simd_swizzle!(v, [3, 0, 1, 2]);
+/// assert_eq!(r.to_array(), [3., 0., 1., 2.]);
+///
+/// // Changing the number of lanes
+/// let r = simd_swizzle!(v, [3, 1]);
+/// assert_eq!(r.to_array(), [3., 1.]);
 /// ```
 ///
 /// ## Two source vectors
 /// ```
 /// # #![feature(portable_simd)]
-/// # use core_simd::{Simd, simd_shuffle, Which};
+/// # use core_simd::{Simd, simd_swizzle, Which};
 /// use Which::*;
 /// let a = Simd::<f32, 4>::from_array([0., 1., 2., 3.]);
 /// let b = Simd::<f32, 4>::from_array([4., 5., 6., 7.]);
-/// let v = simd_shuffle!(a, b, [First(0), First(1), Second(2), Second(3)]);
-/// assert_eq!(v.to_array(), [0., 1., 6., 7.]);
+///
+/// // Keeping the same size
+/// let r = simd_swizzle!(a, b, [First(0), First(1), Second(2), Second(3)]);
+/// assert_eq!(r.to_array(), [0., 1., 6., 7.]);
+///
+/// // Changing the number of lanes
+/// let r = simd_swizzle!(a, b, [First(0), Second(0)]);
+/// assert_eq!(r.to_array(), [0., 4.]);
 /// ```
 #[macro_export]
-macro_rules! simd_shuffle {
+macro_rules! simd_swizzle {
     {
         $vector:expr, $index:expr $(,)?
     } => {
         {
             use $crate::simd::Swizzle;
-            struct Shuffle;
-            impl Swizzle<{$index.len()}, {$index.len()}> for Shuffle {
+            struct SwizzleImpl;
+            impl<const LANES: usize> Swizzle<LANES, {$index.len()}> for SwizzleImpl {
                 const INDEX: [usize; {$index.len()}] = $index;
             }
-            Shuffle::swizzle($vector)
+            SwizzleImpl::swizzle($vector)
         }
     };
     {
@@ -49,11 +61,11 @@ macro_rules! simd_shuffle {
     } => {
         {
             use $crate::simd::{Which, Swizzle2};
-            struct Shuffle;
-            impl Swizzle2<{$index.len()}, {$index.len()}> for Shuffle {
+            struct SwizzleImpl;
+            impl<const LANES: usize> Swizzle2<LANES, {$index.len()}> for SwizzleImpl {
                 const INDEX: [Which; {$index.len()}] = $index;
             }
-            Shuffle::swizzle2($first, $second)
+            SwizzleImpl::swizzle2($first, $second)
         }
     }
 }

From cd7ecba19fa79e45660b5f5d315c302bd5994b10 Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Fri, 1 Oct 2021 16:46:36 -0400
Subject: [PATCH 236/249] Remove adt_const_params feature

---
 crates/core_simd/src/lib.rs | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/crates/core_simd/src/lib.rs b/crates/core_simd/src/lib.rs
index 1d8cdad8900a..037779200b85 100644
--- a/crates/core_simd/src/lib.rs
+++ b/crates/core_simd/src/lib.rs
@@ -1,7 +1,5 @@
 #![cfg_attr(not(feature = "std"), no_std)]
-#![allow(incomplete_features)]
 #![feature(
-    adt_const_params,
     const_fn_trait_bound,
     const_panic,
     platform_intrinsics,
@@ -11,6 +9,7 @@
     stdsimd
 )]
 #![cfg_attr(feature = "generic_const_exprs", feature(generic_const_exprs))]
+#![cfg_attr(feature = "generic_const_exprs", allow(incomplete_features))]
 #![warn(missing_docs)]
 #![deny(unsafe_op_in_unsafe_fn)]
 #![unstable(feature = "portable_simd", issue = "86656")]

From 765bee6362371185159cc68fbb708c64a621ae3c Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Sat, 2 Oct 2021 05:07:51 -0400
Subject: [PATCH 237/249] Update crates/core_simd/src/swizzle.rs

Co-authored-by: Jubilee <46493976+workingjubilee@users.noreply.github.com>
---
 crates/core_simd/src/swizzle.rs | 4 +---
 1 file changed, 1 insertion(+), 3 deletions(-)

diff --git a/crates/core_simd/src/swizzle.rs b/crates/core_simd/src/swizzle.rs
index d4702784dc5f..9ce46cfe816d 100644
--- a/crates/core_simd/src/swizzle.rs
+++ b/crates/core_simd/src/swizzle.rs
@@ -1,9 +1,7 @@
 use crate::simd::intrinsics;
 use crate::{LaneCount, Simd, SimdElement, SupportedLaneCount};
 
-/// Rearrange vector elements.
-///
-/// A new vector is constructed by specifying the the lanes of the source vector or vectors to use.
+/// Constructs a new vector by selecting values from the lanes of the source vector or vectors to use.
 ///
 /// When swizzling one vector, the indices of the result vector are indicated by a `const` array
 /// of `usize`, like [`Swizzle`].

From 5b4282edcd01089df26e2e2c268c4dc1c361dd91 Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Thu, 7 Oct 2021 17:43:53 -0400
Subject: [PATCH 238/249] Improve docs

---
 crates/core_simd/src/swizzle.rs | 18 ++++++++----------
 1 file changed, 8 insertions(+), 10 deletions(-)

diff --git a/crates/core_simd/src/swizzle.rs b/crates/core_simd/src/swizzle.rs
index 9ce46cfe816d..d4d171d570e6 100644
--- a/crates/core_simd/src/swizzle.rs
+++ b/crates/core_simd/src/swizzle.rs
@@ -47,11 +47,11 @@ macro_rules! simd_swizzle {
     } => {
         {
             use $crate::simd::Swizzle;
-            struct SwizzleImpl;
-            impl<const LANES: usize> Swizzle<LANES, {$index.len()}> for SwizzleImpl {
+            struct Impl;
+            impl<const LANES: usize> Swizzle<LANES, {$index.len()}> for Impl {
                 const INDEX: [usize; {$index.len()}] = $index;
             }
-            SwizzleImpl::swizzle($vector)
+            Impl::swizzle($vector)
         }
     };
     {
@@ -59,11 +59,11 @@ macro_rules! simd_swizzle {
     } => {
         {
             use $crate::simd::{Which, Swizzle2};
-            struct SwizzleImpl;
-            impl<const LANES: usize> Swizzle2<LANES, {$index.len()}> for SwizzleImpl {
+            struct Impl;
+            impl<const LANES: usize> Swizzle2<LANES, {$index.len()}> for Impl {
                 const INDEX: [Which; {$index.len()}] = $index;
             }
-            SwizzleImpl::swizzle2($first, $second)
+            Impl::swizzle2($first, $second)
         }
     }
 }
@@ -118,6 +118,7 @@ pub trait Swizzle2<const INPUT_LANES: usize, const OUTPUT_LANES: usize> {
 }
 
 /// The `simd_shuffle` intrinsic expects `u32`, so do error checking and conversion here.
+/// This trait hides `INDEX_IMPL` from the public API.
 trait SwizzleImpl<const INPUT_LANES: usize, const OUTPUT_LANES: usize> {
     const INDEX_IMPL: [u32; OUTPUT_LANES];
 }
@@ -142,6 +143,7 @@ where
 }
 
 /// The `simd_shuffle` intrinsic expects `u32`, so do error checking and conversion here.
+/// This trait hides `INDEX_IMPL` from the public API.
 trait Swizzle2Impl<const INPUT_LANES: usize, const OUTPUT_LANES: usize> {
     const INDEX_IMPL: [u32; OUTPUT_LANES];
 }
@@ -258,8 +260,6 @@ where
     /// The second result contains the last `LANES / 2` lanes from `self` and `other`,
     /// alternating, starting with the lane `LANES / 2` from the start of `self`.
     ///
-    /// This particular permutation is efficient on many architectures.
-    ///
     /// ```
     /// #![feature(portable_simd)]
     /// # use core_simd::Simd;
@@ -322,8 +322,6 @@ where
     /// The second result takes every other lane of `self` and then `other`, starting with
     /// the second lane.
     ///
-    /// This particular permutation is efficient on many architectures.
-    ///
     /// ```
     /// #![feature(portable_simd)]
     /// # use core_simd::Simd;

From ab8eec7cbabdefdab319d90cc8b0eca61fc7f3dd Mon Sep 17 00:00:00 2001
From: Jubilee Young <workingjubilee@gmail.com>
Date: Thu, 21 Oct 2021 18:20:06 -0700
Subject: [PATCH 239/249] Fixup import pathing for core

This changes simd_swizzle! to a decl_macro to give it a path,
so it can be imported using a path and not the crate root.
It also adds various uses that were missed and adjusts paths.
---
 crates/core_simd/examples/matrix_inversion.rs |  4 +--
 crates/core_simd/src/lib.rs                   |  2 +-
 crates/core_simd/src/swizzle.rs               | 28 +++++++++++--------
 3 files changed, 19 insertions(+), 15 deletions(-)

diff --git a/crates/core_simd/examples/matrix_inversion.rs b/crates/core_simd/examples/matrix_inversion.rs
index 468319325e22..fcee1b96ae12 100644
--- a/crates/core_simd/examples/matrix_inversion.rs
+++ b/crates/core_simd/examples/matrix_inversion.rs
@@ -2,8 +2,8 @@
 // Code ported from the `packed_simd` crate
 // Run this code with `cargo test --example matrix_inversion`
 #![feature(array_chunks, portable_simd)]
-use core_simd::Which::*;
-use core_simd::*;
+use core_simd::simd::*;
+use Which::*;
 
 // Gotta define our own 4x4 matrix since Rust doesn't ship multidim arrays yet :^)
 #[derive(Copy, Clone, Debug, PartialEq, PartialOrd)]
diff --git a/crates/core_simd/src/lib.rs b/crates/core_simd/src/lib.rs
index 037779200b85..960a66400839 100644
--- a/crates/core_simd/src/lib.rs
+++ b/crates/core_simd/src/lib.rs
@@ -1,7 +1,7 @@
 #![cfg_attr(not(feature = "std"), no_std)]
 #![feature(
     const_fn_trait_bound,
-    const_panic,
+    decl_macro,
     platform_intrinsics,
     repr_simd,
     simd_ffi,
diff --git a/crates/core_simd/src/swizzle.rs b/crates/core_simd/src/swizzle.rs
index d4d171d570e6..88e7f3b223ec 100644
--- a/crates/core_simd/src/swizzle.rs
+++ b/crates/core_simd/src/swizzle.rs
@@ -1,5 +1,5 @@
 use crate::simd::intrinsics;
-use crate::{LaneCount, Simd, SimdElement, SupportedLaneCount};
+use crate::simd::{LaneCount, Simd, SimdElement, SupportedLaneCount};
 
 /// Constructs a new vector by selecting values from the lanes of the source vector or vectors to use.
 ///
@@ -12,7 +12,8 @@ use crate::{LaneCount, Simd, SimdElement, SupportedLaneCount};
 /// ## One source vector
 /// ```
 /// # #![feature(portable_simd)]
-/// # use core_simd::{Simd, simd_swizzle};
+/// # #[cfg(feature = "std")] use core_simd::{Simd, simd_swizzle};
+/// # #[cfg(not(feature = "std"))] use core::simd::{Simd, simd_swizzle};
 /// let v = Simd::<f32, 4>::from_array([0., 1., 2., 3.]);
 ///
 /// // Keeping the same size
@@ -27,7 +28,8 @@ use crate::{LaneCount, Simd, SimdElement, SupportedLaneCount};
 /// ## Two source vectors
 /// ```
 /// # #![feature(portable_simd)]
-/// # use core_simd::{Simd, simd_swizzle, Which};
+/// # #[cfg(feature = "std")] use core_simd::{Simd, simd_swizzle, Which};
+/// # #[cfg(not(feature = "std"))] use core::simd::{Simd, simd_swizzle, Which};
 /// use Which::*;
 /// let a = Simd::<f32, 4>::from_array([0., 1., 2., 3.]);
 /// let b = Simd::<f32, 4>::from_array([4., 5., 6., 7.]);
@@ -40,11 +42,11 @@ use crate::{LaneCount, Simd, SimdElement, SupportedLaneCount};
 /// let r = simd_swizzle!(a, b, [First(0), Second(0)]);
 /// assert_eq!(r.to_array(), [0., 4.]);
 /// ```
-#[macro_export]
-macro_rules! simd_swizzle {
-    {
+#[allow(unused_macros)]
+pub macro simd_swizzle {
+    (
         $vector:expr, $index:expr $(,)?
-    } => {
+    ) => {
         {
             use $crate::simd::Swizzle;
             struct Impl;
@@ -53,10 +55,10 @@ macro_rules! simd_swizzle {
             }
             Impl::swizzle($vector)
         }
-    };
-    {
+    },
+    (
         $first:expr, $second:expr, $index:expr $(,)?
-    } => {
+    ) => {
         {
             use $crate::simd::{Which, Swizzle2};
             struct Impl;
@@ -262,7 +264,8 @@ where
     ///
     /// ```
     /// #![feature(portable_simd)]
-    /// # use core_simd::Simd;
+    /// # #[cfg(feature = "std")] use core_simd::Simd;
+    /// # #[cfg(not(feature = "std"))] use core::simd::Simd;
     /// let a = Simd::from_array([0, 1, 2, 3]);
     /// let b = Simd::from_array([4, 5, 6, 7]);
     /// let (x, y) = a.interleave(b);
@@ -324,7 +327,8 @@ where
     ///
     /// ```
     /// #![feature(portable_simd)]
-    /// # use core_simd::Simd;
+    /// # #[cfg(feature = "std")] use core_simd::Simd;
+    /// # #[cfg(not(feature = "std"))] use core::simd::Simd;
     /// let a = Simd::from_array([0, 4, 1, 5]);
     /// let b = Simd::from_array([2, 6, 3, 7]);
     /// let (x, y) = a.deinterleave(b);

From 7c2d295a76fcbc6d4c438c5f324903265ee77c94 Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Wed, 22 Sep 2021 23:00:59 +0000
Subject: [PATCH 240/249] Hide mask impl details in sealed trait.

---
 crates/core_simd/src/masks.rs | 38 +++++++++++++++++++++++------------
 1 file changed, 25 insertions(+), 13 deletions(-)

diff --git a/crates/core_simd/src/masks.rs b/crates/core_simd/src/masks.rs
index 1b1677330fb6..d460da0d04f8 100644
--- a/crates/core_simd/src/masks.rs
+++ b/crates/core_simd/src/masks.rs
@@ -16,26 +16,36 @@ use crate::simd::{LaneCount, Simd, SimdElement, SupportedLaneCount};
 use core::cmp::Ordering;
 use core::fmt;
 
-/// Marker trait for types that may be used as SIMD mask elements.
-pub unsafe trait MaskElement: SimdElement {
-    #[doc(hidden)]
-    fn valid<const LANES: usize>(values: Simd<Self, LANES>) -> bool
-    where
-        LaneCount<LANES>: SupportedLaneCount;
+mod sealed {
+    use super::*;
 
-    #[doc(hidden)]
-    fn eq(self, other: Self) -> bool;
+    /// Not only does this seal the `MaskElement` trait, but these functions prevent other traits
+    /// from bleeding into the parent bounds.
+    ///
+    /// For example, `eq` could be provided by requiring `MaskElement: PartialEq`, but that would
+    /// prevent us from ever removing that bound, or from implementing `MaskElement` on
+    /// non-`PartialEq` types in the future.
+    pub trait Sealed {
+        fn valid<const LANES: usize>(values: Simd<Self, LANES>) -> bool
+        where
+            LaneCount<LANES>: SupportedLaneCount,
+            Self: SimdElement;
 
-    #[doc(hidden)]
-    const TRUE: Self;
+        fn eq(self, other: Self) -> bool;
 
-    #[doc(hidden)]
-    const FALSE: Self;
+        const TRUE: Self;
+
+        const FALSE: Self;
+    }
 }
+use sealed::Sealed;
+
+/// Marker trait for types that may be used as SIMD mask elements.
+pub unsafe trait MaskElement: SimdElement + Sealed {}
 
 macro_rules! impl_element {
     { $ty:ty } => {
-        unsafe impl MaskElement for $ty {
+        impl Sealed for $ty {
             fn valid<const LANES: usize>(value: Simd<Self, LANES>) -> bool
             where
                 LaneCount<LANES>: SupportedLaneCount,
@@ -48,6 +58,8 @@ macro_rules! impl_element {
             const TRUE: Self = -1;
             const FALSE: Self = 0;
         }
+
+        unsafe impl MaskElement for $ty {}
     }
 }
 

From 772bf2090e04db68790549bae36e5180423f9f65 Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Thu, 21 Oct 2021 00:23:21 +0000
Subject: [PATCH 241/249] Hide select impl in sealed trait

---
 crates/core_simd/src/select.rs | 36 ++++++++++++++++------------------
 1 file changed, 17 insertions(+), 19 deletions(-)

diff --git a/crates/core_simd/src/select.rs b/crates/core_simd/src/select.rs
index c3d69a83088b..d976231a03a8 100644
--- a/crates/core_simd/src/select.rs
+++ b/crates/core_simd/src/select.rs
@@ -2,24 +2,16 @@ use crate::simd::intrinsics;
 use crate::simd::{LaneCount, Mask, MaskElement, Simd, SimdElement, SupportedLaneCount};
 
 mod sealed {
-    pub trait Sealed {}
+    pub trait Sealed<Mask> {
+        fn select(mask: Mask, true_values: Self, false_values: Self) -> Self;
+    }
 }
 use sealed::Sealed;
 
 /// Supporting trait for vector `select` function
-pub trait Select<Mask>: Sealed {
-    #[doc(hidden)]
-    fn select(mask: Mask, true_values: Self, false_values: Self) -> Self;
-}
+pub trait Select<Mask>: Sealed<Mask> {}
 
-impl<T, const LANES: usize> Sealed for Simd<T, LANES>
-where
-    T: SimdElement,
-    LaneCount<LANES>: SupportedLaneCount,
-{
-}
-
-impl<T, const LANES: usize> Select<Mask<T::Mask, LANES>> for Simd<T, LANES>
+impl<T, const LANES: usize> Sealed<Mask<T::Mask, LANES>> for Simd<T, LANES>
 where
     T: SimdElement,
     LaneCount<LANES>: SupportedLaneCount,
@@ -30,11 +22,22 @@ where
     }
 }
 
-impl<T, const LANES: usize> Sealed for Mask<T, LANES>
+impl<T, const LANES: usize> Select<Mask<T::Mask, LANES>> for Simd<T, LANES>
+where
+    T: SimdElement,
+    LaneCount<LANES>: SupportedLaneCount,
+{
+}
+
+impl<T, const LANES: usize> Sealed<Self> for Mask<T, LANES>
 where
     T: MaskElement,
     LaneCount<LANES>: SupportedLaneCount,
 {
+    #[inline]
+    fn select(mask: Self, true_values: Self, false_values: Self) -> Self {
+        mask & true_values | !mask & false_values
+    }
 }
 
 impl<T, const LANES: usize> Select<Self> for Mask<T, LANES>
@@ -42,11 +45,6 @@ where
     T: MaskElement,
     LaneCount<LANES>: SupportedLaneCount,
 {
-    #[doc(hidden)]
-    #[inline]
-    fn select(mask: Self, true_values: Self, false_values: Self) -> Self {
-        mask & true_values | !mask & false_values
-    }
 }
 
 impl<T, const LANES: usize> Mask<T, LANES>

From 4e00aa68c718427a9dfd8e934529ad4615b1ab7c Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Sat, 6 Nov 2021 00:34:23 +0000
Subject: [PATCH 242/249] rotate_{left,right} -> rotate_lanes_{left,right}

---
 crates/core_simd/examples/matrix_inversion.rs | 38 +++++++++----------
 crates/core_simd/src/swizzle.rs               | 12 +++---
 crates/core_simd/tests/swizzle.rs             | 24 ++++++------
 3 files changed, 37 insertions(+), 37 deletions(-)

diff --git a/crates/core_simd/examples/matrix_inversion.rs b/crates/core_simd/examples/matrix_inversion.rs
index fcee1b96ae12..c51a566deb59 100644
--- a/crates/core_simd/examples/matrix_inversion.rs
+++ b/crates/core_simd/examples/matrix_inversion.rs
@@ -180,58 +180,58 @@ pub fn simd_inv4x4(m: Matrix4x4) -> Option<Matrix4x4> {
     let row2 = simd_swizzle!(tmp, row3, SHUFFLE02);
     let row3 = simd_swizzle!(row3, tmp, SHUFFLE13);
 
-    let tmp = (row2 * row3).reverse().rotate_right::<2>();
+    let tmp = (row2 * row3).reverse().rotate_lanes_right::<2>();
     let minor0 = row1 * tmp;
     let minor1 = row0 * tmp;
-    let tmp = tmp.rotate_right::<2>();
+    let tmp = tmp.rotate_lanes_right::<2>();
     let minor0 = (row1 * tmp) - minor0;
     let minor1 = (row0 * tmp) - minor1;
-    let minor1 = minor1.rotate_right::<2>();
+    let minor1 = minor1.rotate_lanes_right::<2>();
 
-    let tmp = (row1 * row2).reverse().rotate_right::<2>();
+    let tmp = (row1 * row2).reverse().rotate_lanes_right::<2>();
     let minor0 = (row3 * tmp) + minor0;
     let minor3 = row0 * tmp;
-    let tmp = tmp.rotate_right::<2>();
+    let tmp = tmp.rotate_lanes_right::<2>();
 
     let minor0 = minor0 - row3 * tmp;
     let minor3 = row0 * tmp - minor3;
-    let minor3 = minor3.rotate_right::<2>();
+    let minor3 = minor3.rotate_lanes_right::<2>();
 
-    let tmp = (row3 * row1.rotate_right::<2>())
+    let tmp = (row3 * row1.rotate_lanes_right::<2>())
         .reverse()
-        .rotate_right::<2>();
-    let row2 = row2.rotate_right::<2>();
+        .rotate_lanes_right::<2>();
+    let row2 = row2.rotate_lanes_right::<2>();
     let minor0 = row2 * tmp + minor0;
     let minor2 = row0 * tmp;
-    let tmp = tmp.rotate_right::<2>();
+    let tmp = tmp.rotate_lanes_right::<2>();
     let minor0 = minor0 - row2 * tmp;
     let minor2 = row0 * tmp - minor2;
-    let minor2 = minor2.rotate_right::<2>();
+    let minor2 = minor2.rotate_lanes_right::<2>();
 
-    let tmp = (row0 * row1).reverse().rotate_right::<2>();
+    let tmp = (row0 * row1).reverse().rotate_lanes_right::<2>();
     let minor2 = minor2 + row3 * tmp;
     let minor3 = row2 * tmp - minor3;
-    let tmp = tmp.rotate_right::<2>();
+    let tmp = tmp.rotate_lanes_right::<2>();
     let minor2 = row3 * tmp - minor2;
     let minor3 = minor3 - row2 * tmp;
 
-    let tmp = (row0 * row3).reverse().rotate_right::<2>();
+    let tmp = (row0 * row3).reverse().rotate_lanes_right::<2>();
     let minor1 = minor1 - row2 * tmp;
     let minor2 = row1 * tmp + minor2;
-    let tmp = tmp.rotate_right::<2>();
+    let tmp = tmp.rotate_lanes_right::<2>();
     let minor1 = row2 * tmp + minor1;
     let minor2 = minor2 - row1 * tmp;
 
-    let tmp = (row0 * row2).reverse().rotate_right::<2>();
+    let tmp = (row0 * row2).reverse().rotate_lanes_right::<2>();
     let minor1 = row3 * tmp + minor1;
     let minor3 = minor3 - row1 * tmp;
-    let tmp = tmp.rotate_right::<2>();
+    let tmp = tmp.rotate_lanes_right::<2>();
     let minor1 = minor1 - row3 * tmp;
     let minor3 = row1 * tmp + minor3;
 
     let det = row0 * minor0;
-    let det = det.rotate_right::<2>() + det;
-    let det = det.reverse().rotate_right::<2>() + det;
+    let det = det.rotate_lanes_right::<2>() + det;
+    let det = det.reverse().rotate_lanes_right::<2>() + det;
 
     if det.horizontal_sum() == 0. {
         return None;
diff --git a/crates/core_simd/src/swizzle.rs b/crates/core_simd/src/swizzle.rs
index 88e7f3b223ec..62cda68f0a94 100644
--- a/crates/core_simd/src/swizzle.rs
+++ b/crates/core_simd/src/swizzle.rs
@@ -203,10 +203,10 @@ where
     }
 
     /// Rotates the vector such that the first `OFFSET` elements of the slice move to the end
-    /// while the last `LANES - OFFSET` elements move to the front. After calling `rotate_left`, the
-    /// element previously in lane `OFFSET` will become the first element in the slice.
+    /// while the last `LANES - OFFSET` elements move to the front. After calling `rotate_lanes_left`,
+    /// the element previously in lane `OFFSET` will become the first element in the slice.
     #[inline]
-    pub fn rotate_left<const OFFSET: usize>(self) -> Self {
+    pub fn rotate_lanes_left<const OFFSET: usize>(self) -> Self {
         const fn rotate_index<const OFFSET: usize, const LANES: usize>() -> [usize; LANES] {
             let offset = OFFSET % LANES;
             let mut index = [0; LANES];
@@ -228,10 +228,10 @@ where
     }
 
     /// Rotates the vector such that the first `LANES - OFFSET` elements of the vector move to
-    /// the end while the last `OFFSET` elements move to the front. After calling `rotate_right`, the
-    /// element previously at index `LANES - OFFSET` will become the first element in the slice.
+    /// the end while the last `OFFSET` elements move to the front. After calling `rotate_lanes_right`,
+    /// the element previously at index `LANES - OFFSET` will become the first element in the slice.
     #[inline]
-    pub fn rotate_right<const OFFSET: usize>(self) -> Self {
+    pub fn rotate_lanes_right<const OFFSET: usize>(self) -> Self {
         const fn rotate_index<const OFFSET: usize, const LANES: usize>() -> [usize; LANES] {
             let offset = LANES - OFFSET % LANES;
             let mut index = [0; LANES];
diff --git a/crates/core_simd/tests/swizzle.rs b/crates/core_simd/tests/swizzle.rs
index d4abc46b9327..51c63611aba6 100644
--- a/crates/core_simd/tests/swizzle.rs
+++ b/crates/core_simd/tests/swizzle.rs
@@ -34,18 +34,18 @@ fn reverse() {
 #[cfg_attr(target_arch = "wasm32", wasm_bindgen_test)]
 fn rotate() {
     let a = Simd::from_array([1, 2, 3, 4]);
-    assert_eq!(a.rotate_left::<0>().to_array(), [1, 2, 3, 4]);
-    assert_eq!(a.rotate_left::<1>().to_array(), [2, 3, 4, 1]);
-    assert_eq!(a.rotate_left::<2>().to_array(), [3, 4, 1, 2]);
-    assert_eq!(a.rotate_left::<3>().to_array(), [4, 1, 2, 3]);
-    assert_eq!(a.rotate_left::<4>().to_array(), [1, 2, 3, 4]);
-    assert_eq!(a.rotate_left::<5>().to_array(), [2, 3, 4, 1]);
-    assert_eq!(a.rotate_right::<0>().to_array(), [1, 2, 3, 4]);
-    assert_eq!(a.rotate_right::<1>().to_array(), [4, 1, 2, 3]);
-    assert_eq!(a.rotate_right::<2>().to_array(), [3, 4, 1, 2]);
-    assert_eq!(a.rotate_right::<3>().to_array(), [2, 3, 4, 1]);
-    assert_eq!(a.rotate_right::<4>().to_array(), [1, 2, 3, 4]);
-    assert_eq!(a.rotate_right::<5>().to_array(), [4, 1, 2, 3]);
+    assert_eq!(a.rotate_lanes_left::<0>().to_array(), [1, 2, 3, 4]);
+    assert_eq!(a.rotate_lanes_left::<1>().to_array(), [2, 3, 4, 1]);
+    assert_eq!(a.rotate_lanes_left::<2>().to_array(), [3, 4, 1, 2]);
+    assert_eq!(a.rotate_lanes_left::<3>().to_array(), [4, 1, 2, 3]);
+    assert_eq!(a.rotate_lanes_left::<4>().to_array(), [1, 2, 3, 4]);
+    assert_eq!(a.rotate_lanes_left::<5>().to_array(), [2, 3, 4, 1]);
+    assert_eq!(a.rotate_lanes_right::<0>().to_array(), [1, 2, 3, 4]);
+    assert_eq!(a.rotate_lanes_right::<1>().to_array(), [4, 1, 2, 3]);
+    assert_eq!(a.rotate_lanes_right::<2>().to_array(), [3, 4, 1, 2]);
+    assert_eq!(a.rotate_lanes_right::<3>().to_array(), [2, 3, 4, 1]);
+    assert_eq!(a.rotate_lanes_right::<4>().to_array(), [1, 2, 3, 4]);
+    assert_eq!(a.rotate_lanes_right::<5>().to_array(), [4, 1, 2, 3]);
 }
 
 #[test]

From d2e87281fcffbf26635c03a1060ca3fc18dcf418 Mon Sep 17 00:00:00 2001
From: Proloy Mishra <67726964+pro465@users.noreply.github.com>
Date: Tue, 9 Nov 2021 06:58:43 +0530
Subject: [PATCH 243/249] add `Simd::from_slice` (#177)

* add `Simd::from_slice`

uses a zeroed initial array and loops so that it can be const.
unfortunately, parameterizing the assert with slice length
needs `#![feature(const_fn_fn_ptr_basics)]` to work.
---
 crates/core_simd/src/vector.rs | 18 ++++++++++++++++++
 1 file changed, 18 insertions(+)

diff --git a/crates/core_simd/src/vector.rs b/crates/core_simd/src/vector.rs
index 893eff674ffb..7c5ec2bc314c 100644
--- a/crates/core_simd/src/vector.rs
+++ b/crates/core_simd/src/vector.rs
@@ -57,6 +57,24 @@ where
         self.0
     }
 
+    /// Converts a slice to a SIMD vector containing `slice[..LANES]`
+    /// # Panics
+    /// `from_slice` will panic if the slice's `len` is less than the vector's `Simd::LANES`.
+    #[must_use]
+    pub const fn from_slice(slice: &[T]) -> Self {
+        assert!(
+            slice.len() >= LANES,
+            "slice length must be at least the number of lanes"
+        );
+        let mut array = [slice[0]; LANES];
+        let mut i = 0;
+        while i < LANES {
+            array[i] = slice[i];
+            i += 1;
+        }
+        Self(array)
+    }
+
     /// Reads from potentially discontiguous indices in `slice` to construct a SIMD vector.
     /// If an index is out-of-bounds, the lane is instead selected from the `or` vector.
     ///

From 349a61143c81f7e15baf07f457938f469d4b9e7b Mon Sep 17 00:00:00 2001
From: Caleb Zulawski <caleb.zulawski@gmail.com>
Date: Mon, 18 Oct 2021 23:20:31 +0000
Subject: [PATCH 244/249] Delete travis config, move tests to github actions.

---
 .github/workflows/ci.yml                 | 35 +++++++++-
 .travis.yml                              | 82 ------------------------
 crates/core_simd/src/masks/full_masks.rs |  4 +-
 crates/core_simd/tests/masks.rs          |  4 ++
 4 files changed, 39 insertions(+), 86 deletions(-)
 delete mode 100644 .travis.yml

diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
index a9768f53852c..3a2c4477f765 100644
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -94,8 +94,7 @@ jobs:
           - { target: i586-pc-windows-msvc, target_feature: +sse2, os: windows-latest }
 
           # Annoyingly, the x86_64-unknown-linux-gnu runner *almost* always has
-          # avx512vl, but occasionally doesn't. As a result, we still run that
-          # one under travis.
+          # avx512vl, but occasionally doesn't.  Maybe one day we can enable it.
 
     steps:
       - uses: actions/checkout@v2
@@ -141,6 +140,31 @@ jobs:
       - name: Test (release)
         run: cargo test --verbose --target=${{ matrix.target }} --release
 
+  wasm-tests:
+    name: "wasm (firefox, ${{ matrix.name }})"
+    runs-on: ubuntu-latest
+    strategy:
+      matrix:
+        include:
+          - { name: default, RUSTFLAGS: "" }
+          - { name: simd128, RUSTFLAGS: "-C target-feature=+simd128" }
+    steps:
+      - uses: actions/checkout@v2
+      - name: Setup Rust
+        run: |
+          rustup update nightly --no-self-update
+          rustup default nightly
+      - name: Install wasm-pack
+        run: curl https://rustwasm.github.io/wasm-pack/installer/init.sh -sSf | sh
+      - name: Test (debug)
+        run: wasm-pack test --firefox --headless crates/core_simd
+        env:
+            RUSTFLAGS: ${{ matrix.rustflags }}
+      - name: Test (release)
+        run: wasm-pack test --firefox --headless crates/core_simd --release
+        env:
+            RUSTFLAGS: ${{ matrix.rustflags }}
+
   cross-tests:
     name: "${{ matrix.target }} (via cross)"
     runs-on: ubuntu-latest
@@ -163,12 +187,19 @@ jobs:
           # 32-bit arm has a few idiosyncracies like having subnormal flushing
           # to zero on by default. Ideally we'd set
           - armv7-unknown-linux-gnueabihf
+          - aarch64-unknown-linux-gnu
           # Note: The issue above means neither of these mips targets will use
           # MSA (mips simd) but MIPS uses a nonstandard binary representation
           # for NaNs which makes it worth testing on despite that.
           - mips-unknown-linux-gnu
           - mips64-unknown-linux-gnuabi64
           - riscv64gc-unknown-linux-gnu
+          # TODO this test works, but it appears to time out
+          # - powerpc-unknown-linux-gnu
+          # TODO this test is broken, but it appears to be a problem with QEMU, not us.
+          # - powerpc64le-unknown-linux-gnu
+          # TODO enable this once a new version of cross is released
+          # - powerpc64-unknown-linux-gnu
 
     steps:
       - uses: actions/checkout@v2
diff --git a/.travis.yml b/.travis.yml
deleted file mode 100644
index 6b284c87ecbe..000000000000
--- a/.travis.yml
+++ /dev/null
@@ -1,82 +0,0 @@
-branches:
-  only:
-    - master
-
-language: rust
-rust:
-  - nightly
-
-matrix:
-  fast_finish: true
-  include:
-    # Linux (aarch64)
-    - name: "aarch64-unknown-linux-gnu (neon)"
-      os: linux
-      arch: arm64
-
-    - name: "aarch64-unknown-linux-gnu (neon, sve)"
-      os: linux
-      arch: arm64
-      env: RUSTFLAGS=-Ctarget-feature=+sve
-
-    - name: "aarch64-unknown-linux-gnu (native, see log for cfg)"
-      os: linux
-      arch: arm64
-      env: RUSTFLAGS=-Ctarget-cpu=native
-
-    # Linux (powerpc64le)
-    - name: "powerpc64le-unknown-linux-gnu (altivec, vsx, power8-*)"
-      os: linux
-      arch: ppc64le
-
-    - name: "powerpc64le-unknown-linux-gnu (native, see log for cfg)"
-      os: linux
-      arch: ppc64le
-      env: RUSTFLAGS=-Ctarget-cpu=native
-
-    # Linux (x86_64) (for AVX512, which sadly seems to only *usually* be present
-    # on the github actions linux runner...)
-    - name: "x86_64-unknown-linux-gnu+avx512vl"
-      os: linux
-      arch: amd64
-      env: RUSTFLAGS=-Ctarget-feature=+avx512vl
-
-    # WebAssembly (wasm-bindgen)
-    - name: "wasm32-unknown-unknown (firefox)"
-      os: linux
-      arch: amd64
-      addons:
-        firefox: latest-nightly
-      install:
-        - curl https://rustwasm.github.io/wasm-pack/installer/init.sh -sSf | sh
-      script:
-        - wasm-pack test --firefox --headless crates/core_simd
-        - wasm-pack test --firefox --headless crates/core_simd --release
-
-    # FIXME: See https://github.com/rust-lang/stdsimd/issues/92
-    # - name: "wasm32-unknown-unknown+simd128 (firefox)"
-    #   os: linux
-    #   arch: amd64
-    #   addons:
-    #     firefox: latest-nightly
-    #   install:
-    #     - curl https://rustwasm.github.io/wasm-pack/installer/init.sh -sSf | sh
-    #   script:
-    #     - export RUSTFLAGS="-C target-feature=+simd128"
-    #     - wasm-pack test --firefox --headless crates/core_simd
-    #     - wasm-pack test --firefox --headless crates/core_simd --release
-
-script:
-  - echo "## Requested target configuration (RUSTFLAGS=$RUSTFLAGS)"
-  - rustc --print=cfg $RUSTFLAGS
-
-  - echo "## Supported target configuration"
-  - rustc --print=cfg -Ctarget-cpu=native
-
-  - echo "\n---\n"
-
-  - echo "## Running tests (debug)"
-  - cargo test -v
-
-  - echo "## Running tests (release)"
-  - cargo test -v --release
diff --git a/crates/core_simd/src/masks/full_masks.rs b/crates/core_simd/src/masks/full_masks.rs
index b653bce05b9e..dd981cedb932 100644
--- a/crates/core_simd/src/masks/full_masks.rs
+++ b/crates/core_simd/src/masks/full_masks.rs
@@ -119,7 +119,7 @@ where
             // There is a bug where LLVM appears to implement this operation with the wrong
             // bit order.
             // TODO fix this in a better way
-            if cfg!(any(target_arch = "mips", target_arch = "mips64")) {
+            if cfg!(target_endian = "big") {
                 for x in bitmask.as_mut() {
                     *x = x.reverse_bits();
                 }
@@ -136,7 +136,7 @@ where
             // There is a bug where LLVM appears to implement this operation with the wrong
             // bit order.
             // TODO fix this in a better way
-            if cfg!(any(target_arch = "mips", target_arch = "mips64")) {
+            if cfg!(target_endian = "big") {
                 for x in bitmask.as_mut() {
                     *x = x.reverse_bits();
                 }
diff --git a/crates/core_simd/tests/masks.rs b/crates/core_simd/tests/masks.rs
index c2d400d79d49..6a8ecd33a73c 100644
--- a/crates/core_simd/tests/masks.rs
+++ b/crates/core_simd/tests/masks.rs
@@ -86,6 +86,10 @@ macro_rules! test_mask_api {
 
 mod mask_api {
     test_mask_api! { i8 }
+    test_mask_api! { i16 }
+    test_mask_api! { i32 }
+    test_mask_api! { i64 }
+    test_mask_api! { isize }
 }
 
 #[test]

From c52083e25650d517435a100fc81f60e8a60d9d5b Mon Sep 17 00:00:00 2001
From: Jubilee Young <workingjubilee@gmail.com>
Date: Tue, 9 Nov 2021 21:04:18 -0800
Subject: [PATCH 245/249] Use the right name for AVX512F

---
 .github/workflows/ci.yml | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
index 3a2c4477f765..25023688a22d 100644
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -235,7 +235,7 @@ jobs:
       matrix:
         rustflags:
           - ""
-          - "-Ctarget-feature=+avx512" # AVX-512 uses packed bit masks, so enable it to test more code paths
+          - "-Ctarget-feature=+avx512f" # AVX-512 uses packed bit masks, so enable it to test more code paths
         features:
           - ""
           - "--features std"

From 949f71c0dc8716e285c164eae225db2d18333f0d Mon Sep 17 00:00:00 2001
From: Jubilee Young <workingjubilee@gmail.com>
Date: Tue, 9 Nov 2021 21:06:38 -0800
Subject: [PATCH 246/249] Deny warnings in CI and fix

---
 .github/workflows/ci.yml           | 8 ++++----
 crates/core_simd/src/intrinsics.rs | 3 +++
 crates/core_simd/src/vendor.rs     | 1 +
 3 files changed, 8 insertions(+), 4 deletions(-)

diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
index 25023688a22d..90007a2f8f64 100644
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -109,12 +109,12 @@ jobs:
         run: |
           case "${{ matrix.target_feature }}" in
             default)
-              ;;
+              echo "RUSTFLAGS=-Dwarnings" >> $GITHUB_ENV;;
             native)
-              echo "RUSTFLAGS=-Ctarget-cpu=native" >> $GITHUB_ENV
+              echo "RUSTFLAGS=-Dwarnings -Ctarget-cpu=native" >> $GITHUB_ENV
               ;;
             *)
-              echo "RUSTFLAGS=-Ctarget-feature=${{ matrix.target_feature }}" >> $GITHUB_ENV
+              echo "RUSTFLAGS=-Dwarnings -Ctarget-feature=${{ matrix.target_feature }}" >> $GITHUB_ENV
               ;;
           esac
 
@@ -251,4 +251,4 @@ jobs:
       - name: Check build
         run: cargo check --all-targets --no-default-features ${{ matrix.features }}
         env:
-            RUSTFLAGS: ${{ matrix.rustflags }}
+            RUSTFLAGS: -Dwarnings ${{ matrix.rustflags }}
diff --git a/crates/core_simd/src/intrinsics.rs b/crates/core_simd/src/intrinsics.rs
index 5f55cdf0399e..6a6d26d10a7f 100644
--- a/crates/core_simd/src/intrinsics.rs
+++ b/crates/core_simd/src/intrinsics.rs
@@ -68,7 +68,9 @@ extern "platform-intrinsic" {
     // reductions
     pub(crate) fn simd_reduce_add_ordered<T, U>(x: T, y: U) -> U;
     pub(crate) fn simd_reduce_mul_ordered<T, U>(x: T, y: U) -> U;
+    #[allow(unused)]
     pub(crate) fn simd_reduce_all<T>(x: T) -> bool;
+    #[allow(unused)]
     pub(crate) fn simd_reduce_any<T>(x: T) -> bool;
     pub(crate) fn simd_reduce_max<T, U>(x: T) -> U;
     pub(crate) fn simd_reduce_min<T, U>(x: T) -> U;
@@ -77,6 +79,7 @@ extern "platform-intrinsic" {
     pub(crate) fn simd_reduce_xor<T, U>(x: T) -> U;
 
     // truncate integer vector to bitmask
+    #[allow(unused)]
     pub(crate) fn simd_bitmask<T, U>(x: T) -> U;
 
     // select
diff --git a/crates/core_simd/src/vendor.rs b/crates/core_simd/src/vendor.rs
index 8c8af43bf133..bdb9d45eb9d7 100644
--- a/crates/core_simd/src/vendor.rs
+++ b/crates/core_simd/src/vendor.rs
@@ -1,4 +1,5 @@
 /// Provides implementations of `From<$a> for $b` and `From<$b> for $a` that transmutes the value.
+#[allow(unused)]
 macro_rules! from_transmute {
     { unsafe $a:ty => $b:ty } => {
         from_transmute!{ @impl $a => $b }

From 7d91357875da59d52284d506dcb457f7f88bf6bf Mon Sep 17 00:00:00 2001
From: Jubilee Young <workingjubilee@gmail.com>
Date: Tue, 9 Nov 2021 23:49:16 -0800
Subject: [PATCH 247/249] Dynamically detect AVX512 in CI

We would like to check for errors with AVX512,
but we don't pick our CPU. So, detect available features.
This variance in checks stochastically reveals issues.
Nondeterminism is acceptable as our goal is protecting downstream.
---
 .github/workflows/ci.yml | 18 ++++++++++++------
 1 file changed, 12 insertions(+), 6 deletions(-)

diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
index 90007a2f8f64..d50dfa1be4cb 100644
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -228,14 +228,14 @@ jobs:
         run: cross test --verbose --target=${{ matrix.target }} --release
 
   features:
-    name: "Check cargo features (${{ matrix.features }} ${{ matrix.rustflags }})"
+    name: "Check cargo features (${{ matrix.simd }} × ${{ matrix.features }})"
     runs-on: ubuntu-latest
     strategy:
       fail-fast: false
       matrix:
-        rustflags:
+        simd:
           - ""
-          - "-Ctarget-feature=+avx512f" # AVX-512 uses packed bit masks, so enable it to test more code paths
+          - "avx512"
         features:
           - ""
           - "--features std"
@@ -248,7 +248,13 @@ jobs:
         run: |
           rustup update nightly --no-self-update
           rustup default nightly
+      - name: Detect AVX512
+        run: echo "CPU_FEATURE=$(lscpu | grep -o avx512[a-z]* | sed s/avx/+avx/ | tr '\n' ',' )" >> $GITHUB_ENV
       - name: Check build
-        run: cargo check --all-targets --no-default-features ${{ matrix.features }}
-        env:
-            RUSTFLAGS: -Dwarnings ${{ matrix.rustflags }}
+        if: ${{ matrix.simd == '' }}
+        run: RUSTFLAGS="-Dwarnings" cargo check --all-targets --no-default-features ${{ matrix.features }}
+      - name: Check AVX
+        if: ${{ matrix.simd == 'avx512' && contains(env.CPU_FEATURE, 'avx512') }}
+        run: |
+          echo "Found AVX features: $CPU_FEATURE"
+          RUSTFLAGS="-Dwarnings -Ctarget-feature=$CPU_FEATURE" cargo check --all-targets --no-default-features ${{ matrix.features }}

From 6ddf7ad8e19dfc3c73501bc4dbe066e46ced0f36 Mon Sep 17 00:00:00 2001
From: Jubilee Young <workingjubilee@gmail.com>
Date: Thu, 11 Nov 2021 10:54:27 -0800
Subject: [PATCH 248/249] Restrict Arm types to Arm v7+

This mostly mirrors the restrictions in std::arch.
It can be loosened slightly with later refactoring.
---
 crates/core_simd/src/vendor.rs     | 5 ++++-
 crates/core_simd/src/vendor/arm.rs | 4 ++--
 2 files changed, 6 insertions(+), 3 deletions(-)

diff --git a/crates/core_simd/src/vendor.rs b/crates/core_simd/src/vendor.rs
index bdb9d45eb9d7..5f6a480f081c 100644
--- a/crates/core_simd/src/vendor.rs
+++ b/crates/core_simd/src/vendor.rs
@@ -21,7 +21,10 @@ mod x86;
 #[cfg(any(target_arch = "wasm32"))]
 mod wasm32;
 
-#[cfg(any(target_arch = "arm", target_arch = "aarch64"))]
+#[cfg(any(
+    target_arch = "aarch64",
+    all(target_arch = "arm", target_feature = "v7")
+))]
 mod arm;
 
 #[cfg(any(target_arch = "powerpc", target_arch = "powerpc64"))]
diff --git a/crates/core_simd/src/vendor/arm.rs b/crates/core_simd/src/vendor/arm.rs
index 3e9487dfb33d..720c84cdd887 100644
--- a/crates/core_simd/src/vendor/arm.rs
+++ b/crates/core_simd/src/vendor/arm.rs
@@ -1,6 +1,6 @@
 use crate::simd::*;
 
-#[cfg(target_arch = "arm")]
+#[cfg(all(target_arch = "arm", target_feature = "v7"))]
 use core::arch::arm::*;
 
 #[cfg(target_arch = "aarch64")]
@@ -35,7 +35,7 @@ from_transmute! { unsafe i64x2 => int64x2_t }
 from_transmute! { unsafe Simd<u64, 1> => poly64x1_t }
 from_transmute! { unsafe u64x2 => poly64x2_t }
 
-#[cfg(target_arch = "arm")]
+#[cfg(all(target_arch = "arm", target_feature = "v7"))]
 mod arm {
     use super::*;
     from_transmute! { unsafe Simd<u8, 4> => uint8x4_t }

From 1ce1c645cf27c4acdefe6ec8a11d1f0491954a99 Mon Sep 17 00:00:00 2001
From: Jubilee Young <workingjubilee@gmail.com>
Date: Fri, 12 Nov 2021 16:42:48 -0800
Subject: [PATCH 249/249] Rewrite Arm transmutes, reading std::arch closer

---
 crates/core_simd/src/vendor.rs     |  6 +-
 crates/core_simd/src/vendor/arm.rs | 91 +++++++++++++++++++-----------
 2 files changed, 59 insertions(+), 38 deletions(-)

diff --git a/crates/core_simd/src/vendor.rs b/crates/core_simd/src/vendor.rs
index 5f6a480f081c..e8ce7176b4f2 100644
--- a/crates/core_simd/src/vendor.rs
+++ b/crates/core_simd/src/vendor.rs
@@ -15,16 +15,14 @@ macro_rules! from_transmute {
     };
 }
 
+/// Conversions to x86's SIMD types.
 #[cfg(any(target_arch = "x86", target_arch = "x86_64"))]
 mod x86;
 
 #[cfg(any(target_arch = "wasm32"))]
 mod wasm32;
 
-#[cfg(any(
-    target_arch = "aarch64",
-    all(target_arch = "arm", target_feature = "v7")
-))]
+#[cfg(any(target_arch = "aarch64", target_arch = "arm",))]
 mod arm;
 
 #[cfg(any(target_arch = "powerpc", target_arch = "powerpc64"))]
diff --git a/crates/core_simd/src/vendor/arm.rs b/crates/core_simd/src/vendor/arm.rs
index 720c84cdd887..ff3b69ccf959 100644
--- a/crates/core_simd/src/vendor/arm.rs
+++ b/crates/core_simd/src/vendor/arm.rs
@@ -1,53 +1,76 @@
+#![allow(unused)]
 use crate::simd::*;
 
-#[cfg(all(target_arch = "arm", target_feature = "v7"))]
+#[cfg(target_arch = "arm")]
 use core::arch::arm::*;
 
 #[cfg(target_arch = "aarch64")]
 use core::arch::aarch64::*;
 
-from_transmute! { unsafe f32x2 => float32x2_t }
-from_transmute! { unsafe f32x4 => float32x4_t }
-
-from_transmute! { unsafe u8x8 => uint8x8_t }
-from_transmute! { unsafe u8x16 => uint8x16_t }
-from_transmute! { unsafe i8x8 => int8x8_t }
-from_transmute! { unsafe i8x16 => int8x16_t }
-from_transmute! { unsafe u8x8 => poly8x8_t }
-from_transmute! { unsafe u8x16 => poly8x16_t }
-
-from_transmute! { unsafe u16x4 => uint16x4_t }
-from_transmute! { unsafe u16x8 => uint16x8_t }
-from_transmute! { unsafe i16x4 => int16x4_t }
-from_transmute! { unsafe i16x8 => int16x8_t }
-from_transmute! { unsafe u16x4 => poly16x4_t }
-from_transmute! { unsafe u16x8 => poly16x8_t }
-
-from_transmute! { unsafe u32x2 => uint32x2_t }
-from_transmute! { unsafe u32x4 => uint32x4_t }
-from_transmute! { unsafe i32x2 => int32x2_t }
-from_transmute! { unsafe i32x4 => int32x4_t }
-
-from_transmute! { unsafe Simd<u64, 1> => uint64x1_t }
-from_transmute! { unsafe u64x2 => uint64x2_t }
-from_transmute! { unsafe Simd<i64, 1> => int64x1_t }
-from_transmute! { unsafe i64x2 => int64x2_t }
-from_transmute! { unsafe Simd<u64, 1> => poly64x1_t }
-from_transmute! { unsafe u64x2 => poly64x2_t }
-
-#[cfg(all(target_arch = "arm", target_feature = "v7"))]
-mod arm {
+#[cfg(any(
+    target_arch = "aarch64",
+    all(target_arch = "arm", target_feature = "v7"),
+))]
+mod neon {
+    use super::*;
+
+    from_transmute! { unsafe f32x2 => float32x2_t }
+    from_transmute! { unsafe f32x4 => float32x4_t }
+
+    from_transmute! { unsafe u8x8 => uint8x8_t }
+    from_transmute! { unsafe u8x16 => uint8x16_t }
+    from_transmute! { unsafe i8x8 => int8x8_t }
+    from_transmute! { unsafe i8x16 => int8x16_t }
+    from_transmute! { unsafe u8x8 => poly8x8_t }
+    from_transmute! { unsafe u8x16 => poly8x16_t }
+
+    from_transmute! { unsafe u16x4 => uint16x4_t }
+    from_transmute! { unsafe u16x8 => uint16x8_t }
+    from_transmute! { unsafe i16x4 => int16x4_t }
+    from_transmute! { unsafe i16x8 => int16x8_t }
+    from_transmute! { unsafe u16x4 => poly16x4_t }
+    from_transmute! { unsafe u16x8 => poly16x8_t }
+
+    from_transmute! { unsafe u32x2 => uint32x2_t }
+    from_transmute! { unsafe u32x4 => uint32x4_t }
+    from_transmute! { unsafe i32x2 => int32x2_t }
+    from_transmute! { unsafe i32x4 => int32x4_t }
+
+    from_transmute! { unsafe Simd<u64, 1> => uint64x1_t }
+    from_transmute! { unsafe u64x2 => uint64x2_t }
+    from_transmute! { unsafe Simd<i64, 1> => int64x1_t }
+    from_transmute! { unsafe i64x2 => int64x2_t }
+    from_transmute! { unsafe Simd<u64, 1> => poly64x1_t }
+    from_transmute! { unsafe u64x2 => poly64x2_t }
+}
+
+#[cfg(any(
+    all(target_feature = "v5te", not(target_feature = "mclass")),
+    all(target_feature = "mclass", target_feature = "dsp"),
+))]
+mod dsp {
     use super::*;
-    from_transmute! { unsafe Simd<u8, 4> => uint8x4_t }
-    from_transmute! { unsafe Simd<i8, 4> => int8x4_t }
 
     from_transmute! { unsafe Simd<u16, 2> => uint16x2_t }
     from_transmute! { unsafe Simd<i16, 2> => int16x2_t }
 }
 
+#[cfg(any(
+    all(target_feature = "v6", not(target_feature = "mclass")),
+    all(target_feature = "mclass", target_feature = "dsp"),
+))]
+mod simd32 {
+    use super::*;
+
+    from_transmute! { unsafe Simd<u8, 4> => uint8x4_t }
+    from_transmute! { unsafe Simd<i8, 4> => int8x4_t }
+}
+
 #[cfg(target_arch = "aarch64")]
 mod aarch64 {
+    use super::neon::*;
     use super::*;
+
     from_transmute! { unsafe Simd<f64, 1> => float64x1_t }
     from_transmute! { unsafe f64x2 => float64x2_t }
 }