Add reproducible unprivileged Linux build and mount checks

This commit is contained in:
2026-10-09 18:32:37 -04:00
parent 8b177e21a0
commit 988aad9951
11 changed files with 212 additions and 10 deletions
+2
View File
@@ -21,6 +21,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
cooperative cancellation, and per-scan thread pools. cooperative cancellation, and per-scan thread pools.
- Optional `-x` / `--one-file-system` boundary handling. Partial CLI reports - Optional `-x` / `--one-file-system` boundary handling. Partial CLI reports
return a nonzero exit status and include `complete` in JSON output. return a nonzero exit status and include `complete` in JSON output.
- Repeatable Linux ARM64/AMD64 Docker checks with unprivileged permission
tests, a real mounted-filesystem boundary and an optimized-binary smoke test.
## [1.0.0] - 2026-01-22 ## [1.0.0] - 2026-01-22
+23
View File
@@ -149,6 +149,29 @@ for each request. Clones of the control expose progress and cancellation;
cancellation is cooperative between filesystem calls. `build_tree` remains as cancellation is cooperative between filesystem calls. `build_tree` remains as
a convenience helper, while `scan_tree` retains structured diagnostics. a convenience helper, while `scan_tree` retains structured diagnostics.
## Development checks
Run `cargo test --locked` locally as an unprivileged user. For Linux build,
permission, filesystem-boundary and release smoke checks with Docker:
```bash
./scripts/check-linux.sh linux/arm64
./scripts/check-linux.sh linux/amd64
```
The script pins the official Rust 1.88.0 Bookworm image by digest, adds rustfmt
and Clippy, then runs as UID 65532 with dropped capabilities. Source is mounted
read-only and copied into the temporary container. Fixtures live on Linux
filesystems, including two distinct tmpfs mounts; no privileged container or
host directory scan is required. AMD64 on an ARM64 host requires emulation.
Logs are saved in `target/linux-checks/`; containers and their build output are
removed on exit. Docker retains the reusable check images/build cache.
`SIZED_LINUX_JOBS` changes the default two build workers; `SIZED_LINUX_IMAGE`
can select a different toolchain image for an explicit compatibility check.
These are backend checks; desktop X11/Wayland acceptance belongs to SizeQueen.
See [source review and release process](SHIPMENT.md) and the [live queue](TODO.md).
## License ## License
This project is licensed under the **GNU General Public License v3.0 (GPL-3.0)**. This project is licensed under the **GNU General Public License v3.0 (GPL-3.0)**.
+25
View File
@@ -2,6 +2,31 @@
This document defines the process for versioning and distributing the `sized` project independently of the Git hosting provider (GitHub, GitLab, Gitea). This document defines the process for versioning and distributing the `sized` project independently of the Git hosting provider (GitHub, GitLab, Gitea).
## Source review before a release
Use a named branch from `main` and keep a pull request focused on one outcome.
The `sizequeen-scan-hardening` branch corrects accounting/error handling and
adds the scan controls needed by SizeQueen; shared-crate extraction follows
in a separate PR so reviewers can distinguish behaviour changes from packaging.
1. Run local locked tests and strict Clippy as an unprivileged user. Run
`./scripts/check-linux.sh linux/arm64` and
`./scripts/check-linux.sh linux/amd64` for the repeatable Linux checks,
including actual mount boundaries and an optimized-binary smoke test.
2. Push the review branch. Open a PR against `main` when its scope is ready,
explaining changed behaviour, test evidence and remaining limits. For this
branch, call out nonzero status on partial scans, added JSON status fields,
and library API changes. Record current work and evidence in `TODO.md`.
3. Review and merge independently of distribution. A branch push or PR merge
does not publish a package, change repository visibility or create a tag.
4. Choose the release version after reviewing library/API compatibility, then
use the release steps below when explicitly authorized. Reconcile legacy
GitLab download/package links before announcing a Gitea release.
Use the same Linux check script in Gitea CI when a Docker-capable runner is
configured. The current branch provides the local entry point; it does not
configure a runner or enable automatic publishing.
## 1. Versioning and Tagging ## 1. Versioning and Tagging
The project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). The project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+3 -3
View File
@@ -10,12 +10,12 @@ fn setup_test_dir() -> TempDir {
// Create a moderately deep structure // Create a moderately deep structure
for i in 0..10 { for i in 0..10 {
let dir_path = base_path.join(format!("dir_{}", i)); let dir_path = base_path.join(format!("dir_{i}"));
fs::create_dir(&dir_path).unwrap(); fs::create_dir(&dir_path).unwrap();
for j in 0..10 { for j in 0..10 {
let file_path = dir_path.join(format!("file_{}.txt", j)); let file_path = dir_path.join(format!("file_{j}.txt"));
let mut file = File::create(file_path).unwrap(); let mut file = File::create(file_path).unwrap();
writeln!(file, "Some content for file {}-{}", i, j).unwrap(); writeln!(file, "Some content for file {i}-{j}").unwrap();
} }
} }
temp_dir temp_dir
+26
View File
@@ -0,0 +1,26 @@
#!/usr/bin/env bash
set -euo pipefail
test "$(uname -s)" = Linux
test "$(id -u)" != 0
printf 'Linux checks: uid=%s architecture=%s\n' "$(id -u)" "$(uname -m)"
rustc --version
cargo --version
# Only source inputs are copied. Cargo output and all fixtures disappear with
# the container; the host checkout is read-only and no personal tree is scanned.
check_root=$(mktemp -d /tmp/sized-check.XXXXXX)
cp /source/Cargo.toml /source/Cargo.lock "$check_root/"
cp -R /source/src /source/tests /source/benches "$check_root/"
cd "$check_root"
cargo fmt --all -- --check
cargo test --locked
cargo test --locked --test linux_mount -- --ignored
cargo clippy --locked --all-targets -- -D warnings
cargo build --locked --release
fixture_root=$(mktemp -d /tmp/sized-release.XXXXXX)
printf 'Linux release smoke test\n' > "$fixture_root/payload"
./target/release/sized --version
./target/release/sized "$fixture_root" --threads 2 --one-file-system --apparent --format json
printf 'Linux checks passed.\n'
+27
View File
@@ -0,0 +1,27 @@
#!/usr/bin/env bash
set -euo pipefail
# Pin the multi-platform official image, not a moving tag. Tests run on Linux
# filesystems rather than the source bind mount, whose permission semantics vary.
repo_root=$(cd "$(dirname "$0")/.." && pwd)
platform=${1:-linux/$(docker version --format '{{.Server.Arch}}')}
case "$platform" in
linux/arm64|linux/amd64) ;;
*) printf 'Usage: %s [linux/arm64|linux/amd64]\n' "$0" >&2; exit 2 ;;
esac
image=${SIZED_LINUX_IMAGE:-rust:1.88.0-bookworm@sha256:af306cfa71d987911a781c37b59d7d67d934f49684058f96cf72079c3626bfe0}
check_image="sized-linux-check:${platform#linux/}"
mkdir -p "$repo_root/target/linux-checks"
log_file="$repo_root/target/linux-checks/${platform#linux/}.log"
docker build --platform "$platform" --build-arg "BASE_IMAGE=$image" \
--tag "$check_image" - < "$repo_root/scripts/linux-check.Dockerfile"
docker run --rm --platform "$platform" \
--user 65532:65532 --cap-drop ALL --security-opt no-new-privileges \
--mount "type=bind,src=$repo_root,dst=/source,readonly" \
--tmpfs /tmp/sized-mount-test:rw,nosuid,nodev,noexec,size=16m,uid=65532,gid=65532,mode=0700 \
--tmpfs /tmp/sized-mount-test/foreign:rw,nosuid,nodev,noexec,size=16m,uid=65532,gid=65532,mode=0700 \
--env CARGO_HOME=/tmp/sized-cargo \
--env CARGO_BUILD_JOBS="${SIZED_LINUX_JOBS:-2}" \
--env SIZED_TEST_MOUNT_ROOT=/tmp/sized-mount-test \
"$check_image" bash /source/scripts/check-linux-container.sh 2>&1 | tee "$log_file"
+3
View File
@@ -0,0 +1,3 @@
ARG BASE_IMAGE=rust:1.88.0-bookworm@sha256:af306cfa71d987911a781c37b59d7d67d934f49684058f96cf72079c3626bfe0
FROM ${BASE_IMAGE}
RUN rustup component add rustfmt clippy
+5 -5
View File
@@ -131,7 +131,7 @@ impl FromStr for SortColumn {
"size" | "s" | "disk" | "d" => Ok(SortColumn::Disk), "size" | "s" | "disk" | "d" => Ok(SortColumn::Disk),
"apparent" | "a" => Ok(SortColumn::Apparent), "apparent" | "a" => Ok(SortColumn::Apparent),
"blocks" | "b" => Ok(SortColumn::Blocks), "blocks" | "b" => Ok(SortColumn::Blocks),
_ => Err(format!("Unknown column: {}", s)), _ => Err(format!("Unknown column: {s}")),
} }
} }
} }
@@ -148,7 +148,7 @@ impl FromStr for SortDirection {
match s.to_lowercase().as_str() { match s.to_lowercase().as_str() {
"asc" | "a" => Ok(SortDirection::Asc), "asc" | "a" => Ok(SortDirection::Asc),
"dsc" | "d" | "desc" => Ok(SortDirection::Dsc), "dsc" | "d" | "desc" => Ok(SortDirection::Dsc),
_ => Err(format!("Unknown direction: {}", s)), _ => Err(format!("Unknown direction: {s}")),
} }
} }
} }
@@ -182,7 +182,7 @@ pub fn run(args: Args, mut writer: &mut dyn Write) -> bool {
match Byte::parse_str(size_str, true) { match Byte::parse_str(size_str, true) {
Ok(byte) => byte.as_u64(), Ok(byte) => byte.as_u64(),
Err(e) => { Err(e) => {
eprintln!("Error parsing size '{}': {}", size_str, e); eprintln!("Error parsing size '{size_str}': {e}");
std::process::exit(1); std::process::exit(1);
} }
} }
@@ -478,7 +478,7 @@ fn print_text(
} }
} }
writeln!(writer, "{}", summary).ok(); writeln!(writer, "{summary}").ok();
} else { } else {
writeln!(writer, "Total size: {}", "Access Denied".bold().red()).ok(); writeln!(writer, "Total size: {}", "Access Denied".bold().red()).ok();
} }
@@ -649,7 +649,7 @@ fn print_text(
table.add_row(row); table.add_row(row);
} }
} }
writeln!(writer, "{}", table).ok(); writeln!(writer, "{table}").ok();
} }
fn print_csv(writer: &mut dyn Write, children: &[&Node], args: &Args) { fn print_csv(writer: &mut dyn Write, children: &[&Node], args: &Args) {
+1 -1
View File
@@ -18,7 +18,7 @@ fn main() {
OutputFormat::Json => "json", OutputFormat::Json => "json",
_ => "txt", _ => "txt",
}; };
PathBuf::from(format!("{}_{}.{}", timestamp, dir_name, ext)) PathBuf::from(format!("{timestamp}_{dir_name}.{ext}"))
} else { } else {
path_arg.clone() path_arg.clone()
}; };
+1 -1
View File
@@ -58,7 +58,7 @@ pub enum EntryType {
impl std::fmt::Display for EntryType { impl std::fmt::Display for EntryType {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
write!(f, "{:?}", self) write!(f, "{self:?}")
} }
} }
+96
View File
@@ -0,0 +1,96 @@
#![cfg(target_os = "linux")]
use sized::{scan_tree, ScanControl, ScanOptions};
use std::{fs, io::Write, os::unix::fs::MetadataExt, path::PathBuf, process::Command};
use tempfile::NamedTempFile;
#[test]
#[ignore = "requires the two owned tmpfs mounts from scripts/check-linux.sh"]
fn real_mount_boundary_is_respected_by_scanner_and_cli() {
let root = PathBuf::from(
std::env::var_os("SIZED_TEST_MOUNT_ROOT").expect("missing mounted Linux fixture"),
);
let foreign = root.join("foreign");
let root_metadata = fs::metadata(&root).unwrap();
let foreign_metadata = fs::metadata(&foreign).unwrap();
assert_ne!(
root_metadata.dev(),
foreign_metadata.dev(),
"not a real boundary"
);
let mut local_file = NamedTempFile::new_in(&root).unwrap();
let mut mounted_file = NamedTempFile::new_in(&foreign).unwrap();
local_file.write_all(b"local allocation").unwrap();
mounted_file.write_all(&[7; 8192]).unwrap();
local_file.as_file().sync_all().unwrap();
mounted_file.as_file().sync_all().unwrap();
for bounded in [false, true] {
let report = scan_tree(
&root,
&ScanOptions {
stay_on_filesystem: bounded,
threads: Some(2),
..ScanOptions::default()
},
&ScanControl::default(),
)
.unwrap();
assert!(report.root.complete);
assert!(report.issues.is_empty());
let mount = report
.root
.children
.iter()
.find(|n| n.path == foreign)
.unwrap();
assert_eq!(mount.identity.unwrap().device, foreign_metadata.dev());
assert_eq!(mount.skipped_mount, bounded);
assert_eq!(report.entries_scanned, if bounded { 3 } else { 4 });
let expected_blocks =
root_metadata.blocks() + local_file.as_file().metadata().unwrap().blocks();
if bounded {
assert!(mount.children.is_empty());
assert_eq!(mount.blocks, 0);
assert_eq!(mount.size_bytes, 0);
assert_eq!(report.root.blocks, expected_blocks);
} else {
assert_eq!(mount.children.len(), 1);
assert_eq!(mount.children[0].path, mounted_file.path());
assert_eq!(mount.children[0].size_bytes, 8192);
assert_eq!(
report.root.blocks,
expected_blocks
+ foreign_metadata.blocks()
+ mounted_file.as_file().metadata().unwrap().blocks()
);
}
let mut command = Command::new(env!("CARGO_BIN_EXE_sized"));
command
.arg(&root)
.args(["--format", "json", "--apparent", "--threads", "2"]);
if bounded {
command.arg("--one-file-system");
}
let output = command.output().unwrap();
assert!(
output.status.success(),
"{}",
String::from_utf8_lossy(&output.stderr)
);
assert!(output.stderr.is_empty());
let json: serde_json::Value = serde_json::from_slice(&output.stdout).unwrap();
assert_eq!(json["complete"], true);
assert_eq!(json["total_blocks"], report.root.blocks);
let mount_json = json["entries"]
.as_array()
.unwrap()
.iter()
.find(|entry| entry["path"] == foreign.to_str().unwrap())
.unwrap();
assert_eq!(mount_json["skipped_mount"], bounded);
assert_eq!(mount_json["blocks"], mount.blocks);
assert_eq!(mount_json["apparent_size"], mount.size_bytes);
}
}