Correct scanner accounting and verify release packaging #2
@@ -21,6 +21,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
||||
cooperative cancellation, and per-scan thread pools.
|
||||
- Optional `-x` / `--one-file-system` boundary handling. Partial CLI reports
|
||||
return a nonzero exit status and include `complete` in JSON output.
|
||||
- Repeatable Linux ARM64/AMD64 Docker checks with unprivileged permission
|
||||
tests, a real mounted-filesystem boundary and an optimized-binary smoke test.
|
||||
|
||||
## [1.0.0] - 2026-01-22
|
||||
|
||||
|
||||
@@ -149,6 +149,29 @@ for each request. Clones of the control expose progress and cancellation;
|
||||
cancellation is cooperative between filesystem calls. `build_tree` remains as
|
||||
a convenience helper, while `scan_tree` retains structured diagnostics.
|
||||
|
||||
## Development checks
|
||||
|
||||
Run `cargo test --locked` locally as an unprivileged user. For Linux build,
|
||||
permission, filesystem-boundary and release smoke checks with Docker:
|
||||
|
||||
```bash
|
||||
./scripts/check-linux.sh linux/arm64
|
||||
./scripts/check-linux.sh linux/amd64
|
||||
```
|
||||
|
||||
The script pins the official Rust 1.88.0 Bookworm image by digest, adds rustfmt
|
||||
and Clippy, then runs as UID 65532 with dropped capabilities. Source is mounted
|
||||
read-only and copied into the temporary container. Fixtures live on Linux
|
||||
filesystems, including two distinct tmpfs mounts; no privileged container or
|
||||
host directory scan is required. AMD64 on an ARM64 host requires emulation.
|
||||
Logs are saved in `target/linux-checks/`; containers and their build output are
|
||||
removed on exit. Docker retains the reusable check images/build cache.
|
||||
`SIZED_LINUX_JOBS` changes the default two build workers; `SIZED_LINUX_IMAGE`
|
||||
can select a different toolchain image for an explicit compatibility check.
|
||||
These are backend checks; desktop X11/Wayland acceptance belongs to SizeQueen.
|
||||
|
||||
See [source review and release process](SHIPMENT.md) and the [live queue](TODO.md).
|
||||
|
||||
## License
|
||||
|
||||
This project is licensed under the **GNU General Public License v3.0 (GPL-3.0)**.
|
||||
|
||||
+25
@@ -2,6 +2,31 @@
|
||||
|
||||
This document defines the process for versioning and distributing the `sized` project independently of the Git hosting provider (GitHub, GitLab, Gitea).
|
||||
|
||||
## Source review before a release
|
||||
|
||||
Use a named branch from `main` and keep a pull request focused on one outcome.
|
||||
The `sizequeen-scan-hardening` branch corrects accounting/error handling and
|
||||
adds the scan controls needed by SizeQueen; shared-crate extraction follows
|
||||
in a separate PR so reviewers can distinguish behaviour changes from packaging.
|
||||
|
||||
1. Run local locked tests and strict Clippy as an unprivileged user. Run
|
||||
`./scripts/check-linux.sh linux/arm64` and
|
||||
`./scripts/check-linux.sh linux/amd64` for the repeatable Linux checks,
|
||||
including actual mount boundaries and an optimized-binary smoke test.
|
||||
2. Push the review branch. Open a PR against `main` when its scope is ready,
|
||||
explaining changed behaviour, test evidence and remaining limits. For this
|
||||
branch, call out nonzero status on partial scans, added JSON status fields,
|
||||
and library API changes. Record current work and evidence in `TODO.md`.
|
||||
3. Review and merge independently of distribution. A branch push or PR merge
|
||||
does not publish a package, change repository visibility or create a tag.
|
||||
4. Choose the release version after reviewing library/API compatibility, then
|
||||
use the release steps below when explicitly authorized. Reconcile legacy
|
||||
GitLab download/package links before announcing a Gitea release.
|
||||
|
||||
Use the same Linux check script in Gitea CI when a Docker-capable runner is
|
||||
configured. The current branch provides the local entry point; it does not
|
||||
configure a runner or enable automatic publishing.
|
||||
|
||||
## 1. Versioning and Tagging
|
||||
|
||||
The project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
||||
|
||||
@@ -10,12 +10,12 @@ fn setup_test_dir() -> TempDir {
|
||||
|
||||
// Create a moderately deep structure
|
||||
for i in 0..10 {
|
||||
let dir_path = base_path.join(format!("dir_{}", i));
|
||||
let dir_path = base_path.join(format!("dir_{i}"));
|
||||
fs::create_dir(&dir_path).unwrap();
|
||||
for j in 0..10 {
|
||||
let file_path = dir_path.join(format!("file_{}.txt", j));
|
||||
let file_path = dir_path.join(format!("file_{j}.txt"));
|
||||
let mut file = File::create(file_path).unwrap();
|
||||
writeln!(file, "Some content for file {}-{}", i, j).unwrap();
|
||||
writeln!(file, "Some content for file {i}-{j}").unwrap();
|
||||
}
|
||||
}
|
||||
temp_dir
|
||||
|
||||
Executable
+26
@@ -0,0 +1,26 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
test "$(uname -s)" = Linux
|
||||
test "$(id -u)" != 0
|
||||
printf 'Linux checks: uid=%s architecture=%s\n' "$(id -u)" "$(uname -m)"
|
||||
rustc --version
|
||||
cargo --version
|
||||
|
||||
# Only source inputs are copied. Cargo output and all fixtures disappear with
|
||||
# the container; the host checkout is read-only and no personal tree is scanned.
|
||||
check_root=$(mktemp -d /tmp/sized-check.XXXXXX)
|
||||
cp /source/Cargo.toml /source/Cargo.lock "$check_root/"
|
||||
cp -R /source/src /source/tests /source/benches "$check_root/"
|
||||
cd "$check_root"
|
||||
cargo fmt --all -- --check
|
||||
cargo test --locked
|
||||
cargo test --locked --test linux_mount -- --ignored
|
||||
cargo clippy --locked --all-targets -- -D warnings
|
||||
cargo build --locked --release
|
||||
|
||||
fixture_root=$(mktemp -d /tmp/sized-release.XXXXXX)
|
||||
printf 'Linux release smoke test\n' > "$fixture_root/payload"
|
||||
./target/release/sized --version
|
||||
./target/release/sized "$fixture_root" --threads 2 --one-file-system --apparent --format json
|
||||
printf 'Linux checks passed.\n'
|
||||
Executable
+27
@@ -0,0 +1,27 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
# Pin the multi-platform official image, not a moving tag. Tests run on Linux
|
||||
# filesystems rather than the source bind mount, whose permission semantics vary.
|
||||
repo_root=$(cd "$(dirname "$0")/.." && pwd)
|
||||
platform=${1:-linux/$(docker version --format '{{.Server.Arch}}')}
|
||||
case "$platform" in
|
||||
linux/arm64|linux/amd64) ;;
|
||||
*) printf 'Usage: %s [linux/arm64|linux/amd64]\n' "$0" >&2; exit 2 ;;
|
||||
esac
|
||||
image=${SIZED_LINUX_IMAGE:-rust:1.88.0-bookworm@sha256:af306cfa71d987911a781c37b59d7d67d934f49684058f96cf72079c3626bfe0}
|
||||
check_image="sized-linux-check:${platform#linux/}"
|
||||
mkdir -p "$repo_root/target/linux-checks"
|
||||
log_file="$repo_root/target/linux-checks/${platform#linux/}.log"
|
||||
|
||||
docker build --platform "$platform" --build-arg "BASE_IMAGE=$image" \
|
||||
--tag "$check_image" - < "$repo_root/scripts/linux-check.Dockerfile"
|
||||
docker run --rm --platform "$platform" \
|
||||
--user 65532:65532 --cap-drop ALL --security-opt no-new-privileges \
|
||||
--mount "type=bind,src=$repo_root,dst=/source,readonly" \
|
||||
--tmpfs /tmp/sized-mount-test:rw,nosuid,nodev,noexec,size=16m,uid=65532,gid=65532,mode=0700 \
|
||||
--tmpfs /tmp/sized-mount-test/foreign:rw,nosuid,nodev,noexec,size=16m,uid=65532,gid=65532,mode=0700 \
|
||||
--env CARGO_HOME=/tmp/sized-cargo \
|
||||
--env CARGO_BUILD_JOBS="${SIZED_LINUX_JOBS:-2}" \
|
||||
--env SIZED_TEST_MOUNT_ROOT=/tmp/sized-mount-test \
|
||||
"$check_image" bash /source/scripts/check-linux-container.sh 2>&1 | tee "$log_file"
|
||||
@@ -0,0 +1,3 @@
|
||||
ARG BASE_IMAGE=rust:1.88.0-bookworm@sha256:af306cfa71d987911a781c37b59d7d67d934f49684058f96cf72079c3626bfe0
|
||||
FROM ${BASE_IMAGE}
|
||||
RUN rustup component add rustfmt clippy
|
||||
+5
-5
@@ -131,7 +131,7 @@ impl FromStr for SortColumn {
|
||||
"size" | "s" | "disk" | "d" => Ok(SortColumn::Disk),
|
||||
"apparent" | "a" => Ok(SortColumn::Apparent),
|
||||
"blocks" | "b" => Ok(SortColumn::Blocks),
|
||||
_ => Err(format!("Unknown column: {}", s)),
|
||||
_ => Err(format!("Unknown column: {s}")),
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -148,7 +148,7 @@ impl FromStr for SortDirection {
|
||||
match s.to_lowercase().as_str() {
|
||||
"asc" | "a" => Ok(SortDirection::Asc),
|
||||
"dsc" | "d" | "desc" => Ok(SortDirection::Dsc),
|
||||
_ => Err(format!("Unknown direction: {}", s)),
|
||||
_ => Err(format!("Unknown direction: {s}")),
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -182,7 +182,7 @@ pub fn run(args: Args, mut writer: &mut dyn Write) -> bool {
|
||||
match Byte::parse_str(size_str, true) {
|
||||
Ok(byte) => byte.as_u64(),
|
||||
Err(e) => {
|
||||
eprintln!("Error parsing size '{}': {}", size_str, e);
|
||||
eprintln!("Error parsing size '{size_str}': {e}");
|
||||
std::process::exit(1);
|
||||
}
|
||||
}
|
||||
@@ -478,7 +478,7 @@ fn print_text(
|
||||
}
|
||||
}
|
||||
|
||||
writeln!(writer, "{}", summary).ok();
|
||||
writeln!(writer, "{summary}").ok();
|
||||
} else {
|
||||
writeln!(writer, "Total size: {}", "Access Denied".bold().red()).ok();
|
||||
}
|
||||
@@ -649,7 +649,7 @@ fn print_text(
|
||||
table.add_row(row);
|
||||
}
|
||||
}
|
||||
writeln!(writer, "{}", table).ok();
|
||||
writeln!(writer, "{table}").ok();
|
||||
}
|
||||
|
||||
fn print_csv(writer: &mut dyn Write, children: &[&Node], args: &Args) {
|
||||
|
||||
+1
-1
@@ -18,7 +18,7 @@ fn main() {
|
||||
OutputFormat::Json => "json",
|
||||
_ => "txt",
|
||||
};
|
||||
PathBuf::from(format!("{}_{}.{}", timestamp, dir_name, ext))
|
||||
PathBuf::from(format!("{timestamp}_{dir_name}.{ext}"))
|
||||
} else {
|
||||
path_arg.clone()
|
||||
};
|
||||
|
||||
+1
-1
@@ -58,7 +58,7 @@ pub enum EntryType {
|
||||
|
||||
impl std::fmt::Display for EntryType {
|
||||
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
|
||||
write!(f, "{:?}", self)
|
||||
write!(f, "{self:?}")
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -0,0 +1,96 @@
|
||||
#![cfg(target_os = "linux")]
|
||||
|
||||
use sized::{scan_tree, ScanControl, ScanOptions};
|
||||
use std::{fs, io::Write, os::unix::fs::MetadataExt, path::PathBuf, process::Command};
|
||||
use tempfile::NamedTempFile;
|
||||
|
||||
#[test]
|
||||
#[ignore = "requires the two owned tmpfs mounts from scripts/check-linux.sh"]
|
||||
fn real_mount_boundary_is_respected_by_scanner_and_cli() {
|
||||
let root = PathBuf::from(
|
||||
std::env::var_os("SIZED_TEST_MOUNT_ROOT").expect("missing mounted Linux fixture"),
|
||||
);
|
||||
let foreign = root.join("foreign");
|
||||
let root_metadata = fs::metadata(&root).unwrap();
|
||||
let foreign_metadata = fs::metadata(&foreign).unwrap();
|
||||
assert_ne!(
|
||||
root_metadata.dev(),
|
||||
foreign_metadata.dev(),
|
||||
"not a real boundary"
|
||||
);
|
||||
let mut local_file = NamedTempFile::new_in(&root).unwrap();
|
||||
let mut mounted_file = NamedTempFile::new_in(&foreign).unwrap();
|
||||
local_file.write_all(b"local allocation").unwrap();
|
||||
mounted_file.write_all(&[7; 8192]).unwrap();
|
||||
local_file.as_file().sync_all().unwrap();
|
||||
mounted_file.as_file().sync_all().unwrap();
|
||||
|
||||
for bounded in [false, true] {
|
||||
let report = scan_tree(
|
||||
&root,
|
||||
&ScanOptions {
|
||||
stay_on_filesystem: bounded,
|
||||
threads: Some(2),
|
||||
..ScanOptions::default()
|
||||
},
|
||||
&ScanControl::default(),
|
||||
)
|
||||
.unwrap();
|
||||
assert!(report.root.complete);
|
||||
assert!(report.issues.is_empty());
|
||||
let mount = report
|
||||
.root
|
||||
.children
|
||||
.iter()
|
||||
.find(|n| n.path == foreign)
|
||||
.unwrap();
|
||||
assert_eq!(mount.identity.unwrap().device, foreign_metadata.dev());
|
||||
assert_eq!(mount.skipped_mount, bounded);
|
||||
assert_eq!(report.entries_scanned, if bounded { 3 } else { 4 });
|
||||
let expected_blocks =
|
||||
root_metadata.blocks() + local_file.as_file().metadata().unwrap().blocks();
|
||||
if bounded {
|
||||
assert!(mount.children.is_empty());
|
||||
assert_eq!(mount.blocks, 0);
|
||||
assert_eq!(mount.size_bytes, 0);
|
||||
assert_eq!(report.root.blocks, expected_blocks);
|
||||
} else {
|
||||
assert_eq!(mount.children.len(), 1);
|
||||
assert_eq!(mount.children[0].path, mounted_file.path());
|
||||
assert_eq!(mount.children[0].size_bytes, 8192);
|
||||
assert_eq!(
|
||||
report.root.blocks,
|
||||
expected_blocks
|
||||
+ foreign_metadata.blocks()
|
||||
+ mounted_file.as_file().metadata().unwrap().blocks()
|
||||
);
|
||||
}
|
||||
|
||||
let mut command = Command::new(env!("CARGO_BIN_EXE_sized"));
|
||||
command
|
||||
.arg(&root)
|
||||
.args(["--format", "json", "--apparent", "--threads", "2"]);
|
||||
if bounded {
|
||||
command.arg("--one-file-system");
|
||||
}
|
||||
let output = command.output().unwrap();
|
||||
assert!(
|
||||
output.status.success(),
|
||||
"{}",
|
||||
String::from_utf8_lossy(&output.stderr)
|
||||
);
|
||||
assert!(output.stderr.is_empty());
|
||||
let json: serde_json::Value = serde_json::from_slice(&output.stdout).unwrap();
|
||||
assert_eq!(json["complete"], true);
|
||||
assert_eq!(json["total_blocks"], report.root.blocks);
|
||||
let mount_json = json["entries"]
|
||||
.as_array()
|
||||
.unwrap()
|
||||
.iter()
|
||||
.find(|entry| entry["path"] == foreign.to_str().unwrap())
|
||||
.unwrap();
|
||||
assert_eq!(mount_json["skipped_mount"], bounded);
|
||||
assert_eq!(mount_json["blocks"], mount.blocks);
|
||||
assert_eq!(mount_json["apparent_size"], mount.size_bytes);
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user