feat: v0.2.0 - Performance optimization and distribution pipeline

- Implemented multi-threaded directory traversal using ignore::WalkParallel
- Added dynamic thread allocation for balanced CPU utilization
- Refactored core logic to single-pass, in-memory tree construction (O(N))
- Added --compare, --units, and --precision flags
- Established SCM-neutral distribution pipeline (Makefile, release.sh, Homebrew)
- Comprehensive updates to README.md, MANUAL.md, and CHANGELOG.md
- Decoupled from SCM-specific workflows
This commit is contained in:
2026-01-22 15:00:48 -05:00
parent db9059e2fc
commit 276c6404a9
13 changed files with 763 additions and 138 deletions
+1
View File
@@ -1 +1,2 @@
/target /target
/dist
+34 -26
View File
@@ -7,36 +7,44 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [0.2.0] - 2026-01-22 ## [0.2.0] - 2026-01-22
### Changed
- Refactored core traversal logic to a **single-pass, in-memory tree construction** algorithm. This visits every file exactly once, replacing the previous $O(N \times Depth)$ behavior with $O(N)$.
- Switched default CSV output to use the `csv` crate for robust escaping (fixes issues with filenames containing commas).
- Optimized JSON output to use zero-copy serialization, improving performance and memory usage.
### Added ### Added
- Added `MANUAL.md` with comprehensive usage documentation. - **Comparison Mode**: Added `--compare` flag to display total size vs. non-ignored size (respecting `.gitignore`).
- Added comprehensive test suite (unit and integration tests). - **Unit Control**: Added `--units <binary|decimal>` flag to toggle standard IEC (MiB) vs. SI (MB) unit systems.
- Added performance benchmarks using `criterion`. - **Precision Control**: Added `--precision <number>` to control decimal places in output (default: 2).
- **Physical vs. Logical Distinction**: Explicitly show **Apparent Size** (logical) and **Disk Usage** (actual blocks consumed) in all output formats.
- **Release Assets**:
- Project **Makefile** for easy system-wide installation via `make install`.
- Universal **`scripts/release.sh`** for generating optimized binaries, man pages, shell completions, and `.deb` packages.
- Official **man page** (`sized.1`) for standard Unix documentation documentation.
- Updated **Homebrew Formula** for tap-based distribution.
- **Improved Errors**: Graceful handling for inaccessible directories (e.g., system protected folders) with "Access Denied" status.
- Added a full user guide in `MANUAL.md`.
### Changed
- **Performance Overhaul**:
- Replaced serial/par-bridge traversal with `ignore::WalkParallel` for true multi-threaded directory scanning.
- Implemented **dynamic thread allocation** to balance CPU load between directory breadth and depth.
- Refactored core logic to a **single-pass, in-memory tree construction** algorithm ($O(N)$ complexity).
- **Architecture**: Refactored the project into a usable Rust library (`src/lib.rs`) and a thin CLI wrapper (`src/main.rs`).
- **Dependencies**:
- Switched to `byte-unit` for more flexible and accurate unit formatting.
- Replaced `walkdir` with `ignore` for faster traversal and native `.gitignore` support.
- Replaced `atty` and `chrono` with standard library features and the lightweight `time` crate.
- **Serialization**: Optimized JSON and CSV output for zero-copy performance and detailed metric reporting.
### Fixed
- Resolved performance bottleneck in extremely deep directory structures.
- Fixed inconsistent total size reporting when `.gitignore` was active.
### Removed ### Removed
- Removed `walkdir` dependency (replaced by `ignore::WalkBuilder` for all traversal). - Dependency on `walkdir`, `humansize`, `chrono`, and `atty`.
- Removed `humansize` (consolidated to `byte-unit`).
- Removed `chrono` (replaced with lightweight `time`).
- Removed `atty` (unused).
## [0.1.0] - 2026-01-22 ## [0.1.0] - 2026-01-22
### Added ### Added
- Core directory traversal and size calculation logic. - Initial release with core directory traversal and size calculation.
- Parallel processing using `rayon` for high performance. - Basic parallel processing using `rayon`.
- Rich table output format using `comfy-table` with colored entries. - Rich table output via `comfy-table`.
- Recursive depth control via `-d`/`--depth`. - Basic filtering (`--min-size`, `--depth`) and sorting (`--sort`).
- Minimum size filtering via `-m`/`--min-size`. - Output format support: Text, CSV, and JSON.
- Multi-column sorting support (`--sort`). - Save to file and shell completion generation.
- Relative path display by default, with `-f`/`--path-full` override.
- "Total size" summary with dual units (Binary/Decimal) and block counts.
- Concurrency control via `-j`/`--threads`.
- `.gitignore` and `.ignore` file support via `-i`/`--ignore`.
- Multiple output formats: Text (Table), CSV, JSON (`--format`).
- Save to file functionality with auto-naming (`--save`).
- Shell completion generation (`--completions`).
+24
View File
@@ -2,6 +2,30 @@
name = "sized" name = "sized"
version = "0.2.0" version = "0.2.0"
edition = "2021" edition = "2021"
description = "A modern, fast, and concurrent disk usage analyzer"
license = "MIT"
repository = "https://gitea.speelman.ca/gamertan/sized"
readme = "README.md"
categories = ["command-line-utilities", "filesystem"]
keywords = ["du", "size", "disk", "usage", "parallel"]
[package.metadata.deb]
maintainer = "Cole Speelman"
copyright = "2026, Cole Speelman"
license-file = ["LICENSE", "4"]
extended-description = """
sized is a modern, fast, and concurrent disk usage analyzer.
Features include apparent vs disk usage, binary/decimal units, and gitignore support.
"""
section = "utils"
priority = "optional"
assets = [
["target/release/sized", "usr/bin/sized", "755"],
["sized.1", "usr/share/man/man1/sized.1", "644"],
["README.md", "usr/share/doc/sized/README.md", "644"],
]
[dependencies] [dependencies]
clap = { version = "4.4", features = ["derive"] } clap = { version = "4.4", features = ["derive"] }
+23
View File
@@ -0,0 +1,23 @@
class Sized < Formula
desc "A modern, fast, and concurrent disk usage analyzer"
homepage "https://gitea.speelman.ca/gamertan/sized"
url "https://gitea.speelman.ca/gamertan/sized/archive/v0.2.0.tar.gz"
sha256 "REPLACE_WITH_ACTUAL_SHA256" # Run `shasum -a 256 v0.2.0.tar.gz` after upload
license "MIT"
depends_on "rust" => :build
def install
system "cargo", "install", *std_cargo_args
# Install man page
man1.install "sized.1"
# Generate and install shell completions
generate_completions_from_executable(bin/"sized", "--completions")
end
test do
system "#{bin}/sized", "--version"
end
end
+21
View File
@@ -0,0 +1,21 @@
MIT License
Copyright (c) 2026 Cole Speelman
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
+82 -54
View File
@@ -8,80 +8,85 @@
- [Filtering and sorting](#filtering-and-sorting) - [Filtering and sorting](#filtering-and-sorting)
- [Output Formats](#output-formats) - [Output Formats](#output-formats)
- [Advanced Features](#advanced-features) - [Advanced Features](#advanced-features)
- [Concurrency](#concurrency) - [Comparison Mode](#comparison-mode)
- [Gitignore Support](#gitignore-support) - [Gitignore Support](#gitignore-support)
- [Concurrency](#concurrency)
- [Exporting Data](#exporting-data) - [Exporting Data](#exporting-data)
- [Shell Completions](#shell-completions) - [Shell Completions](#shell-completions)
## Installation ## Installation
Currently, `sized` can be installed from source: ### From Binaries (Recommended)
Download the latest pre-compiled binaries from the [Gitea Releases](https://gitea.speelman.ca/gamertan/sized/releases) page.
### From Source
```bash ```bash
git clone <repository_url> git clone https://gitea.speelman.ca/gamertan/sized.git
cd sized cd sized
cargo install --path . make install # Installs binary and man page
``` ```
## Basic Usage ## Basic Usage
By default, `sized` analyzes the current directory recursively and displays a table of the immediate children, sorted by name. By default, `sized` analyzes the current directory recursively and displays a table of the immediate children.
```bash ```bash
sized sized [path]
``` ```
To analyze a specific path: ### Key Concepts
- **Apparent Size**: The logical size of the entry (file length). This is what most file explorers show.
- **Disk Usage**: The actual physical space consumed on disk (Blocks * 512 bytes). This is the "true" footprint.
- **Blocks**: The actual filesystem blocks allocated. Useful for identifying sparse files or filesystem overhead.
### Path Display
- **Relative Path** (Default): `sized` shows paths relative to the current directory.
- **Full Path**: Use `-f` or `--path-full` to see absolute paths (e.g., `/Users/dev/project`).
- **Relative Toggle**: Use `--path-relative` to explicitly force relative paths (useful if overriding aliases).
### Depth Control (`-d` / `--depth`)
By default, `sized` shows the immediate children of the target directory (depth 0). You can increase the recursion depth shown in the output.
```bash ```bash
sized /path/to/directory # Show current directory and its children's children
sized -d 1
``` ```
The output includes: > [!NOTE]
- **Type**: Icon indicating if it's a directory (📁) or file (📄). > Regardless of the display depth, `sized` always calculates the *total* size of all subdirectories accurately by traversing the entire tree.
- **Name**: The relative path to the entry.
- **Size**: Human-readable size (e.g., 10 MB, 2.5 GB).
- **% of Parent**: The percentage of the total size of the *current view* (immediate children) that this entry consumes.
- **Last Modified**: Time since the file was last modified.
### Total Size Header
The header displays the total size of the scanned directory in both Binary (MiB/GiB) and Decimal (MB/GB) units, along with the total block count.
## Filtering and Sorting ## Filtering and Sorting
### Sorting (`--sort`) ### Sorting (`--sort`)
Sort the output table by a specific column. Sort the output table by a specific column. `sized` supports multi-column sorting.
**Syntax**: `--sort <COLUMN:DIRECTION>` **Syntax**: `--sort <COLUMN:DIRECTION>,[COLUMN:DIRECTION]`
- **Columns**: `name`, `size`, `type` - **Columns**: `name` (n), `size` (s), `type` (t), `blocks` (b)
- **Directions**: `asc` (ascending), `desc` (descending) - **Directions**: `asc` (a), `dsc` (d)
**Examples**: **Examples**:
```bash ```bash
# Sort by size (largest first) - Default behavior # Sort by size (largest first) - Default behavior
sized --sort size:desc sized --sort size:dsc
# Sort by name (A-Z) # Primary sort by Type, secondary sort by Size descending
sized --sort name:asc sized --sort type:asc,size:dsc
``` ```
### Minimum Size (`-m` / `--min-size`) ### Minimum Size (`-m` / `--min-size`)
Hide entries smaller than a specific size to reduce noise. Hide entries smaller than a specific size to reduce noise. Supports standard units (KB, MB, GB, etc.).
**Examples**:
```bash ```bash
# Show only files/dirs larger than 10 MB # Show only items larger than 100MB
sized -m 10MB sized -m 100MB
# Show only files/dirs larger than 1 GB
sized --min-size 1GB
``` ```
### Depth Control (`-d` / `--depth`) ### Limiting Results (`-n` / `--number`)
Limit the recursion depth for the *calculation*. Note that the display currently shows immediate children, but this flag controls how deep `sized` looks to calculate directory sizes. Limit the number of rows displayed in each table.
*(Note: Deeply nested directory sizes are always fully calculated unless limited)*
```bash ```bash
sized -d 2 # Show only the top 10 largest items
sized --sort size:dsc -n 10
``` ```
## Output Formats (`--format`) ## Output Formats (`--format`)
@@ -95,54 +100,77 @@ sized --format text
``` ```
### CSV ### CSV
Comma-Separated Values, suitable for spreadsheets. Comma-Separated Values.
```bash ```bash
sized --format csv sized --format csv
``` ```
Columns: `path`, `size_bytes`, `files`, `dirs`
### JSON ### JSON
Computed metrics in NDJSON (Newline Delimited JSON) format. Computed metrics in NDJSON format.
```bash ```bash
sized --format json sized --format json
``` ```
## Advanced Features ## Advanced Features
### Path Display ### Unit System (`--units`)
- **Default**: Relative paths (`./folder`) Switch between Binary (IEC) and Decimal (SI) units for the output.
- **Full Path**: Use `-f` or `--path-full` to see absolute paths (`/users/name/folder`).
- **Relative Path**: Use `--path-relative` to explicitly force relative paths.
### Concurrency (`-j` / `--threads`) - **Binary** (Default): GiB, MiB, KiB (multiples of 1024).
`sized` uses parallel processing. By default, it uses a number of threads equal to your CPU cores. You can limit this for system stability or increase it (though usually not clear). - **Decimal**: GB, MB, KB (multiples of 1000).
```bash ```bash
# Limit to 4 threads # Use decimal units (SI)
sized -j 4 sized --units decimal
``` ```
### Gitignore Support (`-i` / `--ignore`) ### Precision (`--precision`)
Respect `.gitignore` and `.ignore` files. This is useful for checking the size of a project *as it would be committed*, ignoring `target/`, `node_modules/`, etc. Specify the number of decimal places for formatted sizes. Defaults to `2`.
```bash ```bash
# Higher precision for small files
sized --precision 4
```
### Comparison Mode (`--compare`)
Compare the total physical size of a directory against its "filtered" size (files that would be included in a commit, respecting `.gitignore`).
```bash
# Show both total and non-ignored metrics
sized -i --compare
```
This adds **"Filtered"** rows and headers to the output, allowing you to see how much space is consumed by ignored files (like `node_modules` or `target`).
### Gitignore Support (`-i` / `--ignore`)
Respect `.ignore` and `.gitignore` files during traversal. This is highly recommended for developer workflows.
```bash
# Exclude git-ignored files from calculations
sized -i sized -i
``` ```
### Exporting Data (`--save`) ### Concurrency (`-j` / `--threads`)
Save the output directly to a file. If you use `_AUTO_` (or provide no argument to the flag), it generates a filename with the current timestamp. `sized` uses parallel processing powered by `rayon` and `ignore`. By default, it uses a number of threads equal to your logical CPU cores.
```bash ```bash
# Save to a specific file # Limit to 4 threads on a high-core system
sized --save report.txt sized -j 4
```
# Save to a timestamped file (e.g., 20250122-120000_sized_report.txt) ### Exporting Data (`--save`)
Save the output directly to a file. If no filename is provided, `sized` generates a timestamped one (e.g., `sized_report_20260122_1430.txt`).
```bash
# Save to an auto-generated file
sized --save sized --save
# Save to a specific path
sized --save reports/my_audit.json --format json
``` ```
## Shell Completions ## Shell Completions
Generate shell completion scripts for Bash, Zsh, Fish, PowerShell, or Elvish. Generate shell completion scripts for Bash, Zsh, or Fish.
```bash ```bash
# Generate for Zsh # Generate for Zsh
+25
View File
@@ -0,0 +1,25 @@
PREFIX ?= /usr/local
BINDIR = $(PREFIX)/bin
MANDIR = $(PREFIX)/share/man/man1
.PHONY: all build install clean help
all: build
build:
cargo build --release
install: build
install -d $(BINDIR)
install -m 755 target/release/sized $(BINDIR)/sized
install -d $(MANDIR)
install -m 644 sized.1 $(MANDIR)/sized.1
clean:
cargo clean
help:
@echo "Usage:"
@echo " make build - Build the optimized binary"
@echo " make install - Install binary and man page (may require sudo)"
@echo " make clean - Remove build artifacts"
+56 -5
View File
@@ -6,22 +6,71 @@ A fast, concurrent, and feature-rich command-line tool for visualizing disk usag
- **Fast & Concurrent**: Uses `rayon` to process directories in parallel, making it extremely fast on modern multi-core systems. - **Fast & Concurrent**: Uses `rayon` to process directories in parallel, making it extremely fast on modern multi-core systems.
- **Rich Output**: Beautifully formatted tables with colors, distinguishing files and directories. - **Rich Output**: Beautifully formatted tables with colors, distinguishing files and directories.
- **Dual Units**: Displays sizes in both Binary (MiB, GiB) and Decimal (MB, GB) units for clarity. - **Physical vs logical**: Clearly separates **Apparent Size** (logical) from **Disk Usage** (actual blocks), essential for cloud files and sparse storage.
- **Unit Selection**: Switch between Binary (IEC) and Decimal (SI) unit systems via `--units`.
- **Precision control**: Configure decimal places using `--precision`.
- **Advanced Filtering**: Filter by minimum size (`-m 10MB`) to find large items quickly. - **Advanced Filtering**: Filter by minimum size (`-m 10MB`) to find large items quickly.
- **Sorting**: Flexible sorting by name, type, size, or block count (`--sort`). - **Sorting**: Flexible sorting by name, type, size (apparent), or blocks (usage) (`--sort`).
- **Gitignore Support**: Respects `.gitignore` and `.ignore` files to keep output clean (`-i`). - **Gitignore Support**: Respects `.gitignore` and `.ignore` files to keep output clean (`-i`).
- **Export Options**: Export data to CSV or JSON for further analysis (`--format csv/json`). - **Export Options**: Export data to CSV or JSON for further analysis (`--format csv/json`).
- **Save to File**: Automatically save reports with timestamped filenames (`--save`). - **Save to File**: Automatically save reports with timestamped filenames (`--save`).
## Installation ## Installation
### From Source ### 🚀 Direct Download (macOS & Linux)
Ensure you have Rust and Cargo installed. Clone the repository and install: Download the latest archive for your architecture from the [Releases](https://gitea.speelman.ca/gamertan/sized/releases) page.
1. **Extract the archive**:
```bash
tar -xzf sized-v0.2.0-Darwin-arm64.tar.gz
```
2. **Install Binary & Man Page**:
```bash
# Move binary to path
sudo mv sized /usr/local/bin/
# Install man page
sudo mkdir -p /usr/local/share/man/man1
sudo cp sized.1 /usr/local/share/man/man1/
```
3. **macOS Security (Gatekeeper)**:
Since the binary isn't code-signed for the App Store, macOS may block it. To allow it:
```bash
sudo xattr -d com.apple.quarantine /usr/local/bin/sized
```
*Alternatively, run `sized` once, let it fail, then go to **System Settings > Privacy & Security** and click **"Allow Anyway"**.*
### 🍺 Homebrew (macOS & Linux)
If you have a homebrew tap:
```bash
brew install gamertan/tap/sized
```
### 📦 Debian / Ubuntu (.deb)
1. Download the `.deb` package from the [Releases](https://gitea.speelman.ca/gamertan/sized/releases) page.
2. Install using `dpkg`:
```bash
sudo dpkg -i sized_0.2.0_amd64.deb
```
### 🦀 From Source (Rust toolchain required)
```bash
git clone https://gitea.speelman.ca/gamertan/sized.git
cd sized
make install # Installs binary and man page
```
Alternatively, via Cargo:
```bash ```bash
cargo install --path . cargo install --path .
``` ```
## Documentation
- **[Manual](MANUAL.md)**: Detailed explanations of all flags and features.
- **[Man Page](sized.1)**: Standard unix man pages (installed via `make install`).
## Usage ## Usage
### Basic Usage ### Basic Usage
@@ -41,7 +90,9 @@ sized /path/to/directory
|------|-------------|---------| |------|-------------|---------|
| `-d`, `--depth` | Recursion depth (0 = current dir only) | `sized -d 1` | | `-d`, `--depth` | Recursion depth (0 = current dir only) | `sized -d 1` |
| `-m`, `--min-size` | Filter by minimum size | `sized -m 100MB` | | `-m`, `--min-size` | Filter by minimum size | `sized -m 100MB` |
| `-s`, `--sort` | Sort columns (size, name, type, blocks) | `sized --sort size:asc` | | `--sort` | Sort columns (name, type, size, blocks) | `sized --sort size:asc` |
| `--units` | Unit system (binary, decimal) | `sized --units decimal` |
| `--precision` | Decimal places for sizes | `sized --precision 3` |
| `-f`, `--path-full` | Force absolute paths in headers | `sized -f` | | `-f`, `--path-full` | Force absolute paths in headers | `sized -f` |
| `-i`, `--ignore` | Respect .gitignore files | `sized -i` | | `-i`, `--ignore` | Respect .gitignore files | `sized -i` |
| `-j`, `--threads` | Set number of threads | `sized -j 4` | | `-j`, `--threads` | Set number of threads | `sized -j 4` |
+56
View File
@@ -0,0 +1,56 @@
# Project Release Process
This document defines the process for versioning and distributing the `sized` project independently of the Git hosting provider (GitHub, GitLab, Gitea).
## 1. Versioning and Tagging
The project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
1. Synchronize the version in `Cargo.toml`.
2. Update `CHANGELOG.md` with the release notes.
3. Commit and create a git tag:
```bash
git tag -a v0.2.0 -m "Release v0.2.0"
```
## 2. Asset Generation
The `scripts/release.sh` script is the authoritative way to generate release assets locally or in any CI environment.
```bash
./scripts/release.sh
```
This script generates:
- **Optimized Binary**: The production-ready `sized` executable.
- **Documentation**: The `sized.1` man page.
- **Shell Completions**: Scripts for Bash, Zsh, and Fish.
- **System Installers**: A `.deb` package for Debian-based Linux distributions.
All assets are gathered in the `dist/v<VERSION>/` directory.
## 3. Distribution Channels
### crates.io
To update the project on the central Rust registry:
```bash
cargo publish
```
### System Package Managers
For Homebrew, GitLab, or Gitea-based distribution:
1. Push the git tag to your SCM.
2. Run the release script to generate assets.
3. Attach the contents of the `dist/` folder to your SCM's "Release" or "Tag" entry.
4. Update downstream formulas (like `homebrew-tap`) with the new source URL and SHA256 of the generated tarball.
## 4. Automation & Hooks
To automate asset generation locally, you can use a git `post-checkout` or a wrapper script. Since gitea and gitlab use different CI formats, it is recommended to simply call `./scripts/release.sh` within your preferred CI runner (e.g., `gitlab-ci.yml` or `gitea-actions`).
## 5. Manual Installation
For system-wide installation from source, use the `Makefile`:
```bash
make install
```
Defaults to `PREFIX=/usr/local`. Overridable via `PREFIX=/your/path make install`.
+1 -1
View File
@@ -26,7 +26,7 @@ fn criterion_benchmark(c: &mut Criterion) {
let path = temp_dir.path(); let path = temp_dir.path();
c.bench_function("build_tree small", |b| { c.bench_function("build_tree small", |b| {
b.iter(|| build_tree(black_box(path), black_box(false))) b.iter(|| build_tree(black_box(path), black_box(false), black_box(false)))
}); });
} }
+53
View File
@@ -0,0 +1,53 @@
#!/bin/bash
set -e
VERSION=$(grep '^version =' Cargo.toml | cut -d '"' -f 2)
DIST_DIR="dist/v$VERSION"
echo "Building release assets for sized v$VERSION..."
# 1. Clean and Prepare
rm -rf "$DIST_DIR"
mkdir -p "$DIST_DIR"
# 2. Build Release Binary
cargo build --release
# 3. Generate Man Page
cargo run --release -- --generate-man-page "$DIST_DIR"
# 4. Generate Completions
cargo run --release -- --completions bash > "$DIST_DIR/sized.bash"
cargo run --release -- --completions zsh > "$DIST_DIR/_sized"
cargo run --release -- --completions fish > "$DIST_DIR/sized.fish"
# 5. Build Debian Package (if cargo-deb is installed)
if command -v cargo-deb &> /dev/null; then
echo "Building Debian package..."
# On macOS, we use --no-strip to avoid 'unrecognized option: --strip-unneeded'
# and --no-build because we already built the release binary.
cargo deb --no-build --no-strip
cp target/debian/*.deb "$DIST_DIR/"
echo "Note: .deb package contains the binary for $(uname -s)-$(uname -m)"
else
echo "Warning: cargo-deb not found. Skipping .deb packaging."
fi
# 6. Build Alpine Package (Placeholder/Hook)
# Note: For real .apk building, one usually uses a docker container or abuild.
# This serves as a reminder for Alpine users.
if [ -f "APKBUILD" ] && command -v abuild &> /dev/null; then
echo "Building Alpine package..."
abuild -r
fi
# 7. Create General Release Tarball
echo "Creating release tarball..."
tar -czf "dist/sized-v$VERSION-$(uname -s)-$(uname -m).tar.gz" -C "$DIST_DIR" .
# 8. Copy Binary
cp target/release/sized "$DIST_DIR/"
echo "Success! Assets are ready in $DIST_DIR"
ls -F "$DIST_DIR"
echo "Tarball: dist/sized-v$VERSION-$(uname -s)-$(uname -m).tar.gz"
+74
View File
@@ -0,0 +1,74 @@
.ie \n(.g .ds Aq \(aq
.el .ds Aq '
.TH sized 1 "sized 0.2.0"
.SH NAME
sized
.SH SYNOPSIS
\fBsized\fR [\fB\-v\fR|\fB\-\-version\fR] [\fB\-m\fR|\fB\-\-min\-size\fR] [\fB\-n\fR|\fB\-\-number\fR] [\fB\-\-sort\fR] [\fB\-d\fR|\fB\-\-depth\fR] [\fB\-f\fR|\fB\-\-path\-full\fR] [\fB\-\-path\-relative\fR] [\fB\-j\fR|\fB\-\-threads\fR] [\fB\-\-completions\fR] [\fB\-i\fR|\fB\-\-ignore\fR] [\fB\-\-format\fR] [\fB\-\-save\fR] [\fB\-\-precision\fR] [\fB\-\-units\fR] [\fB\-\-compare\fR] [\fB\-h\fR|\fB\-\-help\fR] [\fIPATH\fR]
.SH DESCRIPTION
.SH OPTIONS
.TP
\fB\-v\fR, \fB\-\-version\fR
Print version
.TP
\fB\-m\fR, \fB\-\-min\-size\fR \fI<MIN_SIZE>\fR
Minimum size filter (e.g. "10MB", "1gb")
.TP
\fB\-n\fR, \fB\-\-number\fR \fI<NUMBER>\fR
Limit the number of results per table
.TP
\fB\-\-sort\fR \fI<SORT>\fR
Sort by columns (e.g. "type:asc,size:dsc"). defaults to size:dsc. Columns: name, type, size, blocks
.TP
\fB\-d\fR, \fB\-\-depth\fR \fI<DEPTH>\fR [default: 0]
Depth to traverse (0 = only immediate children)
.TP
\fB\-f\fR, \fB\-\-path\-full\fR
Display full absolute paths
.TP
\fB\-\-path\-relative\fR
Display paths relative to current directory (default)
.TP
\fB\-j\fR, \fB\-\-threads\fR \fI<THREADS>\fR
Number of threads to use (defaults to available logical CPUs)
.TP
\fB\-\-completions\fR \fI<COMPLETIONS>\fR
Generate shell completions
.br
.br
[\fIpossible values: \fRbash, elvish, fish, powershell, zsh]
.TP
\fB\-i\fR, \fB\-\-ignore\fR
Respect .gitignore and .ignore files
.TP
\fB\-\-format\fR \fI<FORMAT>\fR [default: text]
Output format
.br
.br
[\fIpossible values: \fRtext, csv, json]
.TP
\fB\-\-save\fR [\fI<SAVE>\fR]
Save output to file (optional filename, defaults to timestamped name)
.TP
\fB\-\-precision\fR \fI<PRECISION>\fR [default: 2]
Precision for size output (decimal places)
.TP
\fB\-\-units\fR \fI<UNITS>\fR [default: binary]
Unit system to use
.br
.br
[\fIpossible values: \fRbinary, decimal]
.TP
\fB\-\-compare\fR
Show comparison between total and non\-ignored files
.TP
\fB\-h\fR, \fB\-\-help\fR
Print help
.TP
[\fIPATH\fR] [default: .]
Directory to analyze
.SH VERSION
v0.2.0
+301 -40
View File
@@ -72,6 +72,22 @@ pub struct Args {
/// Save output to file (optional filename, defaults to timestamped name) /// Save output to file (optional filename, defaults to timestamped name)
#[arg(long, num_args(0..=1), default_missing_value = "_AUTO_")] #[arg(long, num_args(0..=1), default_missing_value = "_AUTO_")]
pub save: Option<PathBuf>, pub save: Option<PathBuf>,
/// Precision for size output (decimal places)
#[arg(long, default_value = "2")]
pub precision: usize,
/// Unit system to use
#[arg(long, value_enum, default_value_t = UnitSystem::Binary)]
pub units: UnitSystem,
/// Show comparison between total and non-ignored files
#[arg(long)]
pub compare: bool,
/// Generate man page to the specified output directory
#[arg(long, hide = true)]
pub generate_man_page: Option<std::path::PathBuf>,
} }
#[derive(ValueEnum, Clone, Debug, PartialEq, Eq)] #[derive(ValueEnum, Clone, Debug, PartialEq, Eq)]
@@ -81,6 +97,12 @@ pub enum OutputFormat {
Json, Json,
} }
#[derive(ValueEnum, Clone, Debug, PartialEq, Eq)]
pub enum UnitSystem {
Binary,
Decimal,
}
#[derive(Debug, Clone, PartialEq, Eq)] #[derive(Debug, Clone, PartialEq, Eq)]
pub enum SortColumn { pub enum SortColumn {
Name, Name,
@@ -124,7 +146,10 @@ pub struct Node {
pub path: PathBuf, pub path: PathBuf,
pub size_bytes: u64, pub size_bytes: u64,
pub blocks: u64, pub blocks: u64,
pub size_bytes_filtered: u64,
pub blocks_filtered: u64,
pub entry_type: EntryType, pub entry_type: EntryType,
pub accessible: bool,
#[serde(skip)] #[serde(skip)]
pub children: Vec<Node>, pub children: Vec<Node>,
} }
@@ -148,6 +173,8 @@ impl std::fmt::Display for EntryType {
pub struct JsonMetrics { pub struct JsonMetrics {
pub size_bytes: u64, pub size_bytes: u64,
pub blocks: u64, pub blocks: u64,
pub size_bytes_filtered: u64,
pub blocks_filtered: u64,
} }
#[derive(Serialize)] #[derive(Serialize)]
@@ -155,6 +182,7 @@ pub struct JsonEntry<'a> {
pub path: &'a PathBuf, pub path: &'a PathBuf,
pub metrics: JsonMetrics, pub metrics: JsonMetrics,
pub entry_type: String, pub entry_type: String,
pub accessible: bool,
} }
#[derive(Serialize)] #[derive(Serialize)]
@@ -162,7 +190,10 @@ pub struct JsonReport<'a> {
pub path: &'a PathBuf, pub path: &'a PathBuf,
pub total_size: u64, pub total_size: u64,
pub blocks: u64, pub blocks: u64,
pub total_size_filtered: u64,
pub blocks_filtered: u64,
pub entries: Vec<JsonEntry<'a>>, pub entries: Vec<JsonEntry<'a>>,
pub accessible: bool,
} }
pub fn run(args: Args, mut writer: &mut dyn Write) { pub fn run(args: Args, mut writer: &mut dyn Write) {
@@ -173,6 +204,19 @@ pub fn run(args: Args, mut writer: &mut dyn Write) {
return; return;
} }
if let Some(out_dir) = args.generate_man_page {
let cmd = Args::command();
let man = clap_mangen::Man::new(cmd);
let mut buffer: Vec<u8> = Default::default();
man.render(&mut buffer).expect("Failed to render man page");
std::fs::create_dir_all(&out_dir).expect("Failed to create man page directory");
let file_path = out_dir.join("sized.1");
std::fs::write(&file_path, buffer).expect("Failed to write man page");
println!("Man page generated at {}", file_path.display());
return;
}
if let Some(threads) = args.threads { if let Some(threads) = args.threads {
rayon::ThreadPoolBuilder::new() rayon::ThreadPoolBuilder::new()
.num_threads(threads) .num_threads(threads)
@@ -205,7 +249,7 @@ pub fn run(args: Args, mut writer: &mut dyn Write) {
writeln!(writer, "Path,Type,Size(Bytes),Blocks").expect("Failed to write CSV header"); writeln!(writer, "Path,Type,Size(Bytes),Blocks").expect("Failed to write CSV header");
} }
let root_node = build_tree(&target_path, args.ignore); let root_node = build_tree(&target_path, args.ignore, args.compare);
if root_node.size_bytes < min_bytes { if root_node.size_bytes < min_bytes {
if args.format == OutputFormat::Text { if args.format == OutputFormat::Text {
@@ -217,47 +261,109 @@ pub fn run(args: Args, mut writer: &mut dyn Write) {
process_node_recursive(&mut writer, &root_node, 0, &args, min_bytes, &sort_criteria); process_node_recursive(&mut writer, &root_node, 0, &args, min_bytes, &sort_criteria);
} }
pub fn build_tree(path: &Path, ignore: bool) -> Node { pub fn build_tree(path: &Path, ignore: bool, compare: bool) -> Node {
let metadata = path.metadata(); let metadata = path.symlink_metadata();
if let Ok(meta) = metadata { if let Ok(meta) = metadata {
if meta.is_file() { // Treat symlinks as files (nodes) but do not recurse
if meta.is_file() || meta.is_symlink() {
return Node { return Node {
path: path.to_path_buf(), path: path.to_path_buf(),
size_bytes: meta.len(), size_bytes: meta.len(),
blocks: meta.blocks(), blocks: meta.blocks(),
size_bytes_filtered: meta.len(), // Single file is its own filtered size for now
blocks_filtered: meta.blocks(),
entry_type: EntryType::File, entry_type: EntryType::File,
accessible: true,
children: vec![], children: vec![],
}; };
} }
} }
let walker = WalkBuilder::new(path) // Check if we can read the directory (handle permissions)
if let Err(e) = std::fs::read_dir(path) {
if e.kind() == std::io::ErrorKind::PermissionDenied {
// Return an "empty" directory node marked as inaccessible
let meta = path.symlink_metadata().ok();
let size = meta.as_ref().map(|m| m.len()).unwrap_or(0);
let blocks = meta.as_ref().map(|m| m.blocks()).unwrap_or(0);
return Node {
path: path.to_path_buf(),
size_bytes: size,
blocks,
size_bytes_filtered: size,
blocks_filtered: blocks,
entry_type: EntryType::Dir,
accessible: false,
children: vec![],
};
}
}
// Total walker (always everything if compare is true, else respects 'ignore' arg)
let total_ignore = if compare { false } else { ignore };
let walker_total = WalkBuilder::new(path)
.standard_filters(false) .standard_filters(false)
.hidden(false) .hidden(false)
.git_ignore(ignore) .git_ignore(total_ignore)
.ignore(ignore) .ignore(total_ignore)
.max_depth(Some(1)) .max_depth(Some(1))
.build(); .build();
let child_paths: Vec<PathBuf> = walker let child_paths_total: Vec<PathBuf> = walker_total
.into_iter() .into_iter()
.filter_map(|e| e.ok()) .filter_map(|e| e.ok())
.filter(|e| e.path() != path) .filter(|e| e.path() != path)
.map(|e| e.path().to_path_buf()) .map(|e| e.path().to_path_buf())
.collect(); .collect();
let children: Vec<Node> = child_paths // Filtered set (only if compare is true)
let non_ignored_set: std::collections::HashSet<PathBuf> = if compare {
let walker_filtered = WalkBuilder::new(path)
.standard_filters(false)
.hidden(false)
.git_ignore(true)
.ignore(true)
.max_depth(Some(1))
.build();
walker_filtered
.into_iter()
.filter_map(|e| e.ok())
.filter(|e| e.path() != path)
.map(|e| e.path().to_path_buf())
.collect()
} else {
std::collections::HashSet::new()
};
let children: Vec<Node> = child_paths_total
.par_iter() .par_iter()
.map(|p| build_tree(p, ignore)) .map(|p| build_tree(p, ignore, compare))
.collect(); .collect();
let (size_bytes, blocks) = children let mut size_bytes = 0;
.iter() let mut blocks = 0;
.fold((0, 0), |acc, c| (acc.0 + c.size_bytes, acc.1 + c.blocks)); let mut size_bytes_filtered = 0;
let mut blocks_filtered = 0;
for child in &children {
size_bytes += child.size_bytes;
blocks += child.blocks;
if compare {
if non_ignored_set.contains(&child.path) {
size_bytes_filtered += child.size_bytes_filtered;
blocks_filtered += child.blocks_filtered;
}
} else {
size_bytes_filtered += child.size_bytes_filtered;
blocks_filtered += child.blocks_filtered;
}
}
let (self_size, self_blocks) = path let (self_size, self_blocks) = path
.metadata() .symlink_metadata()
.map(|m| (m.len(), m.blocks())) .map(|m| (m.len(), m.blocks()))
.unwrap_or((0, 0)); .unwrap_or((0, 0));
@@ -265,7 +371,10 @@ pub fn build_tree(path: &Path, ignore: bool) -> Node {
path: path.to_path_buf(), path: path.to_path_buf(),
size_bytes: size_bytes + self_size, size_bytes: size_bytes + self_size,
blocks: blocks + self_blocks, blocks: blocks + self_blocks,
size_bytes_filtered: size_bytes_filtered + self_size, // self is always part of self
blocks_filtered: blocks_filtered + self_blocks,
entry_type: EntryType::Dir, entry_type: EntryType::Dir,
accessible: true,
children, children,
} }
} }
@@ -387,7 +496,7 @@ fn print_output(
depth: usize, depth: usize,
) { ) {
match args.format { match args.format {
OutputFormat::Text => print_text(writer, parent_node, children, relative_path, depth), OutputFormat::Text => print_text(writer, parent_node, children, args, relative_path, depth),
OutputFormat::Csv => print_csv(writer, children), OutputFormat::Csv => print_csv(writer, children),
OutputFormat::Json => print_json(writer, parent_node, children), OutputFormat::Json => print_json(writer, parent_node, children),
} }
@@ -397,6 +506,7 @@ fn print_text(
writer: &mut dyn Write, writer: &mut dyn Write,
parent_node: &Node, parent_node: &Node,
children: &[&Node], children: &[&Node],
args: &Args,
relative_path: &str, relative_path: &str,
depth: usize, depth: usize,
) { ) {
@@ -405,42 +515,112 @@ fn print_text(
} }
writeln!(writer, "\n{}", relative_path.bold().blue()).ok(); writeln!(writer, "\n{}", relative_path.bold().blue()).ok();
let total_byte = Byte::from_u64(parent_node.size_bytes); let unit_type = match args.units {
writeln!( UnitSystem::Binary => UnitType::Binary,
writer, UnitSystem::Decimal => UnitType::Decimal,
"Total size: {} / {} ({} blocks)", };
total_byte
.get_appropriate_unit(UnitType::Binary) if parent_node.accessible {
.to_string() let total_apparent = Byte::from_u64(parent_node.size_bytes).get_appropriate_unit(unit_type);
let total_usage = Byte::from_u64(parent_node.blocks * 512).get_appropriate_unit(unit_type);
let mut summary = format!(
"Total Apparent: {} / Disk Usage: {} ({} blocks)",
format!(
"{:.precision$} {}",
total_apparent.get_value(),
total_apparent.get_unit(),
precision = args.precision
)
.bold() .bold()
.green(), .green(),
parent_node.size_bytes.to_string().bold().green(), format!(
parent_node.blocks "{:.precision$} {}",
total_usage.get_value(),
total_usage.get_unit(),
precision = args.precision
) )
.ok(); .bold()
.green(),
parent_node.blocks
);
if args.compare {
let total_apparent_filtered =
Byte::from_u64(parent_node.size_bytes_filtered).get_appropriate_unit(unit_type);
let total_usage_filtered =
Byte::from_u64(parent_node.blocks_filtered * 512).get_appropriate_unit(unit_type);
summary.push_str(&format!(
"\nFiltered Apparent: {} / Filtered Disk Usage: {} ({} blocks)",
format!(
"{:.precision$} {}",
total_apparent_filtered.get_value(),
total_apparent_filtered.get_unit(),
precision = args.precision
)
.bold()
.cyan(),
format!(
"{:.precision$} {}",
total_usage_filtered.get_value(),
total_usage_filtered.get_unit(),
precision = args.precision
)
.bold()
.cyan(),
parent_node.blocks_filtered
));
}
writeln!(writer, "{}", summary).ok();
} else {
writeln!(writer, "Total size: {}", "Access Denied".bold().red()).ok();
}
if children.is_empty() { if children.is_empty() {
if parent_node.accessible {
writeln!(writer, "(No children)").ok(); writeln!(writer, "(No children)").ok();
}
return; return;
} }
let mut table = Table::new(); let mut table = Table::new();
table table
.load_preset(UTF8_FULL) .load_preset(UTF8_FULL)
.set_content_arrangement(ContentArrangement::Dynamic) .set_content_arrangement(ContentArrangement::Dynamic);
.set_header(vec![
let mut headers = vec![
Cell::new("Name").add_attribute(Attribute::Bold), Cell::new("Name").add_attribute(Attribute::Bold),
Cell::new("Type").add_attribute(Attribute::Bold), Cell::new("Type").add_attribute(Attribute::Bold),
Cell::new("Size (Human)") Cell::new("Apparent")
.add_attribute(Attribute::Bold) .add_attribute(Attribute::Bold)
.set_alignment(CellAlignment::Right), .set_alignment(CellAlignment::Right),
Cell::new("Size (Bytes)") Cell::new("Disk Usage")
.add_attribute(Attribute::Bold) .add_attribute(Attribute::Bold)
.set_alignment(CellAlignment::Right), .set_alignment(CellAlignment::Right),
];
if args.compare {
headers.push(
Cell::new("App. Filtered")
.add_attribute(Attribute::Bold)
.set_alignment(CellAlignment::Right),
);
headers.push(
Cell::new("Disk Filtered")
.add_attribute(Attribute::Bold)
.set_alignment(CellAlignment::Right),
);
}
headers.push(
Cell::new("Blocks") Cell::new("Blocks")
.add_attribute(Attribute::Bold) .add_attribute(Attribute::Bold)
.set_alignment(CellAlignment::Right), .set_alignment(CellAlignment::Right),
]); );
table.set_header(headers);
for child in children { for child in children {
let name = child.path.file_name().unwrap_or_default().to_string_lossy(); let name = child.path.file_name().unwrap_or_default().to_string_lossy();
@@ -450,25 +630,100 @@ fn print_text(
Color::Cyan Color::Cyan
}; };
let child_byte = Byte::from_u64(child.size_bytes); if child.accessible {
let child_byte = Byte::from_u64(child.size_bytes).get_appropriate_unit(unit_type);
let disk_usage_byte =
Byte::from_u64(child.blocks * 512).get_appropriate_unit(unit_type);
table.add_row(vec![ let mut row = vec![
Cell::new(name).fg(color), Cell::new(name).fg(color),
Cell::new(&child.entry_type).fg(Color::Yellow), Cell::new(&child.entry_type).fg(Color::Yellow),
Cell::new( Cell::new(format!(
child_byte "{:.precision$} {}",
.get_appropriate_unit(UnitType::Binary) child_byte.get_value(),
.to_string(), child_byte.get_unit(),
) precision = args.precision
))
.fg(Color::Green) .fg(Color::Green)
.set_alignment(CellAlignment::Right), .set_alignment(CellAlignment::Right),
Cell::new(child.size_bytes.to_string()) // Raw bytes instead of "Decimal" unit Cell::new(format!(
"{:.precision$} {}",
disk_usage_byte.get_value(),
disk_usage_byte.get_unit(),
precision = args.precision
))
.fg(Color::Green) .fg(Color::Green)
.set_alignment(CellAlignment::Right), .set_alignment(CellAlignment::Right),
];
if args.compare {
let child_byte_filtered =
Byte::from_u64(child.size_bytes_filtered).get_appropriate_unit(unit_type);
let disk_usage_byte_filtered =
Byte::from_u64(child.blocks_filtered * 512).get_appropriate_unit(unit_type);
row.push(
Cell::new(format!(
"{:.precision$} {}",
child_byte_filtered.get_value(),
child_byte_filtered.get_unit(),
precision = args.precision
))
.fg(Color::Cyan)
.set_alignment(CellAlignment::Right),
);
row.push(
Cell::new(format!(
"{:.precision$} {}",
disk_usage_byte_filtered.get_value(),
disk_usage_byte_filtered.get_unit(),
precision = args.precision
))
.fg(Color::Cyan)
.set_alignment(CellAlignment::Right),
);
}
row.push(
Cell::new(child.blocks) Cell::new(child.blocks)
.fg(Color::Green) .fg(Color::Green)
.set_alignment(CellAlignment::Right), .set_alignment(CellAlignment::Right),
]); );
table.add_row(row);
} else {
let mut row = vec![
Cell::new(name).fg(color),
Cell::new(&child.entry_type).fg(Color::Yellow),
Cell::new("N/A")
.fg(Color::Red)
.set_alignment(CellAlignment::Right),
Cell::new("Access Denied")
.fg(Color::Red)
.set_alignment(CellAlignment::Right),
];
if args.compare {
row.push(
Cell::new("-")
.fg(Color::Red)
.set_alignment(CellAlignment::Right),
);
row.push(
Cell::new("-")
.fg(Color::Red)
.set_alignment(CellAlignment::Right),
);
}
row.push(
Cell::new("-")
.fg(Color::Red)
.set_alignment(CellAlignment::Right),
);
table.add_row(row);
}
} }
writeln!(writer, "{}", table).ok(); writeln!(writer, "{}", table).ok();
} }
@@ -496,8 +751,11 @@ fn print_json(writer: &mut dyn Write, parent_node: &Node, children: &[&Node]) {
metrics: JsonMetrics { metrics: JsonMetrics {
size_bytes: c.size_bytes, size_bytes: c.size_bytes,
blocks: c.blocks, blocks: c.blocks,
size_bytes_filtered: c.size_bytes_filtered,
blocks_filtered: c.blocks_filtered,
}, },
entry_type: c.entry_type.to_string(), entry_type: c.entry_type.to_string(),
accessible: c.accessible,
}) })
.collect(); .collect();
@@ -505,7 +763,10 @@ fn print_json(writer: &mut dyn Write, parent_node: &Node, children: &[&Node]) {
path: &parent_node.path, path: &parent_node.path,
total_size: parent_node.size_bytes, total_size: parent_node.size_bytes,
blocks: parent_node.blocks, blocks: parent_node.blocks,
total_size_filtered: parent_node.size_bytes_filtered,
blocks_filtered: parent_node.blocks_filtered,
entries: entries_view, entries: entries_view,
accessible: parent_node.accessible,
}; };
serde_json::to_writer(&mut *writer, &report).ok(); serde_json::to_writer(&mut *writer, &report).ok();
@@ -541,7 +802,7 @@ mod tests {
.write_all(b"Hello") .write_all(b"Hello")
.unwrap(); .unwrap();
let node = build_tree(temp_dir.path(), false); let node = build_tree(temp_dir.path(), false, false);
assert_eq!(node.entry_type, EntryType::Dir); assert_eq!(node.entry_type, EntryType::Dir);
assert!(!node.children.is_empty()); assert!(!node.children.is_empty());