Commit graph

33 commits

Author SHA1 Message Date
ehu shubham shaw
de3ec87d16
Ehu shubham shaw contribution --> Hardware support (#306)
* feat: add ZeroClaw firmware for ESP32 and Nucleo

* Introduced new firmware for ZeroClaw on ESP32 and Nucleo-F401RE, enabling JSON-over-serial communication for GPIO control.
* Added `zeroclaw-esp32` with support for commands like `gpio_read` and `gpio_write`, along with capabilities reporting.
* Implemented `zeroclaw-nucleo` firmware with similar functionality for STM32, ensuring compatibility with existing ZeroClaw protocols.
* Updated `.gitignore` to include new firmware targets and added necessary dependencies in `Cargo.toml` for both platforms.
* Created README files for both firmware projects detailing setup, build, and usage instructions.

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>

* feat: enhance hardware peripheral support and documentation

- Added `Peripheral` trait implementation in `src/peripherals/` to manage hardware boards (STM32, RPi GPIO).
- Updated `AGENTS.md` to include new extension points for peripherals and their configuration.
- Introduced comprehensive documentation for adding boards and tools, including a quick start guide and supported boards.
- Enhanced `Cargo.toml` to include optional dependencies for PDF extraction and peripheral support.
- Created new datasheets for Arduino Uno, ESP32, and Nucleo-F401RE, detailing pin aliases and GPIO usage.
- Implemented new tools for hardware memory reading and board information retrieval in the agent loop.

This update significantly improves the integration and usability of hardware peripherals within the ZeroClaw framework.

* feat: add ZeroClaw firmware for ESP32 and Nucleo

* Introduced new firmware for ZeroClaw on ESP32 and Nucleo-F401RE, enabling JSON-over-serial communication for GPIO control.
* Added `zeroclaw-esp32` with support for commands like `gpio_read` and `gpio_write`, along with capabilities reporting.
* Implemented `zeroclaw-nucleo` firmware with similar functionality for STM32, ensuring compatibility with existing ZeroClaw protocols.
* Updated `.gitignore` to include new firmware targets and added necessary dependencies in `Cargo.toml` for both platforms.
* Created README files for both firmware projects detailing setup, build, and usage instructions.

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>

* feat: enhance hardware peripheral support and documentation

- Added `Peripheral` trait implementation in `src/peripherals/` to manage hardware boards (STM32, RPi GPIO).
- Updated `AGENTS.md` to include new extension points for peripherals and their configuration.
- Introduced comprehensive documentation for adding boards and tools, including a quick start guide and supported boards.
- Enhanced `Cargo.toml` to include optional dependencies for PDF extraction and peripheral support.
- Created new datasheets for Arduino Uno, ESP32, and Nucleo-F401RE, detailing pin aliases and GPIO usage.
- Implemented new tools for hardware memory reading and board information retrieval in the agent loop.

This update significantly improves the integration and usability of hardware peripherals within the ZeroClaw framework.

* feat: Introduce hardware auto-discovery and expanded configuration options for agents, hardware, and security.

* chore: update dependencies and improve probe-rs integration

- Updated `Cargo.lock` to remove specific version constraints for several dependencies, including `zerocopy`, `syn`, and `strsim`, allowing for more flexibility in version resolution.
- Upgraded `bincode` and `bitfield` to their latest versions, enhancing serialization and memory management capabilities.
- Updated `Cargo.toml` to reflect the new version of `probe-rs` from `0.24` to `0.30`, improving hardware probing functionality.
- Refactored code in `src/hardware` and `src/tools` to utilize the new `SessionConfig` for session management in `probe-rs`, ensuring better compatibility and performance.
- Cleaned up documentation in `docs/datasheets/nucleo-f401re.md` by removing unnecessary lines.

* fix: apply cargo fmt

* docs: add hardware architecture diagram.

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-16 11:40:10 -05:00
Chummy
23b0f360c2
fix(composio): align v3 execute path and honor configured entity_id (#322) 2026-02-16 23:40:37 +08:00
Chummy
80da3e64e9
feat: unify scheduled tasks from #337 and #338 with security-first integration
Unifies scheduled task capabilities and consolidates overlapping implementations from #337 and #338 into a single security-first integration path.\n\nCo-authored-by: Edvard <ecschoye@stud.ntnu.no>\nCo-authored-by: stawky <stakeswky@gmail.com>
2026-02-16 23:38:29 +08:00
chumyin
2d6ec2fb71 fix(rebase): resolve PR #266 conflicts against latest main 2026-02-16 19:33:04 +08:00
chumyin
34306e32d8 fix(provider): complete ChatResponse integration across runtime surfaces 2026-02-16 19:18:12 +08:00
chumyin
3b4a4de457 refactor(provider): unify Provider responses with ChatResponse
- Switch Provider trait methods to return structured ChatResponse
- Map OpenAI-compatible tool_calls into shared ToolCall type
- Update reliable/router wrappers and provider tests for new interface
- Make agent loop prefer structured tool calls with text fallback parsing
- Adapt gateway replies to structured responses with safe tool-call fallback
2026-02-16 19:16:22 +08:00
Vernon Stinebaker
40c41cf3d2
feat(discord): add listen_to_bots config and fix model IDs across codebase (#280)
* fix(config): apply env overrides at runtime and fix Docker compose defaults

- Call apply_env_overrides() after Config::load_or_init() in main.rs so
  environment variables (API_KEY, PROVIDER, ZEROCLAW_GATEWAY_PORT, etc.)
  are actually applied at runtime, not just in tests
- Add ZEROCLAW_ALLOW_PUBLIC_BIND env var support for gateway bind policy
- Fix docker-compose.yml: correct volume path (/zeroclaw-data not /data),
  add ZEROCLAW_ALLOW_PUBLIC_BIND=true for container networking, make host
  port configurable via HOST_PORT env var
- Add docker-compose.override.yml to .gitignore for local dev overrides

* feat(discord): add listen_to_bots config and fix model IDs across codebase

Add listen_to_bots field to DiscordConfig so bot messages are processed
when explicitly enabled (defaults to false for backward compat). Remove
ZEROCLAW_MODEL from Dockerfile release stage so config.toml is the
source of truth for model selection. Fix all hardcoded model IDs from
the dated anthropic/claude-sonnet-4-20250514 to the valid OpenRouter
identifier anthropic/claude-sonnet-4.
2026-02-16 02:13:36 -05:00
Chummy
b442a07530
fix(memory): prevent autosave key collisions across runtime flows
Fixes #221 - SQLite Memory Override bug.

This PR resolves memory overwrite behavior in autosave paths by replacing fixed memory keys with unique keys, and improves short-horizon recall quality in channel runtime.

**Root Cause**
SQLite memory uses a unique constraint on `memories.key` and writes with `ON CONFLICT(key) DO UPDATE`.
Several autosave paths reused fixed keys (or sender-stable keys), so newer messages overwrote earlier conversation entries.

**Changes**
- Channel runtime: autosave key changed from `channel_sender` to `channel_sender_messageId`
- Added memory-context injection before provider calls (aligned with agent loop behavior)
- Agent loop: autosave keys changed from fixed `user_msg`/`assistant_resp` to UUID-suffixed keys
- Gateway: Webhook/WhatsApp autosave keys changed to UUID-suffixed keys

All CI checks passing.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-15 22:55:52 -05:00
Chummy
b0e1e32819
feat(config): make config writes atomic with rollback-safe replacement (#190)
* feat(runtime): add Docker runtime MVP and runtime-aware command builder

* feat(security): add shell risk classification, approval gates, and action throttling

* feat(gateway): add per-endpoint rate limiting and webhook idempotency

* feat(config): make config writes atomic with rollback-safe replacement

---------

Co-authored-by: chumyin <chumyin@users.noreply.github.com>
2026-02-15 12:18:45 -05:00
Argenis
1cfc63831c
feat(providers): add multi-model router for task-based provider routing
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-15 11:40:58 -05:00
Edvard Schøyen
6725eb2995
fix(gateway): use constant-time comparison for WhatsApp verify_token
Uses constant_time_eq for verify_token to prevent timing attacks. Removes unused whatsapp_app_secret signature verification code for simplification.
2026-02-15 07:42:52 -05:00
argenis de la rosa
47c5006de4 Merge remote-tracking branch 'origin/feat/whatsapp-email-channels'
# Conflicts:
#	Cargo.lock
#	src/config/schema.rs
#	src/cron/mod.rs
#	src/security/secrets.rs
#	src/service/mod.rs
2026-02-15 06:37:51 -05:00
Argenis
5cc02c5813
fix: add WhatsApp webhook signature verification (X-Hub-Signature-256)
Closes #51

- Add HMAC-SHA256 signature verification for WhatsApp webhooks
- Prevents message spoofing attacks (CWE-345)
- Add whatsapp_app_secret config field with ZEROCLAW_WHATSAPP_APP_SECRET env override
- Add 13 comprehensive unit tests

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-15 06:17:24 -05:00
fettpl
0247ac13e8 Merge remote-tracking branch 'origin/main' into fix/gateway-timeout-layer 2026-02-15 02:15:28 +01:00
fettpl
8a304505df fix: apply TimeoutLayer to gateway router for request timeouts
Add tower-http TimeoutLayer with the existing REQUEST_TIMEOUT_SECS (30s)
constant and 408 Request Timeout status code. Previously, the constant
was defined but no timeout middleware was applied, allowing slow
requests to hold connections indefinitely (slow-loris risk).

Closes #60

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-15 00:04:05 +01:00
fettpl
25e5f670bb fix: stop leaking LLM error details to HTTP clients and WhatsApp users
Log full error details server-side with tracing::error! and return
generic messages to clients. Previously, the raw anyhow error chain
(which could include provider URLs, HTTP status codes, or partial
request bodies) was forwarded to end users.

Closes #59

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-14 23:53:39 +01:00
argenis de la rosa
5476195a7f refactor: remove AIEOS identity support
- Remove src/identity/ directory (aieos.rs, mod.rs)
- Remove IdentityConfig struct and identity field from Config
- Remove build_system_prompt_with_identity and load_aieos_from_config functions
- Remove AIEOS-related imports from channels/mod.rs
- Remove identity module declarations from main.rs and lib.rs
- Remove AIEOS tests from config/schema.rs
- Keep OpenClaw markdown-based identity as the only supported format

This simplifies the codebase by removing unused AIEOS complexity.
All 832 tests pass.
2026-02-14 14:05:14 -05:00
argenis de la rosa
03dd9712ca style: clean up formatting and fix gateway tests
- Remove extra blank line in main.rs
- Format symlink_tests.rs with consistent spacing
- Remove problematic axum-specific security tests from gateway module
- Keep only TCP-compatible tests for gateway functionality
- All 840 tests passing with clean formatting
2026-02-14 13:41:13 -05:00
argenis de la rosa
dbf02291b4 fix: escape AppleScript target parameter in iMessage channel
- Add escape_applescript() function to prevent injection attacks
- Add is_valid_imessage_target() validation for phone/email patterns
- Update send() method to escape both message AND target parameters
- Add 40 comprehensive tests covering injection edge cases
- Addresses CWE-78 (OS Command Injection) vulnerability

Fixes #29
2026-02-14 13:38:13 -05:00
argenis de la rosa
ef4444ba43 fix: resolve build errors and add comprehensive symlink tests
- Fixed E0425 error in src/skills/mod.rs by moving println! inside #[cfg(unix)] block where 'dest' variable is in scope
- Added missing 'identity' field to Config struct initializations in src/onboard/wizard.rs
- Fixed import paths for AIEOS identity functions in src/channels/mod.rs
- Added comprehensive symlink edge case tests in src/skills/symlink_tests.rs
- All 840 tests passing, 0 clippy warnings

Resolves issue #28: skills symlink functionality now works correctly on Unix platforms with proper error handling on non-Unix platforms
2026-02-14 13:37:27 -05:00
argenis de la rosa
76074cb789 fix: run Docker container as non-root user (closes #34)
- Switch to gcr.io/distroless/cc-debian12:nonroot
- Add explicit USER 65534:65534 directive
- Add Docker security CI job verifying non-root UID, :nonroot base, and USER directive
- Document CIS Docker Benchmark compliance in SECURITY.md
- Add tests and edge cases for container security
2026-02-14 13:16:33 -05:00
argenis de la rosa
cc08f4bfff feat: Add full WhatsApp Business Cloud API integration
- Add WhatsApp channel module with Cloud API v18.0 support
- Implement webhook-based message reception and API sending
- Add allowlist for phone numbers (E.164 format or wildcard)
- Add WhatsApp webhook endpoints to gateway (/whatsapp GET/POST)
- Add WhatsApp config schema with TOML support
- Wire WhatsApp into channel factory, CLI, and doctor commands
- Add WhatsApp to setup wizard with connection testing
- Add comprehensive test coverage (47 channel tests + 9 URL decoding tests)
- Update README with detailed WhatsApp setup instructions
- Support text messages only, skip media/status updates
- Normalize phone numbers with + prefix
- Handle webhook verification with Meta challenge-response

All 756 tests pass. Ready for production use.
2026-02-14 13:10:16 -05:00
argenis de la rosa
ec2d5cc93d feat: enhance agent personality, tool guidance, and memory hygiene
- Expand communication style presets (professional, expressive, custom)
- Enrich SOUL.md with human-like tone and emoji-awareness guidance
- Add crash recovery and sub-task scoping guidance to AGENTS.md scaffold
- Add 'Use when / Don't use when' guidance to TOOLS.md and runtime prompts
- Implement memory hygiene system with configurable archiving and retention
- Add MemoryConfig options: hygiene_enabled, archive_after_days, purge_after_days, conversation_retention_days
- Archive old daily memory and session files to archive subdirectories
- Purge old archives and prune stale SQLite conversation rows
- Add comprehensive tests for new features
2026-02-14 11:28:39 -05:00
argenis de la rosa
1fd51f1984 fix: resolve all clippy --all-targets warnings across 15 files
- gateway/mod.rs: move send_json before test module (items_after_test_module)
- memory/vector.rs: fix float_cmp, cast_precision_loss, approx_constant
- memory/chunker.rs: fix format_collect, format_push_string, write_with_newline
- memory/sqlite.rs: fix useless_vec
- heartbeat/engine.rs: fix format_collect, write_with_newline
- config/schema.rs: fix needless_raw_string_hashes
- tools/composio.rs: fix needless_raw_string_hashes
- integrations/registry.rs: fix uninlined_format_args, unused import
- tunnel/mod.rs: fix doc_markdown
- skills/mod.rs: allow similar_names in test module
- channels/cli.rs: fix unreadable_literal
- observability/mod.rs: fix manual_string_new
- runtime/mod.rs: fix manual_string_new
- examples/custom_memory.rs: add Default impl (new_without_default)
- examples/custom_channel.rs: fix needless_borrows_for_generic_args
2026-02-14 03:52:57 -05:00
argenis de la rosa
976c5bbf3c hardening: fix 7 production weaknesses found in codebase scan
Scan findings and fixes:

1. Gateway buffer overflow (8KB → 64KB)
   - Fixed: Increased request buffer from 8,192 to 65,536 bytes
   - Large POST bodies (long prompts) were silently truncated

2. Gateway slow-loris attack (no read timeout → 30s)
   - Fixed: tokio::time::timeout(30s) on stream.read()
   - Malicious clients could hold connections indefinitely

3. Webhook secret timing attack (== → constant_time_eq)
   - Fixed: Now uses constant_time_eq() for secret comparison
   - Prevents timing side-channel on webhook authentication

4. Pairing brute force (no limit → 5 attempts + 5min lockout)
   - Fixed: PairingGuard tracks failed attempts with lockout
   - Returns 429 Too Many Requests with retry_after seconds

5. Shell tool hang (no timeout → 60s kill)
   - Fixed: tokio::time::timeout(60s) on Command::output()
   - Commands that hang are killed and return error

6. Shell tool OOM (unbounded output → 1MB cap)
   - Fixed: stdout/stderr truncated at 1MB with warning
   - Prevents memory exhaustion from verbose commands

7. Provider HTTP timeout (none → 120s request + 10s connect)
   - Fixed: All 5 providers (OpenRouter, Anthropic, OpenAI,
     Ollama, Compatible) now have reqwest timeouts
   - Ollama gets 300s (local models are slower)

949 tests passing, 0 clippy warnings, cargo fmt clean
2026-02-14 01:47:08 -05:00
argenis de la rosa
c8d4ceee71 feat: add port 0 (random port) support for gateway security
When --port 0 is passed, the OS assigns a random available ephemeral
port (typically 49152-65535). The actual port is resolved after binding
and used for all log output and tunnel forwarding.

This prevents port-scanning attacks against a known fixed port.

Changes:
  src/gateway/mod.rs — bind first, extract actual_port from listener,
    use actual_port for addr formatting and tunnel.start()
  src/main.rs — update CLI help text, conditional log for port=0

8 new edge case tests:
  - port_zero_binds_to_random_port
  - port_zero_assigns_different_ports
  - port_zero_assigns_high_port
  - specific_port_binds_exactly
  - actual_port_matches_addr_format
  - port_zero_listener_accepts_connections
  - duplicate_specific_port_fails
  - tunnel_gets_actual_port_not_zero

943 tests passing, 0 clippy warnings, cargo fmt clean
2026-02-14 01:21:55 -05:00
argenis de la rosa
b2aff60722 security: pass all 4 checklist items — gateway not public, pairing required, filesystem scoped, tunnel access
Security checklist from @anshnanda / @ledger_eth:
   Gateway not public — default bind 127.0.0.1, refuses 0.0.0.0 without
     tunnel or explicit allow_public_bind=true in config
   Pairing required — one-time 6-digit code printed on startup, exchanged
     for bearer token via POST /pair, enforced on all /webhook requests
   Filesystem scoped (no /) — workspace_only=true by default, null byte
     injection blocked, 14 system dirs + 4 sensitive dotfiles in forbidden
     list, is_resolved_path_allowed() for symlink escape prevention
   Access via Tailscale/SSH tunnel — tunnel system integrated, gateway
     refuses public bind without active tunnel

New files:
  src/security/pairing.rs — PairingGuard with OTP generation, constant-time
    code comparison, bearer token issuance, token persistence

Changed files:
  src/config/schema.rs — GatewayConfig (require_pairing, allow_public_bind,
    paired_tokens), expanded AutonomyConfig forbidden_paths
  src/config/mod.rs — export GatewayConfig
  src/gateway/mod.rs — public bind guard, pairing enforcement on /webhook,
    /pair endpoint, /health no longer leaks version/memory info
  src/security/policy.rs — null byte blocking, is_resolved_path_allowed(),
    expanded forbidden_paths (14 system dirs + 4 dotfiles)
  src/security/mod.rs — export pairing module
  src/onboard/wizard.rs — wire gateway config

935 tests passing (up from 905), 0 clippy warnings, cargo fmt clean
2026-02-14 00:39:51 -05:00
argenis de la rosa
0e7f501fd6 feat: full-stack search engine — FTS5, vector search, hybrid merge, embedding cache, chunker
The Full Stack (All Custom):
- Vector DB: embeddings stored as BLOB, cosine similarity in pure Rust
- Keyword Search: FTS5 virtual tables with BM25 scoring + auto-sync triggers
- Hybrid Merge: weighted fusion of vector + keyword results (configurable weights)
- Embeddings: provider abstraction (OpenAI, custom URL, noop fallback)
- Chunking: line-based markdown chunker with heading preservation
- Caching: embedding_cache table with LRU eviction
- Safe Reindex: rebuild FTS5 + re-embed missing vectors

New modules:
- src/memory/embeddings.rs — EmbeddingProvider trait + OpenAI + Noop + factory
- src/memory/vector.rs — cosine similarity, vec↔bytes, ScoredResult, hybrid_merge
- src/memory/chunker.rs — markdown-aware document splitting

Upgraded:
- src/memory/sqlite.rs — FTS5 schema, embedding column, hybrid recall, cache, reindex
- src/config/schema.rs — MemoryConfig expanded with embedding/search settings
- All callers updated to pass api_key for embedding provider

739 tests passing, 0 clippy warnings (Rust 1.93.1), cargo-deny clean
2026-02-14 00:00:23 -05:00
argenis de la rosa
390cbc0a6c feat: agnostic tunnel system — bring your own tunnel provider
New Tunnel trait + 5 implementations:
- NoneTunnel: local-only, no external exposure (default)
- CloudflareTunnel: wraps cloudflared binary, extracts public URL
- TailscaleTunnel: tailscale serve (tailnet) or funnel (public)
- NgrokTunnel: wraps ngrok binary, supports custom domains
- CustomTunnel: user-provided command with {port}/{host} placeholders

Config schema:
- [tunnel] section with provider selector
- Provider-specific sub-configs: cloudflare, tailscale, ngrok, custom
- Backward compatible (serde default = "none")

Gateway integration:
- Tunnel starts automatically on 'zeroclaw gateway'
- Prints public URL on success, falls back to local on failure

20 new tests (factory, constructors, NoneTunnel async start/health)
649 tests passing, 0 clippy warnings, cargo fmt clean
2026-02-13 16:25:01 -05:00
argenis de la rosa
bc31e4389b style: cargo fmt — fix all formatting for CI
Ran cargo fmt across entire codebase to pass CI's cargo fmt --check.
No logic changes, only whitespace/formatting.
2026-02-13 16:03:50 -05:00
argenis de la rosa
a5887ad2dc docs+tests: architecture diagram, security docs, 75 new edge-case tests
README:
- Add ASCII architecture flow diagram showing all layers
- Add Security Architecture section (Layer 1: Channel Auth,
  Layer 2: Rate Limiting, Layer 3: Tool Sandbox)
- Update test count to 629

New edge-case tests (75 new):
- SecurityPolicy: command injection (semicolon, backtick, dollar-paren,
  env prefix, newline), path traversal (encoded dots, double-dot in
  filename, null byte, symlink, tilde-ssh, /var/run), rate limiter
  boundaries (exactly-at, zero, high), autonomy+command combos,
  from_config fresh tracker
- Discord: exact match not substring, empty user ID, wildcard+specific,
  case sensitivity, base64 edge cases
- Slack: exact match, empty user ID, case sensitivity, wildcard combo
- Telegram: exact match, empty string, case sensitivity, wildcard combo
- Gateway: first-match-wins, empty value, colon in value, different
  headers, empty request, newline-only request
- Config schema: backward compat (Discord/Slack without allowed_users),
  TOML roundtrip, webhook secret presence/absence

629 tests passing, 0 clippy warnings
2026-02-13 16:00:15 -05:00
argenis de la rosa
542bb80743 security: harden architecture against Moltbot security model
- Discord: add allowed_users field + sender validation in listen()
- Slack: add allowed_users field + sender validation in listen()
- Webhook: add X-Webhook-Secret header auth (401 on mismatch)
- SecurityPolicy: add ActionTracker with sliding-window rate limiting
  - record_action() enforces max_actions_per_hour
  - is_rate_limited() checks without recording
- Gateway: print auth status on startup (ENABLED/DISABLED)
- 22 new tests (Discord/Slack allowlists, gateway header extraction,
  rate limiter: starts at zero, records, allows within limit,
  blocks over limit, clone independence)
- 554 tests passing, 0 clippy warnings
2026-02-13 15:31:21 -05:00
argenis de la rosa
05cb353f7f feat: initial release — ZeroClaw v0.1.0
- 22 AI providers (OpenRouter, Anthropic, OpenAI, Mistral, etc.)
- 7 channels (CLI, Telegram, Discord, Slack, iMessage, Matrix, Webhook)
- 5-step onboarding wizard with Project Context personalization
- OpenClaw-aligned system prompt (SOUL.md, IDENTITY.md, USER.md, AGENTS.md, etc.)
- SQLite memory backend with auto-save
- Skills system with on-demand loading
- Security: autonomy levels, command allowlists, cost limits
- 532 tests passing, 0 clippy warnings
2026-02-13 12:19:14 -05:00