Bulk imports of 100+ transactions per chunk hit the default 60s
fastcgi timeout on the main Firefly III vhost too — not just the
importer endpoint. The importer's per-transaction API call to Firefly's
/api/v1/transactions can take 20+s on a fresh DB without ANALYZE,
which compounds with the 30s PHP max_execution_time cap.
- nginx fastcgi_read_timeout=600s on both `firefly` and `firefly-import`
vhosts
- php_admin_value[max_execution_time]=600 + memory_limit=512M on both
PHP-FPM pools
- VANITY_URL on the importer now points to the main Firefly III URL
(was wrongly pointing at the importer's own domain, breaking
clickable transaction-show links in importer log messages)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
SQLite was slow under btrfs CoW, and the no-CoW migration path turned
out to be fragile (WAL deletion without checkpoint = data loss). Move
to PostgreSQL on Unix-socket peer auth — no password needed for the
local-host setup, NixOS provisions the database+user declaratively.
Drop the now-unused +C tmpfiles rule on the sqlite directory; the
leftover database.sqlite* files at /var/lib/firefly-iii/storage/database/
are harmless and can be removed manually after switch is verified.
Migration of existing Firefly III data is not preserved by this
commit — fresh-start path: re-register admin, re-issue PAT, re-POST
the bulk CSV through the importer.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Random-write SQLite traffic fragments CoW filesystems quickly. The `h`
tmpfiles directive sets +C on the database directory; new SQLite files
(WAL, SHM, recreated main DB) inherit no-CoW automatically. No-op on
non-btrfs filesystems.
Migration of existing files must be done manually with checkpoint-first:
systemctl stop phpfpm-firefly-iii.service
sqlite3 .../database.sqlite 'PRAGMA wal_checkpoint(TRUNCATE);'
# then recreate main file inside the +C dir
systemctl start phpfpm-firefly-iii.service
Skipping the wal_checkpoint and naively deleting .sqlite-wal will lose
all writes that haven't been checkpointed (PHP-FPM SIGTERM does not
trigger a checkpoint).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
End-to-end verified: aqbanking-cli fetches Sparda Südwest transactions
via FinTS PIN/TAN + SecureGo+, exports CSV using a custom decimal-amount
profile, POSTs to firefly-iii-data-importer's autoupload endpoint, which
creates transactions in Firefly III via API.
Changes vs. previous WIP commit:
- firefly/access_token sops slot for the importer's Firefly III API auth
(FIREFLY_III_ACCESS_TOKEN_FILE — was the missing piece causing 401s
from the API after the autoupload secret authenticated)
- nginx fastcgi_read_timeout=600s on the importer vhost (prevents 504
while PHP-FPM is still processing the batch)
- PHP-FPM max_execution_time=600s + memory_limit=512M on the importer
pool (PHP's stock 30s aborts mid-import for batches > ~50 transactions)
- timer re-enabled, wantedBy=[timers.target]
Caveats baked into a code comment:
- Sparda online-banking PIN must be [A-Za-z0-9] only. aqbanking 6.8.2's
-P pinfile mangles `:`, `+`, `'`, `?`, `@`, `%`, `*`; bank locks the
access (3 soft / 9 hard strikes) on rejected attempts. Same applies
whenever the sops secret is rotated.
- Bulk historical imports beyond the PSD2 90-day window need interactive
SCA approval per ~30-day chunk and cannot run from the timer; the
daily 35-day rolling window stays inside the no-SCA region.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
End-to-end FinTS pipeline against Sparda Südwest is wired up but
disabled — aqbanking 6.8.2's `-P pinfile` flag does not consume the
file content correctly on this build (verified: pinfile bytes match
the manually-typed PIN exactly, yet the bank receives a wrong PIN).
Three rejected attempts locked the access at Sparda; do not re-arm
the timer until the auth path is replaced (likely python-fints).
What works:
- aqbanking config and FinTS dialog (manual PIN entry)
- getaccsepa workaround for HKCAZ "Mussfeld 9160" rejection
- custom CSV profile (decimal amounts + IBAN columns) wired in
- Firefly importer auto-upload settings + sops secret slot
- inbox + profile-symlink tmpfiles
What's broken:
- Headless PIN delivery via aqbanking-cli -P
- Timer left wantedBy=[] so it cannot fire post-deploy
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Lays the groundwork for Sparda-Bank Südwest transaction sync via
direct FinTS (no third-party data proxy). aqbanking-cli in the system
PATH, persistent state at /var/lib/firefly-aqbanking, sops slot for
the online-banking PIN. Initial enrollment must be done interactively
on the host; systemd timer for automated fetches comes in a follow-up.
Files vanishing during transfer is expected for mail directories
where messages are constantly moved.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Runs on sgx so alerts (via Gmail) still work even if mx is down.
Available at https://status.hoyer.world behind nginx with ACME cert.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Added `html`, `json`, and `rss` to the `search.formats` list in `searx.nix`.
- Enhances flexibility by allowing multiple output formats for search results.
- Moved Searx-related settings from `default.nix` and `nginx.nix` to a dedicated `searx.nix` module for improved modularity and maintainability.
- Updated references and ACME certificate configuration to align with the new structure.
- Simplifies management of Searx service and its associated secrets.
- Added a new Sops secret for `searx/secret_key` with a corresponding configuration path.
- Updated Searx settings to include the `secret_key` reference.
- Ensures secure integration of secret management with Searx service.
- Added `search.hoyer.world` to the `extraDomainNames` list for the `internal.hoyer.world` ACME certificate.
- Ensures proper SSL configuration for the new subdomain.
Create 6 new NixOS modules to reduce duplication across system configs:
- hardware/wooting: Wooting keyboard udev rules and Bluetooth compat
- services/nginx-base: Common nginx server settings
- services/acme-base: ACME certificate defaults
- services/xremap: Key remapping with sensible defaults
- system/no-sleep: Disable sleep/suspend/hibernate targets
- system/kernel-tweaks: PM freeze timeout and zram configuration
Update system configuration files to use these new modules.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Replaced `not-detected.nix` import with `lenovo-thinkpad-x1-11th-gen` module from `nixos-hardware`
- Improved hardware compatibility for the 11th generation Lenovo ThinkPad X1
- Added "obsidian" folder with path "~/obsidian" and device list
- Added "sync" folder with path "~/sync" and device list
- Both folders configured with same device list: sgx, S25, x1, m4
- Maintains consistent folder configuration pattern in fileserver setup
- Renamed `setting.main` to `settings.main` in the Postfix module for consistency with configuration standards.
- Ensured proper functionality of service by aligning with expected key structure.
- Renamed and reorganized configuration keys for consistency (`settings` usage).
- Updated Postfix, systemd, and Syncthing configurations to adhere to the standardized format.
- Improved maintainability and readability of NixOS configurations.
- Introduced a new `wyoming.nix` file with service definitions for `faster-whisper` and `piper`.
- Enabled TCP ports `10200` and `10300` in the firewall for service communication.
- Updated SGX configuration to include `wyoming.nix` in system imports.
- Enabled `libvirtd` in virtualization settings to allow libvirt usage.
- Added `libvirtd` to `user.extraGroups` for better permissions and management.
- Reformatted `netatalk` service dependencies for readability.
- Updated `hosts allow` setting to include `100.64.0.` for enhanced network access control.
- Added `services.tailscale.enable = true` to the configurations of SGX, MX, and X1 systems for VPN support.
- Improves secure connectivity and simplifies network management across these systems.
- Added `emailOnFailure.enable` option to metacfg with a default of `false`.
- Enabled email notifications on failure for SGX and MX systems.
- Enhanced `systemd-email-notify` module to support the new configuration.
- Set `boot.tmp.useTmpfs` to `false` in `x86_64-linux/sgx/default.nix`.
- Applied `lib.mkDefault` to `boot.tmp.useTmpfs` in `services/base/default.nix` for consistency.
- Enabled `services.cratedocs-mcp` with firewall access in the SGX module for enhanced functionality.
- Updated multiple Flake lockfile entries to the latest revisions, ensuring access to updated upstream changes.
- Set minimum protocol to SMB2 and enabled extended attribute (EA) support in Samba settings.
- Added `fruit:nfs_aces` and `fruit:wipe_intentionally_left_blank_rfork` options for improved macOS compatibility.
- Changed the `time-machine` key to `TimeMachineBackup` in the Samba share configuration.
- Aligns key naming to standard conventions and improves readability.
- Disabled Netatalk service by setting `enable` to `false`.
- Improved macOS compatibility in Samba with specific `fruit` and `vfs` options.
- Added a new Time Machine share configuration for backups.
Adjusted the virtual_alias_maps to properly include both root and admin email forwarding. Removed unused rootAlias line and ensured the configuration aligns with intended email routing behavior.
