feat(sgx): finish firefly-sparda-fetch — headless FinTS import
End-to-end verified: aqbanking-cli fetches Sparda Südwest transactions via FinTS PIN/TAN + SecureGo+, exports CSV using a custom decimal-amount profile, POSTs to firefly-iii-data-importer's autoupload endpoint, which creates transactions in Firefly III via API. Changes vs. previous WIP commit: - firefly/access_token sops slot for the importer's Firefly III API auth (FIREFLY_III_ACCESS_TOKEN_FILE — was the missing piece causing 401s from the API after the autoupload secret authenticated) - nginx fastcgi_read_timeout=600s on the importer vhost (prevents 504 while PHP-FPM is still processing the batch) - PHP-FPM max_execution_time=600s + memory_limit=512M on the importer pool (PHP's stock 30s aborts mid-import for batches > ~50 transactions) - timer re-enabled, wantedBy=[timers.target] Caveats baked into a code comment: - Sparda online-banking PIN must be [A-Za-z0-9] only. aqbanking 6.8.2's -P pinfile mangles `:`, `+`, `'`, `?`, `@`, `%`, `*`; bank locks the access (3 soft / 9 hard strikes) on rejected attempts. Same applies whenever the sops secret is rotated. - Bulk historical imports beyond the PSD2 90-day window need interactive SCA approval per ~30-day chunk and cannot run from the timer; the daily 35-day rolling window stays inside the no-SCA region. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
parent
74af9fd5ac
commit
e0d2a2f50d
1 changed files with 26 additions and 6 deletions
|
|
@ -40,6 +40,10 @@ in
|
|||
sopsFile = ../../../.secrets/sgx/firefly.yaml;
|
||||
owner = "firefly-iii-data-importer";
|
||||
};
|
||||
"firefly/access_token" = {
|
||||
sopsFile = ../../../.secrets/sgx/firefly.yaml;
|
||||
owner = "firefly-iii-data-importer";
|
||||
};
|
||||
};
|
||||
|
||||
environment.systemPackages = [ pkgs.aqbanking ];
|
||||
|
|
@ -114,12 +118,13 @@ in
|
|||
'';
|
||||
};
|
||||
|
||||
# Timer disabled while we work around aqbanking 6.8.2's broken
|
||||
# `-P pinfile` handling. The fetch service authenticates with a wrong
|
||||
# PIN against the bank — three runs locked the access at Sparda. Do
|
||||
# not re-enable until the auth path is replaced (likely python-fints).
|
||||
# Sparda online-banking PIN must contain only [A-Za-z0-9] — special
|
||||
# chars (`:`, `+`, `'`, `?`, `@`, `%`, `*`) get mangled by aqbanking
|
||||
# 6.8.2's pinfile path and the bank locks the access after a few
|
||||
# rejected attempts (3 soft / 9 hard). Same applies if the secret in
|
||||
# sops is rotated.
|
||||
timers.firefly-sparda-fetch = {
|
||||
wantedBy = [ ];
|
||||
wantedBy = [ "timers.target" ];
|
||||
timerConfig = {
|
||||
OnCalendar = "daily";
|
||||
Persistent = true;
|
||||
|
|
@ -157,12 +162,27 @@ in
|
|||
CAN_POST_AUTOIMPORT = "true";
|
||||
IMPORT_DIR_ALLOWLIST = inbox;
|
||||
AUTO_IMPORT_SECRET_FILE = config.sops.secrets."firefly/auto_import_secret".path;
|
||||
FIREFLY_III_ACCESS_TOKEN_FILE = config.sops.secrets."firefly/access_token".path;
|
||||
};
|
||||
};
|
||||
|
||||
nginx.virtualHosts = {
|
||||
${domain} = vhostBase;
|
||||
${importDomain} = vhostBase;
|
||||
# Importer's autoupload endpoint blocks until the entire batch
|
||||
# finishes — POSTing 100+ transactions takes minutes. Default 60s
|
||||
# fastcgi timeout makes nginx 504 even though PHP-FPM keeps going.
|
||||
${importDomain} = vhostBase // {
|
||||
extraConfig = ''
|
||||
fastcgi_read_timeout 600s;
|
||||
'';
|
||||
};
|
||||
};
|
||||
|
||||
# PHP's stock max_execution_time = 30s aborts large bulk imports
|
||||
# mid-stream. Match the nginx fastcgi_read_timeout above.
|
||||
phpfpm.pools.firefly-iii-data-importer.settings = {
|
||||
"php_admin_value[max_execution_time]" = "600";
|
||||
"php_admin_value[memory_limit]" = "512M";
|
||||
};
|
||||
};
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue