diff --git a/.gitignore b/.gitignore index 0054e88..b7370a8 100644 --- a/.gitignore +++ b/.gitignore @@ -13,6 +13,7 @@ Cargo.lock # Vault related files vault-credentials.txt +vault-credentials.json vault-config/ # Temporary test files diff --git a/CLAUDE.md b/CLAUDE.md new file mode 100644 index 0000000..fe6b9f5 --- /dev/null +++ b/CLAUDE.md @@ -0,0 +1,24 @@ +# Vault-Hier Development Guidelines + +## Build & Test Commands +- Build & run: `cargo build && cargo run` +- Run tests: `cargo test` (or `cargo test -- --nocapture` for verbose output) +- Run single test: `cargo test test_name -- --nocapture` +- Docker test: `./test_docker.sh` (includes vault initialization) +- Local test: `./test_local.sh` (sets up local vault) +- Lint: `cargo clippy -- -D warnings` +- Format: `cargo fmt --all` + +## Code Style Guidelines +- **Formatting**: Follow rustfmt conventions (run `cargo fmt` before committing) +- **Imports**: Group by crate (stdlib → external → internal) +- **Error Handling**: Use `anyhow` with descriptive messages; propagate with `?` or `thiserror` for actionable errors +- **Naming**: Snake case for functions/variables, CamelCase for types +- **Async**: Use Tokio for async runtime with structured task management +- **Logging**: Use `tracing` macros for structured logging (`info!`, `debug!`, `error!`, `warn!`, `trace!`) +- **Documentation**: Document public APIs with doc comments (`///`) + +## Architecture Notes +- Modular design with separate services (document, vault, API) +- Hierarchical signing with department validation +- JWT-based authentication using Vault transit backend diff --git a/Cargo.toml b/Cargo.toml index 3382a3f..3c7dfe2 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -9,4 +9,9 @@ tokio = { version = "1.28.0", features = ["full"] } serde = { version = "1.0.160", features = ["derive"] } serde_json = "1.0.96" anyhow = "1.0.70" - +axum = { version = "0.6.18", features = ["multipart"] } +uuid = { version = "1.3.0", features = ["v4", "serde"] } +sha2 = "0.10.6" +base64 = "0.21.0" +tracing = "0.1.37" +tracing-subscriber = { version = "0.3.17", features = ["env-filter"] } diff --git a/README.md b/README.md index 06b65ad..5d5f23e 100644 --- a/README.md +++ b/README.md @@ -1,93 +1,90 @@ -# Vault Hierarchical Initializer +# Hierarchical Document Signing with HashiCorp Vault -A Rust-based utility for initializing and unsealing HashiCorp Vault in non-dev (production) mode. +This project implements a hierarchical document signing system using HashiCorp Vault. It allows for secure document signing with a requirement of a specific number of signatures from different departmental groups. -## Overview +## Features -This project provides a Docker-based solution for: +- **Hierarchical Signing**: Requires 3 of 5 signatures to validate a document, with at least 1 signature from each department +- **Department Structure**: Two departments (Legal and Finance) with 5 users each +- **Document API**: Upload, sign, and verify documents through a RESTful API +- **Vault Integration**: Leverages HashiCorp Vault's Transit engine for cryptographic operations -1. Running a HashiCorp Vault server in non-dev (production) mode -2. Automatically initializing the Vault instance -3. Unsealing the Vault after initialization -4. Storing unseal keys and root token securely +## System Architecture -## Prerequisites +The system consists of: -- Docker and Docker Compose installed on your system -- Rust (if you want to build the project locally) +1. **Vault Server**: Provides secure storage and cryptographic operations +2. **Rust Application**: Initializes Vault and provides a REST API for document operations +3. **User Hierarchy**: 10 users organized into 2 departments +4. **Signature Requirements**: 3 of 5 signatures needed, with at least 1 from each department -## Configuration +## API Endpoints -In production mode, Vault: -- Starts sealed and requires a threshold of unseal keys to unseal -- Stores data persistently in mounted volumes -- Requires explicit initialization -- Needs manual unsealing after restarts +- **POST /api/login**: Authenticate with username/password and get a token +- **POST /api/documents**: Upload a new document for signing +- **GET /api/documents/:id**: Retrieve document metadata +- **POST /api/documents/:id/sign**: Sign a document with your user credentials +- **GET /api/documents/:id/verify**: Check if a document has sufficient signatures -The implementation uses: -- 5 key shares with a threshold of 3 keys needed for unsealing -- Persistent volume storage for Vault data +## Getting Started -## Usage +### Prerequisites -### Starting Vault with Docker Compose +- Docker and Docker Compose +- Rust development environment (if building from source) -```bash -docker-compose up -d -``` +### Running with Docker -This will: -1. Start a Vault server in production mode -2. Run the vault-hier utility to initialize Vault if needed -3. Automatically unseal Vault using the threshold number of keys -4. Save the unseal keys and root token to `vault-credentials.txt` in the mounted volume +1. Start the Vault server and initialization program: + ``` + docker-compose up -d + ``` -### Getting Vault Credentials +2. The service will automatically: + - Initialize Vault (if needed) + - Unseal Vault + - Create 10 users in a hierarchical structure + - Start the API server on port 3000 -After initialization, you can find the unseal keys and root token in: +3. User credentials: + - Legal department: legal1/legal1pass through legal5/legal5pass + - Finance department: finance1/finance1pass through finance5/finance5pass -``` -./vault-credentials.txt -``` +### API Usage Examples -Keep these credentials safe! They provide full access to your Vault instance. +1. **Login**: + ```bash + curl -X POST http://localhost:3000/api/login \ + -H "Content-Type: application/json" \ + -d '{"username":"legal1","password":"legal1pass"}' + ``` -### Restarting a Sealed Vault +2. **Upload Document**: + ```bash + curl -X POST http://localhost:3000/api/documents \ + -F "name=Contract" \ + -F "file=@/path/to/document.pdf" + ``` -If your Vault instance restarts, it will start in a sealed state. To unseal it automatically: +3. **Sign Document**: + ```bash + curl -X POST http://localhost:3000/api/documents/DOCUMENT_ID/sign \ + -H "Content-Type: application/json" \ + -d '{"username":"legal1","token":"USER_TOKEN"}' + ``` -```bash -# Set the unseal keys as environment variables -export VAULT_UNSEAL_KEY_1="your-first-key" -export VAULT_UNSEAL_KEY_2="your-second-key" -export VAULT_UNSEAL_KEY_3="your-third-key" - -# Restart the vault-init container to trigger unsealing -docker-compose restart vault-init -``` - -## Development - -### Building the Project Locally - -```bash -cargo build --release -``` - -### Running Tests - -```bash -cargo test -``` - -### Custom Configuration - -To modify the key sharing threshold: -1. Edit the `init_req` struct in `src/main.rs` -2. Rebuild the Docker image +4. **Verify Document**: + ```bash + curl -X GET http://localhost:3000/api/documents/DOCUMENT_ID/verify + ``` ## Security Considerations -- In a production environment, never store unseal keys on the same machine as Vault -- Consider using a key management solution like Shamir's Secret Sharing -- Rotate root tokens regularly and use appropriate authentication methods \ No newline at end of file +- All cryptographic operations are performed by Vault's Transit engine +- Each user has their own signing key +- Root token should be secured in production environments +- Consider adding TLS for production deployments + +## License + +MIT diff --git a/docker-compose.yml b/docker-compose.yml index a01ee30..b62ab73 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -5,7 +5,7 @@ services: ports: - "8200:8200" environment: - - 'VAULT_LOCAL_CONFIG={"storage": {"file": {"path": "/vault/file"}}, "listener": {"tcp": {"address": "0.0.0.0:8200", "tls_disable": true}}, "ui": true, "disable_mlock": true}' + - 'VAULT_LOCAL_CONFIG={"storage": {"file": {"path": "/vault/file"}}, "listener": {"tcp": {"address": "0.0.0.0:8200", "tls_disable": true}}, "ui": false, "disable_mlock": true}' cap_add: - IPC_LOCK volumes: @@ -45,4 +45,4 @@ volumes: networks: vault-net: - driver: bridge \ No newline at end of file + driver: bridge diff --git a/src/api.rs b/src/api.rs new file mode 100644 index 0000000..f632b54 --- /dev/null +++ b/src/api.rs @@ -0,0 +1,241 @@ +use anyhow::Result; +use axum::{ + extract::{Multipart, Path, State}, + http::StatusCode, + response::{IntoResponse, Response}, + routing::{get, post}, + Json, Router, + Server, +}; +use serde::{Deserialize, Serialize}; +use std::sync::Arc; +use tracing::{info, error, debug, instrument}; + +use crate::document_service::{Document, DocumentService, SignatureVerification}; +use crate::vault_setup::VaultClient; + +// API state containing services +#[derive(Clone)] +pub struct ApiState { + document_service: Arc, + vault_client: Arc, +} + +// API error +#[derive(Debug)] +pub struct ApiError(anyhow::Error); + +// Login request +#[derive(Debug, Deserialize)] +pub struct LoginRequest { + username: String, + password: String, +} + +// Login response +#[derive(Debug, Serialize)] +pub struct LoginResponse { + token: String, +} + +// Sign document request +#[derive(Debug, Deserialize)] +pub struct SignDocumentRequest { + username: String, + token: String, +} + +// API response implementations +impl IntoResponse for ApiError { + fn into_response(self) -> Response { + error!("API error: {}", self.0); + ( + StatusCode::INTERNAL_SERVER_ERROR, + format!("Error: {}", self.0), + ) + .into_response() + } +} + +impl From for ApiError +where + E: Into, +{ + fn from(err: E) -> Self { + Self(err.into()) + } +} + +// Start the API server +#[instrument(skip(vault_addr, root_token))] +pub async fn start_api( + vault_addr: &str, + root_token: &str, + api_port: u16, +) -> Result<()> { + info!("Starting API server on port {}...", api_port); + + // Initialize Vault client + let vault_client = VaultClient::new(vault_addr, root_token); + + // Setup required secrets engines and auth methods + vault_client.setup_secrets_engines().await?; + + // Setup 10 users in hierarchical structure + vault_client.setup_hierarchical_users().await?; + + // Initialize document service + let document_service = DocumentService::new(vault_client.clone()); + + // Create API state + let state = ApiState { + document_service: Arc::new(document_service), + vault_client: Arc::new(vault_client), + }; + + // Setup router + let app = Router::new() + .route("/health", get(health_check)) + .route("/api/login", post(login)) + .route("/api/documents", post(upload_document)) + .route("/api/documents/:id", get(get_document)) + .route("/api/documents/:id/sign", post(sign_document)) + .route("/api/documents/:id/verify", get(verify_document)) + .with_state(state); + + info!("API routes configured"); + + // Get the socket address + let addr = std::net::SocketAddr::from(([0, 0, 0, 0], api_port)); + + // Bind and serve + info!("Serving API at {}", addr); + Server::bind(&addr) + .serve(app.into_make_service()) + .await?; + + Ok(()) +} + +// Health check endpoint +#[instrument] +async fn health_check() -> &'static str { + debug!("Health check endpoint called"); + "OK" +} + +// Login endpoint +#[instrument(skip(state, request), fields(username = %request.username))] +async fn login( + State(state): State, + Json(request): Json, +) -> Result, ApiError> { + info!("Login attempt for user: {}", request.username); + + let token = state.vault_client + .login_user(&request.username, &request.password) + .await?; + + info!("User {} successfully authenticated", request.username); + Ok(Json(LoginResponse { token })) +} + +// Upload document endpoint +#[instrument(skip(state, multipart))] +async fn upload_document( + State(state): State, + mut multipart: Multipart, +) -> Result, ApiError> { + info!("Document upload request received"); + + let mut document_name = String::new(); + let mut document_content = Vec::new(); + + // Process multipart form + while let Some(field) = multipart.next_field().await? { + let name = field.name().unwrap_or("").to_string(); + + if name == "name" { + document_name = field.text().await?; + debug!("Received document name: {}", document_name); + } else if name == "file" { + document_content = field.bytes().await?.to_vec(); + debug!("Received document content: {} bytes", document_content.len()); + } + } + + if document_name.is_empty() || document_content.is_empty() { + error!("Missing document name or content"); + return Err(anyhow::anyhow!("Missing document name or content").into()); + } + + // Upload document + let document_id = state.document_service + .upload_document(&document_name, &document_content) + .await?; + + // Return document metadata + let document = state.document_service + .get_document(&document_id) + .await?; + + info!("Document uploaded successfully with ID: {}", document_id); + Ok(Json(document)) +} + +// Get document endpoint +#[instrument(skip(state))] +async fn get_document( + State(state): State, + Path(document_id): Path, +) -> Result, ApiError> { + info!("Fetching document: {}", document_id); + + let document = state.document_service + .get_document(&document_id) + .await?; + + debug!("Retrieved document {} with {} signatures", + document.id, document.signatures.len()); + + Ok(Json(document)) +} + +// Sign document endpoint +#[instrument(skip(state, request), fields(document_id = %document_id, username = %request.username))] +async fn sign_document( + State(state): State, + Path(document_id): Path, + Json(request): Json, +) -> Result, ApiError> { + info!("Signing request for document {} by user {}", document_id, request.username); + + state.document_service + .sign_document(&document_id, &request.username, &request.token) + .await?; + + let document = state.document_service + .get_document(&document_id) + .await?; + + info!("Document {} successfully signed by {}", document_id, request.username); + Ok(Json(document)) +} + +// Verify document endpoint +#[instrument(skip(state))] +async fn verify_document( + State(state): State, + Path(document_id): Path, +) -> Result, ApiError> { + info!("Verifying document signatures: {}", document_id); + + let verification = state.document_service + .verify_document_signatures(&document_id) + .await?; + + info!("Document {} verification result: {}", + document_id, if verification.is_verified { "VERIFIED" } else { "PENDING" }); + + Ok(Json(verification)) +} diff --git a/src/document_service.rs b/src/document_service.rs new file mode 100644 index 0000000..41f357f --- /dev/null +++ b/src/document_service.rs @@ -0,0 +1,485 @@ +use anyhow::{Context, Result}; +use reqwest::StatusCode; +use serde::{Deserialize, Serialize}; +use serde_json::json; +use sha2::{Sha256, Digest}; +use std::collections::HashMap; +use uuid::Uuid; +use base64::{Engine as _, engine::general_purpose::STANDARD as BASE64}; +use tracing::{info, error, debug, instrument}; + +use crate::vault_setup::{Department, User, VaultClient}; + +// Document status +#[derive(Debug, Clone, Serialize, Deserialize)] +pub enum DocumentStatus { + #[serde(rename = "pending")] + Pending, + #[serde(rename = "verified")] + Verified, +} + +// Document metadata +#[derive(Debug, Clone, Serialize, Deserialize)] +pub struct Document { + pub id: String, + pub name: String, + pub hash: String, + pub status: DocumentStatus, + pub signatures: HashMap, // username -> signature +} + +// Signature verification response +#[derive(Debug, Clone, Serialize, Deserialize)] +pub struct SignatureVerification { + pub document_id: String, + pub is_verified: bool, + pub signatures_count: usize, + pub legal_signatures: usize, + pub finance_signatures: usize, + pub required_signatures: usize, + pub required_legal: usize, + pub required_finance: usize, +} + +// Document service to handle document operations +pub struct DocumentService { + vault_client: VaultClient, +} + +impl DocumentService { + pub fn new(vault_client: VaultClient) -> Self { + DocumentService { vault_client } + } + + // Upload a new document and store its metadata + #[instrument(skip(self, content), fields(document_name = %name))] + pub async fn upload_document(&self, name: &str, content: &[u8]) -> Result { + info!("Uploading new document: {}", name); + + // Generate a unique ID + let id = Uuid::new_v4().to_string(); + debug!("Generated document ID: {}", id); + + // Calculate document hash + let mut hasher = Sha256::new(); + hasher.update(content); + let hash = format!("{:x}", hasher.finalize()); + debug!("Document hash: {}", hash); + + // Create document metadata + let document = Document { + id: id.clone(), + name: name.to_string(), + hash, + status: DocumentStatus::Pending, + signatures: HashMap::new(), + }; + + // Store document metadata in Vault + self.store_document_metadata(&document).await?; + + info!("Document uploaded with ID: {}", id); + Ok(id) + } + + // Store document metadata in Vault + #[instrument(skip(self, document), fields(document_id = %document.id))] + async fn store_document_metadata(&self, document: &Document) -> Result<()> { + debug!("Storing document metadata for {}", document.id); + + let url = format!("{}/v1/documents/data/docs/{}", + self.vault_client.addr, document.id); + + let payload = json!({ + "data": { + "id": document.id, + "name": document.name, + "hash": document.hash, + "status": match document.status { + DocumentStatus::Pending => "pending", + DocumentStatus::Verified => "verified", + }, + "signatures": document.signatures, + } + }); + + let response = self.vault_client.client + .post(&url) + .header("X-Vault-Token", &self.vault_client.token) + .json(&payload) + .send() + .await?; + + match response.status() { + StatusCode::OK | StatusCode::NO_CONTENT => { + info!("Successfully stored document metadata for {}", document.id); + Ok(()) + } + status => { + let error_text = response.text().await?; + error!("Failed to store document metadata: {} - {}", status, error_text); + Err(anyhow::anyhow!("Failed to store document metadata: {} - {}", status, error_text)) + } + } + } + + // Get document metadata + #[instrument(skip(self))] + pub async fn get_document(&self, document_id: &str) -> Result { + debug!("Getting document metadata for {}", document_id); + + let url = format!("{}/v1/documents/data/docs/{}", + self.vault_client.addr, document_id); + + let response = self.vault_client.client + .get(&url) + .header("X-Vault-Token", &self.vault_client.token) + .send() + .await?; + + match response.status() { + StatusCode::OK => { + let json: serde_json::Value = response.json().await?; + + // Extract status + let status_str = json["data"]["data"]["status"] + .as_str() + .context("Failed to extract status")?; + + let status = match status_str { + "pending" => DocumentStatus::Pending, + "verified" => DocumentStatus::Verified, + _ => { + error!("Unknown document status: {}", status_str); + return Err(anyhow::anyhow!("Unknown status: {}", status_str)); + } + }; + + // Extract signatures + let signatures_value = &json["data"]["data"]["signatures"]; + let mut signatures = HashMap::new(); + + if let Some(obj) = signatures_value.as_object() { + for (username, sig) in obj { + if let Some(sig_str) = sig.as_str() { + signatures.insert(username.clone(), sig_str.to_string()); + } + } + } + + let document = Document { + id: json["data"]["data"]["id"].as_str().context("Missing id")?.to_string(), + name: json["data"]["data"]["name"].as_str().context("Missing name")?.to_string(), + hash: json["data"]["data"]["hash"].as_str().context("Missing hash")?.to_string(), + status, + signatures, + }; + + debug!("Retrieved document: {} with {} signatures", document.id, document.signatures.len()); + Ok(document) + } + status => { + let error_text = response.text().await?; + error!("Failed to get document: {} - {}", status, error_text); + Err(anyhow::anyhow!("Failed to get document: {} - {}", status, error_text)) + } + } + } + + // Sign a document with the user's key + #[instrument(skip(self, user_token), fields(document_id = %document_id, username = %username))] + pub async fn sign_document(&self, document_id: &str, username: &str, user_token: &str) -> Result<()> { + info!("Signing document {} by user {}", document_id, username); + + // Get document metadata + let document = self.get_document(document_id).await?; + + // Get user info to verify department + let user = self.vault_client.get_user_info(username).await?; + + // Sign the document hash with user's key + let url = format!("{}/v1/transit/sign/{}", + self.vault_client.addr, username); + + let payload = json!({ + "input": BASE64.encode(document.hash.as_bytes()), + }); + + let response = self.vault_client.client + .post(&url) + .header("X-Vault-Token", user_token) + .json(&payload) + .send() + .await?; + + match response.status() { + StatusCode::OK => { + let json: serde_json::Value = response.json().await?; + let signature = json["data"]["signature"] + .as_str() + .context("Failed to extract signature")? + .to_string(); + + debug!("Generated signature for document {}", document_id); + + // Update document with signature + self.add_signature(document_id, username, &signature).await?; + + // Update department signature record + self.record_department_signature(document_id, &user).await?; + + // Check if document now has enough signatures + self.update_document_status(document_id).await?; + + info!("Document {} signed by {}", document_id, username); + Ok(()) + } + status => { + let error_text = response.text().await?; + error!("Failed to sign document: {} - {}", status, error_text); + Err(anyhow::anyhow!("Failed to sign document: {} - {}", status, error_text)) + } + } + } + + // Add a signature to a document + #[instrument(skip(self, signature), fields(document_id = %document_id, username = %username))] + async fn add_signature(&self, document_id: &str, username: &str, signature: &str) -> Result<()> { + debug!("Adding signature from {} to document {}", username, document_id); + + // Get current document + let mut document = self.get_document(document_id).await?; + + // Add signature + document.signatures.insert(username.to_string(), signature.to_string()); + + // Store updated document + self.store_document_metadata(&document).await?; + + debug!("Added signature from {} to document {}", username, document_id); + Ok(()) + } + + // Record department signature + #[instrument(skip(self), fields(document_id = %document_id, department = ?user.department, username = %user.username))] + async fn record_department_signature(&self, document_id: &str, user: &User) -> Result<()> { + let dept_str = match user.department { + Department::Legal => "legal", + Department::Finance => "finance", + }; + + debug!("Recording {} department signature for document {}", dept_str, document_id); + + let url = format!("{}/v1/documents/data/dept/{}/signatures/{}", + self.vault_client.addr, dept_str, document_id); + + // Check if department signatures already exist + let response = self.vault_client.client + .get(&url) + .header("X-Vault-Token", &self.vault_client.token) + .send() + .await; + + let mut signatures = Vec::new(); + + // If record exists, get current signatures + if let Ok(resp) = response { + if resp.status() == StatusCode::OK { + let json: serde_json::Value = resp.json().await?; + if let Some(array) = json["data"]["data"]["signatures"].as_array() { + for sig in array { + if let Some(sig_str) = sig.as_str() { + signatures.push(sig_str.to_string()); + } + } + } + } + } + + // Add user to signatures if not already present + if !signatures.contains(&user.username) { + signatures.push(user.username.clone()); + } + + // Store updated signatures + let payload = json!({ + "data": { + "signatures": signatures, + } + }); + + let response = self.vault_client.client + .post(&url) + .header("X-Vault-Token", &self.vault_client.token) + .json(&payload) + .send() + .await?; + + match response.status() { + StatusCode::OK | StatusCode::NO_CONTENT => { + info!("Recorded signature for {} in {} department", user.username, dept_str); + Ok(()) + } + status => { + let error_text = response.text().await?; + error!("Failed to record department signature: {} - {}", status, error_text); + Err(anyhow::anyhow!("Failed to record department signature: {} - {}", status, error_text)) + } + } + } + + // Update document status if it has enough signatures + #[instrument(skip(self))] + async fn update_document_status(&self, document_id: &str) -> Result<()> { + debug!("Checking if document {} has enough signatures", document_id); + + // Verify signatures + let verification = self.verify_document_signatures(document_id).await?; + + if verification.is_verified { + // Get current document + let mut document = self.get_document(document_id).await?; + + // Update status + document.status = DocumentStatus::Verified; + + // Store updated document + self.store_document_metadata(&document).await?; + + info!("Document {} marked as verified", document_id); + } else { + debug!( + "Document {} not yet verified. Has {}/{} signatures ({} legal, {} finance)", + document_id, + verification.signatures_count, + verification.required_signatures, + verification.legal_signatures, + verification.required_finance + ); + } + + Ok(()) + } + + // Verify document signatures + #[instrument(skip(self))] + pub async fn verify_document_signatures(&self, document_id: &str) -> Result { + info!("Verifying signatures for document {}", document_id); + + // Get document + let document = self.get_document(document_id).await?; + + // Get signing requirements + let url = format!("{}/v1/documents/data/config/signing_requirements", + self.vault_client.addr); + + let response = self.vault_client.client + .get(&url) + .header("X-Vault-Token", &self.vault_client.token) + .send() + .await?; + + let config = match response.status() { + StatusCode::OK => response.json::().await?, + status => { + let error_text = response.text().await?; + error!("Failed to get signing requirements: {} - {}", status, error_text); + return Err(anyhow::anyhow!("Failed to get signing requirements: {} - {}", status, error_text)); + } + }; + + // Get required signatures + let required_signatures = config["data"]["data"]["total_required"] + .as_u64() + .context("Missing total_required")? as usize; + + // Get department requirements + let mut required_legal = 0; + let mut required_finance = 0; + + if let Some(departments) = config["data"]["data"]["departments"].as_array() { + for dept in departments { + let name = dept["name"].as_str().context("Missing department name")?; + let required = dept["required"].as_u64().context("Missing required")? as usize; + + match name { + "legal" => required_legal = required, + "finance" => required_finance = required, + _ => (), + } + } + } + + // Get department signatures + let legal_signatures = self.get_department_signatures(document_id, "legal").await?; + let finance_signatures = self.get_department_signatures(document_id, "finance").await?; + + // Check if requirements are met + let total_signatures = document.signatures.len(); + let is_verified = total_signatures >= required_signatures && + legal_signatures.len() >= required_legal && + finance_signatures.len() >= required_finance; + + let verification = SignatureVerification { + document_id: document_id.to_string(), + is_verified, + signatures_count: total_signatures, + legal_signatures: legal_signatures.len(), + finance_signatures: finance_signatures.len(), + required_signatures, + required_legal, + required_finance, + }; + + info!( + "Verification result for document {}: verified={}, signatures={}/{}, legal={}/{}, finance={}/{}", + document_id, + verification.is_verified, + verification.signatures_count, + verification.required_signatures, + verification.legal_signatures, + verification.required_legal, + verification.finance_signatures, + verification.required_finance + ); + + Ok(verification) + } + + // Get department signatures for a document + #[instrument(skip(self))] + async fn get_department_signatures(&self, document_id: &str, department: &str) -> Result> { + debug!("Getting {} department signatures for document {}", department, document_id); + + let url = format!("{}/v1/documents/data/dept/{}/signatures/{}", + self.vault_client.addr, department, document_id); + + let response = self.vault_client.client + .get(&url) + .header("X-Vault-Token", &self.vault_client.token) + .send() + .await; + + let mut signatures = Vec::new(); + + // If record exists, get signatures + if let Ok(resp) = response { + if resp.status() == StatusCode::OK { + let json: serde_json::Value = resp.json().await?; + if let Some(array) = json["data"]["data"]["signatures"].as_array() { + for sig in array { + if let Some(sig_str) = sig.as_str() { + signatures.push(sig_str.to_string()); + } + } + } + } + } + + debug!("Found {} signatures for {} department on document {}", + signatures.len(), department, document_id); + + Ok(signatures) + } +} diff --git a/src/lib.rs b/src/lib.rs new file mode 100644 index 0000000..7ce0e6f --- /dev/null +++ b/src/lib.rs @@ -0,0 +1,11 @@ +// Modules that implement our hierarchical signing system +pub mod vault_setup; +pub mod vault_init; +pub mod document_service; +pub mod api; + +// Re-export main components for easier access +pub use vault_setup::VaultClient; +pub use vault_init::initialize_vault; +pub use document_service::DocumentService; +pub use api::start_api; diff --git a/src/main.rs b/src/main.rs index 2460556..2efb99b 100644 --- a/src/main.rs +++ b/src/main.rs @@ -1,412 +1,40 @@ -use anyhow::{Context, Result}; -use reqwest::{Client, StatusCode}; -use serde::{Deserialize, Serialize}; -use std::{ - env, - fs::File, - io::Write, - path::Path, - process::Command, - time::Duration, -}; -use tokio::time::sleep; +use anyhow::Result; +use std::env; +use tracing::{info}; +use tracing_subscriber::{fmt, EnvFilter}; -// Vault API response structures -#[derive(Debug, Deserialize)] -struct InitResponse { - keys: Vec, - keys_base64: Vec, - root_token: String, -} - -#[derive(Debug, Deserialize)] -struct SealStatusResponse { - sealed: bool, - t: u8, - n: u8, - progress: u8, -} - -#[derive(Debug, Serialize)] -struct InitRequest { - secret_shares: u8, - secret_threshold: u8, -} - -#[derive(Debug, Serialize)] -struct UnsealRequest { - key: String, -} - -// Function to save Vault credentials to a file -fn save_credentials(response: &InitResponse, file_path: &str) -> Result<()> { - let mut file = File::create(Path::new(file_path))?; - writeln!(file, "Unseal Keys:")?; - for (i, key) in response.keys.iter().enumerate() { - writeln!(file, "Key {}: {}", i + 1, key)?; - } - writeln!(file, "Base64 Unseal Keys:")?; - for (i, key) in response.keys_base64.iter().enumerate() { - writeln!(file, "Key {}: {}", i + 1, key)?; - } - writeln!(file)?; - writeln!(file, "Root Token: {}", response.root_token)?; - - println!("Credentials saved to {}", file_path); - Ok(()) -} - -// Wait for Vault to become available -async fn wait_for_vault(addr: &str) -> Result<()> { - println!("Waiting for Vault to be ready..."); - - let client = Client::new(); - - for i in 1..=30 { - let health_url = format!("{}/v1/sys/health?standbyok=true&sealedok=true&uninitok=true", addr); - - match client.get(&health_url).timeout(Duration::from_secs(1)).send().await { - Ok(response) => { - let status = response.status().as_u16(); - // Accept any of these status codes as "available" - if matches!(status, 200 | 429 | 472 | 473 | 501 | 503) { - println!("Vault is available! Status code: {}", status); - return Ok(()); - } - - println!("Vault returned unexpected status code: {}", status); - }, - Err(e) => { - println!("Error connecting to Vault: {}", e); - } - } - - if i == 30 { - return Err(anyhow::anyhow!("Timed out waiting for Vault to become available")); - } - - println!("Vault is unavailable - sleeping (attempt {}/30)", i); - sleep(Duration::from_secs(2)).await; - } - - Ok(()) -} - -// Function to copy credentials to a mounted volume if available -fn copy_credentials_to_volume(src_path: &str) -> Result<()> { - println!("Searching for credentials file..."); - - if let Ok(metadata) = std::fs::metadata(src_path) { - if metadata.is_file() { - println!("Found credentials at {}, copying...", src_path); - - // Create the data directory if it doesn't exist - if let Err(e) = std::fs::create_dir_all("/app/data") { - println!("Warning: Couldn't create /app/data directory: {}", e); - } else { - let dest_path = "/app/data/vault-credentials.txt"; - - // Check if source and destination are the same - if src_path == dest_path { - println!("Source and destination are the same file, skipping copy"); - } else { - match std::fs::copy(src_path, dest_path) { - Ok(_) => println!("Credentials saved to {}", dest_path), - Err(e) => println!("Failed to copy credentials: {}", e), - } - } - } - } - } else { - // If the file doesn't exist in the current directory, search for it - let output = Command::new("find") - .args(["/", "-name", "vault-credentials.txt", "-type", "f"]) - .output(); - - match output { - Ok(output) => { - let files = String::from_utf8_lossy(&output.stdout); - let files: Vec<&str> = files.split('\n').filter(|s| !s.is_empty()).collect(); - - if !files.is_empty() { - println!("Found credentials at {}, copying...", files[0]); - - // Create the data directory if it doesn't exist - if let Err(e) = std::fs::create_dir_all("/app/data") { - println!("Warning: Couldn't create /app/data directory: {}", e); - } else { - let dest_path = "/app/data/vault-credentials.txt"; - - // Check if source and destination are the same - if files[0] == dest_path { - println!("Source and destination are the same file, skipping copy"); - } else { - match std::fs::copy(files[0], dest_path) { - Ok(_) => println!("Credentials saved to {}", dest_path), - Err(e) => println!("Failed to copy credentials: {}", e), - } - } - } - } else { - println!("Could not find credentials file"); - } - }, - Err(e) => println!("Failed to search for credentials: {}", e), - } - } - - Ok(()) -} - -async fn check_init_status(client: &Client, addr: &str) -> Result { - println!("Checking if Vault is already initialized..."); - - let response = client - .get(format!("{}/v1/sys/init", addr)) - .send() - .await?; - - if response.status().is_success() { - let status = response.json::().await?; - if let Some(initialized) = status.get("initialized").and_then(|v| v.as_bool()) { - return Ok(initialized); - } - } - - // If we couldn't determine, assume not initialized - Ok(false) -} - -async fn check_seal_status(client: &Client, addr: &str) -> Result { - println!("Checking Vault seal status..."); - - let response = client - .get(format!("{}/v1/sys/seal-status", addr)) - .send() - .await?; - - if response.status().is_success() { - let status = response.json::().await?; - println!("Seal status: sealed={}, threshold={}, shares={}, progress={}", - status.sealed, status.t, status.n, status.progress); - return Ok(status); - } else { - let error_text = response.text().await?; - anyhow::bail!("Failed to get seal status: {}", error_text); - } -} - -async fn init_vault(client: &Client, addr: &str) -> Result { - // First check if already initialized - let initialized = check_init_status(client, addr).await?; - - if initialized { - anyhow::bail!("Vault is already initialized. Cannot re-initialize."); - } - - println!("Initializing Vault..."); - - // Configure with 5 key shares and a threshold of 3 - // This is a standard production configuration, requiring 3 out of 5 keys to unseal - let init_req = InitRequest { - secret_shares: 5, - secret_threshold: 3, - }; - - let response = client - .put(format!("{}/v1/sys/init", addr)) - .json(&init_req) - .send() - .await?; - - match response.status() { - StatusCode::OK => { - let init_response = response.json::().await?; - println!("Vault initialized successfully!"); - Ok(init_response) - } - status => { - let error_text = response.text().await?; - anyhow::bail!("Failed to initialize Vault: {} - {}", status, error_text); - } - } -} - -async fn unseal_vault(client: &Client, addr: &str, unseal_keys: &[String]) -> Result<()> { - // First check the current seal status - let mut seal_status = check_seal_status(client, addr).await?; - - if !seal_status.sealed { - println!("Vault is already unsealed!"); - return Ok(()); - } - - println!("Unsealing Vault..."); - - // We need to provide enough keys to meet the threshold - // The threshold is in seal_status.t - let required_keys = seal_status.t as usize; - - if unseal_keys.len() < required_keys { - anyhow::bail!( - "Not enough unseal keys provided. Need {} keys, but only have {}", - required_keys, - unseal_keys.len() - ); - } - - // Apply each key one at a time until unsealed - for (i, key) in unseal_keys.iter().take(required_keys).enumerate() { - println!("Applying unseal key {}/{}...", i + 1, required_keys); - - let unseal_req = UnsealRequest { - key: key.clone(), - }; - - let response = client - .put(format!("{}/v1/sys/unseal", addr)) - .json(&unseal_req) - .send() - .await?; - - if !response.status().is_success() { - let error_text = response.text().await?; - anyhow::bail!("Failed to apply unseal key: {}", error_text); - } - - // Check the updated seal status - seal_status = check_seal_status(client, addr).await?; - - if !seal_status.sealed { - println!("Vault unsealed successfully after applying {} keys!", i + 1); - return Ok(()); - } - } - - // If we get here, we've applied all keys but Vault is still sealed - if seal_status.sealed { - anyhow::bail!("Applied all available unseal keys, but Vault is still sealed"); - } - - Ok(()) -} +// Import our library +use vault_hier::{start_api, initialize_vault}; #[tokio::main] async fn main() -> Result<()> { + // Initialize tracing + fmt() + .with_env_filter(EnvFilter::from_default_env().add_directive("vault_hier=info".parse()?)) + .with_target(false) + .init(); + // Get Vault address from env var or use default let vault_addr = env::var("VAULT_ADDR").unwrap_or_else(|_| "http://127.0.0.1:8200".to_string()); - let client = Client::new(); - - println!("Vault address: {}", vault_addr); - println!("Connecting to Vault at: {}", vault_addr); - - // Wait for Vault to be available - wait_for_vault(&vault_addr).await?; - - // Get Vault status to display - let health_url = format!("{}/v1/sys/health?standbyok=true&sealedok=true&uninitok=true", vault_addr); - match client.get(&health_url).send().await { - Ok(response) => { - if response.status().is_success() { - let status_text = response.text().await?; - println!("Vault status: {}", status_text); - } - }, - Err(e) => println!("Error getting Vault status: {}", e), - } - - // First check if Vault is already initialized - let initialized = check_init_status(&client, &vault_addr).await?; - - if initialized { - println!("Vault is already initialized."); - - // Check if Vault is sealed - let seal_status = check_seal_status(&client, &vault_addr).await?; - - if seal_status.sealed { - println!("Vault is sealed. Looking for unseal keys..."); - - // Try to load unseal keys from environment variables - let mut unseal_keys = Vec::new(); - for i in 1..=5 { - match env::var(format!("VAULT_UNSEAL_KEY_{}", i)) { - Ok(key) => { - println!("Found unseal key {} from environment", i); - unseal_keys.push(key); - }, - Err(_) => { - println!("Unseal key {} not found in environment", i); - } - } - } - - // If we have unseal keys, try to unseal - if !unseal_keys.is_empty() { - println!("Found {} unseal keys. Attempting to unseal...", unseal_keys.len()); - unseal_vault(&client, &vault_addr, &unseal_keys).await?; - } else { - println!("No unseal keys found. Vault remains sealed."); - println!("To unseal, set VAULT_UNSEAL_KEY_1, VAULT_UNSEAL_KEY_2, etc. environment variables."); - } - } else { - println!("Vault is already unsealed."); - } - } else { - // Initialize Vault - println!("Vault is not initialized. Proceeding with initialization..."); - let init_response = init_vault(&client, &vault_addr).await?; - - // Save credentials to files - println!("Saving credentials to file..."); - let current_dir = std::env::current_dir().context("Failed to get current directory")?; - let cred_path = current_dir.join("vault-credentials.txt"); - save_credentials(&init_response, cred_path.to_str().unwrap())?; - println!("Credentials saved to: {}", cred_path.display()); - - // Also save to /app/data as a backup for Docker volume mounting - if let Ok(()) = std::fs::create_dir_all("/app/data") { - let docker_path = "/app/data/vault-credentials.txt"; - save_credentials(&init_response, docker_path)?; - println!("Backup credentials saved to Docker volume at: {}", docker_path); - } - - println!("========================================="); - println!("IMPORTANT: SAVE THESE CREDENTIALS SECURELY"); - println!("========================================="); - println!("Root Token: {}", init_response.root_token); - println!("Unseal Keys (first 3 of 5 needed to unseal):"); - for (i, key) in init_response.keys_base64.iter().enumerate() { - println!("Key {}: {}", i + 1, key); - } - println!("========================================="); - - // Unseal Vault using the first three keys - let unseal_keys = init_response.keys_base64.iter() - .take(3) // We only need threshold number of keys (3) - .cloned() - .collect::>(); - - unseal_vault(&client, &vault_addr, &unseal_keys).await?; - - println!("Vault is now initialized and unsealed"); - - // Store the root token and unseal keys in environment variables - // Using unsafe block as set_var is now considered unsafe in recent Rust - unsafe { - env::set_var("VAULT_TOKEN", &init_response.root_token); - for (i, key) in init_response.keys_base64.iter().enumerate() { - env::set_var(format!("VAULT_UNSEAL_KEY_{}", i + 1), key); - } - } - - println!("Vault initialization and unseal complete!"); - } - - // Copy credentials to the mounted volume (former docker-entrypoint.sh functionality) - copy_credentials_to_volume("vault-credentials.txt")?; - - println!("Operation complete!"); - + + // Get API port from env var or use default + let api_port = env::var("API_PORT") + .unwrap_or_else(|_| "3000".to_string()) + .parse::() + .unwrap_or(3000); + + info!("Vault address: {}", vault_addr); + info!("Connecting to Vault at: {}", vault_addr); + + // Initialize and unseal Vault, get the root token + let root_token = initialize_vault(&vault_addr).await?; + + info!("Starting hierarchical document signing API..."); + + // Start the hierarchical signing API + start_api(&vault_addr, &root_token, api_port).await?; + + info!("API server shutdown. Exiting."); + Ok(()) -} \ No newline at end of file +} diff --git a/src/vault_init.rs b/src/vault_init.rs new file mode 100644 index 0000000..9d489e1 --- /dev/null +++ b/src/vault_init.rs @@ -0,0 +1,173 @@ +use anyhow::{Context, Result}; +use reqwest::Client; +use std::{ + env, + fs, +}; +use tracing::{info, warn, error, debug}; + +use crate::vault_setup::VaultClient; + +/// Initialize and unseal the Vault, returning the root token for further operations +pub async fn initialize_vault(vault_addr: &str) -> Result { + let client = Client::new(); + + // Wait for Vault to be available + VaultClient::wait_for_vault(vault_addr).await?; + + // Display Vault health status + let health_url = format!("{}/v1/sys/health?standbyok=true&sealedok=true&uninitok=true", vault_addr); + match client.get(&health_url).send().await { + Ok(response) => { + if response.status().is_success() { + let status_text = response.text().await?; + info!("Vault status: {}", status_text); + } + }, + Err(e) => warn!("Error getting Vault status: {}", e), + } + + // First check if Vault is already initialized + let initialized = VaultClient::check_init_status(&client, vault_addr).await?; + let mut root_token = String::new(); + + if initialized { + info!("Vault is already initialized."); + + // Check if Vault is sealed + let seal_status = VaultClient::check_seal_status(&client, vault_addr).await?; + + if seal_status.sealed { + info!("Vault is sealed. Looking for unseal keys..."); + + // Try to load unseal keys from environment variables + let mut unseal_keys = Vec::new(); + for i in 1..=5 { + match env::var(format!("VAULT_UNSEAL_KEY_{}", i)) { + Ok(key) => { + info!("Found unseal key {} from environment", i); + unseal_keys.push(key); + }, + Err(_) => { + debug!("Unseal key {} not found in environment", i); + } + } + } + + // If we have unseal keys, try to unseal + if !unseal_keys.is_empty() { + info!("Found {} unseal keys. Attempting to unseal...", unseal_keys.len()); + VaultClient::unseal_vault(&client, vault_addr, &unseal_keys).await?; + } else { + warn!("No unseal keys found. Vault remains sealed."); + info!("To unseal, set VAULT_UNSEAL_KEY_1, VAULT_UNSEAL_KEY_2, etc. environment variables."); + } + } else { + info!("Vault is already unsealed."); + } + + // Try to load root token from environment or credentials file + match env::var("VAULT_TOKEN") { + Ok(token) => { + info!("Found root token from environment"); + root_token = token; + }, + Err(_) => { + // Try to load from credentials file + if let Ok(contents) = fs::read_to_string("vault-credentials.json") { + if let Ok(creds) = serde_json::from_str::(&contents) { + if let Some(token) = creds["root_token"].as_str() { + info!("Found root token from credentials file"); + root_token = token.to_string(); + } + } + } + } + } + + if root_token.is_empty() { + error!("Unable to find root token. Please set VAULT_TOKEN environment variable or provide vault-credentials.json file."); + anyhow::bail!("Unable to find root token. Please set VAULT_TOKEN environment variable or provide vault-credentials.json file."); + } + } else { + // Initialize Vault + info!("Vault is not initialized. Proceeding with initialization..."); + let init_response = VaultClient::init_vault(&client, vault_addr).await?; + + // Save credentials to files + info!("Saving credentials to files..."); + let current_dir = std::env::current_dir().context("Failed to get current directory")?; + + // Save as JSON (new format) + let json_path = current_dir.join("vault-credentials.json"); + VaultClient::save_credentials(&init_response, json_path.to_str().unwrap())?; + info!("JSON credentials saved to: {}", json_path.display()); + + // Save as text (for backward compatibility) + let text_path = current_dir.join("vault-credentials.txt"); + VaultClient::save_credentials(&init_response, text_path.to_str().unwrap())?; + info!("Text credentials saved to: {}", text_path.display()); + + // Also save to /app/data as a backup for Docker volume mounting + if let Ok(()) = std::fs::create_dir_all("/app/data") { + let docker_json_path = "/app/data/vault-credentials.json"; + VaultClient::save_credentials(&init_response, docker_json_path)?; + info!("Backup JSON credentials saved to Docker volume at: {}", docker_json_path); + + let docker_text_path = "/app/data/vault-credentials.txt"; + VaultClient::save_credentials(&init_response, docker_text_path)?; + info!("Backup text credentials saved to Docker volume at: {}", docker_text_path); + } + + info!("========================================="); + info!("IMPORTANT: SAVE THESE CREDENTIALS SECURELY"); + info!("========================================="); + info!("Root Token: {}", init_response.root_token); + info!("Unseal Keys (first 3 of 5 needed to unseal):"); + for (i, key) in init_response.keys_base64.iter().enumerate() { + info!("Key {}: {}", i + 1, key); + } + info!("========================================="); + + // Unseal Vault using the first three keys + let unseal_keys = init_response.keys_base64.iter() + .take(3) // We only need threshold number of keys (3) + .cloned() + .collect::>(); + + VaultClient::unseal_vault(&client, vault_addr, &unseal_keys).await?; + + info!("Vault initialization and unseal complete!"); + + // Set root token + root_token = init_response.root_token; + } + + // Look for any existing credentials and copy them to the mounted volume + if let Ok(metadata) = std::fs::metadata("vault-credentials.json") { + if metadata.is_file() { + info!("Found JSON credentials file, ensuring it's saved to Docker volume..."); + if let Ok(()) = std::fs::create_dir_all("/app/data") { + match std::fs::copy("vault-credentials.json", "/app/data/vault-credentials.json") { + Ok(_) => info!("JSON credentials saved to Docker volume"), + Err(e) => warn!("Failed to copy JSON credentials: {}", e), + } + } + } + } + + if let Ok(metadata) = std::fs::metadata("vault-credentials.txt") { + if metadata.is_file() { + info!("Found text credentials file, ensuring it's saved to Docker volume..."); + if let Ok(()) = std::fs::create_dir_all("/app/data") { + match std::fs::copy("vault-credentials.txt", "/app/data/vault-credentials.txt") { + Ok(_) => info!("Text credentials saved to Docker volume"), + Err(e) => warn!("Failed to copy text credentials: {}", e), + } + } + } + } + + info!("Vault setup complete!"); + Ok(root_token) +} diff --git a/src/vault_setup.rs b/src/vault_setup.rs new file mode 100644 index 0000000..771b3cf --- /dev/null +++ b/src/vault_setup.rs @@ -0,0 +1,707 @@ +use anyhow::{Context, Result}; +use reqwest::{Client, StatusCode}; +use serde::{Deserialize, Serialize}; +use serde_json::json; +use std::{ + fs::File, + io::Write, + path::Path, + time::Duration, +}; +use tokio::time::sleep; +use tracing::{info, error, debug, instrument}; + +// Vault API response structures +#[derive(Debug, Deserialize)] +pub struct InitResponse { + pub keys: Vec, + pub keys_base64: Vec, + pub root_token: String, +} + +#[derive(Debug, Deserialize)] +pub struct SealStatusResponse { + pub sealed: bool, + pub t: u8, + pub n: u8, + pub progress: u8, +} + +#[derive(Debug, Serialize)] +pub struct InitRequest { + pub secret_shares: u8, + pub secret_threshold: u8, +} + +#[derive(Debug, Serialize)] +pub struct UnsealRequest { + pub key: String, +} + +// Department types for organizational structure +#[derive(Debug, Clone, Serialize, Deserialize)] +pub enum Department { + Legal, + Finance, +} + +// User representation with department info +#[derive(Debug, Clone, Serialize, Deserialize)] +pub struct User { + pub username: String, + pub department: Department, +} + +// VaultClient to interact with Vault API +#[derive(Clone)] +pub struct VaultClient { + pub client: Client, + pub addr: String, + pub token: String, +} + +impl VaultClient { + pub fn new(addr: &str, token: &str) -> Self { + VaultClient { + client: Client::new(), + addr: addr.to_string(), + token: token.to_string(), + } + } + + // Wait for Vault to become available + #[instrument(skip(addr))] + pub async fn wait_for_vault(addr: &str) -> Result<()> { + info!("Waiting for Vault to be ready..."); + + let client = Client::new(); + + for i in 1..=30 { + let health_url = format!("{}/v1/sys/health?standbyok=true&sealedok=true&uninitok=true", addr); + + match client.get(&health_url).timeout(Duration::from_secs(1)).send().await { + Ok(response) => { + let status = response.status().as_u16(); + // Accept any of these status codes as "available" + if matches!(status, 200 | 429 | 472 | 473 | 501 | 503) { + info!("Vault is available! Status code: {}", status); + return Ok(()); + } + + debug!("Vault returned unexpected status code: {}", status); + }, + Err(e) => { + debug!("Error connecting to Vault: {}", e); + } + } + + if i == 30 { + error!("Timed out waiting for Vault to become available"); + return Err(anyhow::anyhow!("Timed out waiting for Vault to become available")); + } + + info!("Vault is unavailable - sleeping (attempt {}/30)", i); + sleep(Duration::from_secs(2)).await; + } + + Ok(()) + } + + // Function to check if Vault is initialized + #[instrument(skip(client, addr))] + pub async fn check_init_status(client: &Client, addr: &str) -> Result { + info!("Checking if Vault is already initialized..."); + + let response = client + .get(format!("{}/v1/sys/init", addr)) + .send() + .await?; + + if response.status().is_success() { + let status = response.json::().await?; + if let Some(initialized) = status.get("initialized").and_then(|v| v.as_bool()) { + return Ok(initialized); + } + } + + // If we couldn't determine, assume not initialized + Ok(false) + } + + // Function to check Vault seal status + #[instrument(skip(client, addr))] + pub async fn check_seal_status(client: &Client, addr: &str) -> Result { + info!("Checking Vault seal status..."); + + let response = client + .get(format!("{}/v1/sys/seal-status", addr)) + .send() + .await?; + + if response.status().is_success() { + let status = response.json::().await?; + info!("Seal status: sealed={}, threshold={}, shares={}, progress={}", + status.sealed, status.t, status.n, status.progress); + return Ok(status); + } else { + let error_text = response.text().await?; + error!("Failed to get seal status: {}", error_text); + anyhow::bail!("Failed to get seal status: {}", error_text); + } + } + + // Function to initialize Vault + #[instrument(skip(client, addr))] + pub async fn init_vault(client: &Client, addr: &str) -> Result { + // First check if already initialized + let initialized = Self::check_init_status(client, addr).await?; + + if initialized { + error!("Vault is already initialized. Cannot re-initialize."); + anyhow::bail!("Vault is already initialized. Cannot re-initialize."); + } + + info!("Initializing Vault..."); + + // Configure with 5 key shares and a threshold of 3 + // This is a standard production configuration, requiring 3 out of 5 keys to unseal + let init_req = InitRequest { + secret_shares: 5, + secret_threshold: 3, + }; + + let response = client + .put(format!("{}/v1/sys/init", addr)) + .json(&init_req) + .send() + .await?; + + match response.status() { + StatusCode::OK => { + let init_response = response.json::().await?; + info!("Vault initialized successfully!"); + Ok(init_response) + } + status => { + let error_text = response.text().await?; + error!("Failed to initialize Vault: {} - {}", status, error_text); + anyhow::bail!("Failed to initialize Vault: {} - {}", status, error_text); + } + } + } + + // Function to unseal Vault + #[instrument(skip(client, addr, unseal_keys))] + pub async fn unseal_vault(client: &Client, addr: &str, unseal_keys: &[String]) -> Result<()> { + // First check the current seal status + let mut seal_status = Self::check_seal_status(client, addr).await?; + + if !seal_status.sealed { + info!("Vault is already unsealed!"); + return Ok(()); + } + + info!("Unsealing Vault..."); + + // We need to provide enough keys to meet the threshold + // The threshold is in seal_status.t + let required_keys = seal_status.t as usize; + + if unseal_keys.len() < required_keys { + error!( + "Not enough unseal keys provided. Need {} keys, but only have {}", + required_keys, + unseal_keys.len() + ); + anyhow::bail!( + "Not enough unseal keys provided. Need {} keys, but only have {}", + required_keys, + unseal_keys.len() + ); + } + + // Apply each key one at a time until unsealed + for (i, key) in unseal_keys.iter().take(required_keys).enumerate() { + info!("Applying unseal key {}/{}...", i + 1, required_keys); + + let unseal_req = UnsealRequest { + key: key.clone(), + }; + + let response = client + .put(format!("{}/v1/sys/unseal", addr)) + .json(&unseal_req) + .send() + .await?; + + if !response.status().is_success() { + let error_text = response.text().await?; + error!("Failed to apply unseal key: {}", error_text); + anyhow::bail!("Failed to apply unseal key: {}", error_text); + } + + // Check the updated seal status + seal_status = Self::check_seal_status(client, addr).await?; + + if !seal_status.sealed { + info!("Vault unsealed successfully after applying {} keys!", i + 1); + return Ok(()); + } + } + + // If we get here, we've applied all keys but Vault is still sealed + if seal_status.sealed { + error!("Applied all available unseal keys, but Vault is still sealed"); + anyhow::bail!("Applied all available unseal keys, but Vault is still sealed"); + } + + Ok(()) + } + + // Function to save Vault credentials to a file + pub fn save_credentials(response: &InitResponse, file_path: &str) -> Result<()> { + // For JSON output + if file_path.ends_with(".json") { + let json = serde_json::json!({ + "keys": response.keys, + "keys_base64": response.keys_base64, + "root_token": response.root_token + }); + + let mut file = File::create(Path::new(file_path))?; + file.write_all(serde_json::to_string_pretty(&json)?.as_bytes())?; + info!("Credentials saved to JSON file: {}", file_path); + return Ok(()); + } + + // For plaintext output (legacy format) + let mut file = File::create(Path::new(file_path))?; + writeln!(file, "Unseal Keys:")?; + for (i, key) in response.keys.iter().enumerate() { + writeln!(file, "Key {}: {}", i + 1, key)?; + } + writeln!(file, "Base64 Unseal Keys:")?; + for (i, key) in response.keys_base64.iter().enumerate() { + writeln!(file, "Key {}: {}", i + 1, key)?; + } + writeln!(file)?; + writeln!(file, "Root Token: {}", response.root_token)?; + + info!("Credentials saved to {}", file_path); + Ok(()) + } + + // Enable required secrets engines + #[instrument(skip(self))] + pub async fn setup_secrets_engines(&self) -> Result<()> { + info!("Setting up Vault secrets engines..."); + + // Enable Transit for document signing + self.enable_secrets_engine("transit", "transit").await?; + + // Enable KV v2 for document storage + self.enable_secrets_engine("kv-v2", "documents").await?; + + // Enable userpass for authentication + self.enable_auth_method("userpass").await?; + + info!("Secrets engines setup complete!"); + Ok(()) + } + + // Enable a secrets engine + #[instrument(skip(self))] + async fn enable_secrets_engine(&self, engine_type: &str, path: &str) -> Result<()> { + info!("Enabling {} secrets engine at {}", engine_type, path); + + let url = format!("{}/v1/sys/mounts/{}", self.addr, path); + let payload = json!({ + "type": engine_type, + }); + + let response = self.client + .post(&url) + .header("X-Vault-Token", &self.token) + .json(&payload) + .send() + .await?; + + match response.status() { + StatusCode::NO_CONTENT | StatusCode::OK => { + info!("Successfully enabled {} engine at {}", engine_type, path); + Ok(()) + } + StatusCode::BAD_REQUEST => { + // Check if already exists + let error_text = response.text().await?; + if error_text.contains("path is already in use") { + info!("Secrets engine already enabled at {}", path); + Ok(()) + } else { + error!("Failed to enable secrets engine: {}", error_text); + Err(anyhow::anyhow!("Failed to enable secrets engine: {}", error_text)) + } + } + status => { + let error_text = response.text().await?; + error!("Failed to enable secrets engine: {} - {}", status, error_text); + Err(anyhow::anyhow!("Failed to enable secrets engine: {} - {}", status, error_text)) + } + } + } + + // Enable an auth method + #[instrument(skip(self))] + async fn enable_auth_method(&self, method: &str) -> Result<()> { + info!("Enabling {} auth method", method); + + let url = format!("{}/v1/sys/auth/{}", self.addr, method); + let payload = json!({ + "type": method, + }); + + let response = self.client + .post(&url) + .header("X-Vault-Token", &self.token) + .json(&payload) + .send() + .await?; + + match response.status() { + StatusCode::NO_CONTENT | StatusCode::OK => { + info!("Successfully enabled {} auth method", method); + Ok(()) + } + StatusCode::BAD_REQUEST => { + // Check if already exists + let error_text = response.text().await?; + if error_text.contains("path is already in use") { + info!("Auth method already enabled at {}", method); + Ok(()) + } else { + error!("Failed to enable auth method: {}", error_text); + Err(anyhow::anyhow!("Failed to enable auth method: {}", error_text)) + } + } + status => { + let error_text = response.text().await?; + error!("Failed to enable auth method: {} - {}", status, error_text); + Err(anyhow::anyhow!("Failed to enable auth method: {} - {}", status, error_text)) + } + } + } + + // Create a new user in Vault and associate with department + #[instrument(skip(self, password))] + pub async fn create_user(&self, username: &str, password: &str, department: Department) -> Result<()> { + info!("Creating user {} in department {:?}", username, department); + + // Step 1: Create a policy for the user + let policy_name = format!("{}-policy", username); + self.create_signing_policy(&policy_name, department.clone()).await?; + + // Step 2: Create the user with userpass auth + let url = format!("{}/v1/auth/userpass/users/{}", self.addr, username); + let payload = json!({ + "password": password, + "policies": [policy_name], + }); + + let response = self.client + .post(&url) + .header("X-Vault-Token", &self.token) + .json(&payload) + .send() + .await?; + + match response.status() { + StatusCode::NO_CONTENT | StatusCode::OK => { + info!("Successfully created user {}", username); + + // Step 3: Create a signing key for the user + self.create_signing_key(username).await?; + + // Step 4: Store user metadata in KV store + self.store_user_metadata(username, department).await?; + + Ok(()) + } + status => { + let error_text = response.text().await?; + error!("Failed to create user: {} - {}", status, error_text); + Err(anyhow::anyhow!("Failed to create user: {} - {}", status, error_text)) + } + } + } + + // Create a signing policy for a user based on their department + #[instrument(skip(self))] + async fn create_signing_policy(&self, policy_name: &str, department: Department) -> Result<()> { + let dept_name = match department { + Department::Legal => "legal", + Department::Finance => "finance", + }; + + // Policy content with specific paths for the department + let policy = format!(r#" + # Allow reading document metadata + path "documents/data/docs/*" {{ + capabilities = ["read"] + }} + + # Allow signing with user's key + path "transit/sign/{{{{identity.entity.name}}}}" {{ + capabilities = ["update"] + }} + + # Allow signature verification + path "transit/verify/*" {{ + capabilities = ["update"] + }} + + # Department-specific path + path "documents/data/dept/{}/signatures/*" {{ + capabilities = ["create", "read", "update"] + }} + "#, dept_name); + + let url = format!("{}/v1/sys/policies/acl/{}", self.addr, policy_name); + let payload = json!({ + "policy": policy, + }); + + let response = self.client + .put(&url) + .header("X-Vault-Token", &self.token) + .json(&payload) + .send() + .await?; + + match response.status() { + StatusCode::NO_CONTENT | StatusCode::OK => { + info!("Successfully created policy {}", policy_name); + Ok(()) + } + status => { + let error_text = response.text().await?; + error!("Failed to create policy: {} - {}", status, error_text); + Err(anyhow::anyhow!("Failed to create policy: {} - {}", status, error_text)) + } + } + } + + // Create a signing key for a user in the Transit engine + #[instrument(skip(self))] + async fn create_signing_key(&self, username: &str) -> Result<()> { + let url = format!("{}/v1/transit/keys/{}", self.addr, username); + let payload = json!({ + "type": "ed25519", + }); + + let response = self.client + .post(&url) + .header("X-Vault-Token", &self.token) + .json(&payload) + .send() + .await?; + + match response.status() { + StatusCode::NO_CONTENT | StatusCode::OK => { + info!("Successfully created signing key for {}", username); + Ok(()) + } + status => { + let error_text = response.text().await?; + error!("Failed to create signing key: {} - {}", status, error_text); + Err(anyhow::anyhow!("Failed to create signing key: {} - {}", status, error_text)) + } + } + } + + // Store user metadata in the KV store + #[instrument(skip(self))] + async fn store_user_metadata(&self, username: &str, department: Department) -> Result<()> { + let dept_str = match department { + Department::Legal => "legal", + Department::Finance => "finance", + }; + + let url = format!("{}/v1/documents/data/users/{}", self.addr, username); + let payload = json!({ + "data": { + "username": username, + "department": dept_str, + } + }); + + let response = self.client + .post(&url) + .header("X-Vault-Token", &self.token) + .json(&payload) + .send() + .await?; + + match response.status() { + StatusCode::NO_CONTENT | StatusCode::OK => { + info!("Successfully stored metadata for user {}", username); + Ok(()) + } + status => { + let error_text = response.text().await?; + error!("Failed to store user metadata: {} - {}", status, error_text); + Err(anyhow::anyhow!("Failed to store user metadata: {} - {}", status, error_text)) + } + } + } + + // Create 10 users with departmental hierarchy - 5 in each department + #[instrument(skip(self))] + pub async fn setup_hierarchical_users(&self) -> Result<()> { + info!("Setting up hierarchical user structure"); + + // Create 5 users in Legal department + for i in 1..=5 { + let username = format!("legal{}", i); + let password = format!("legal{}pass", i); + debug!(username, "Creating Legal department user"); + self.create_user(&username, &password, Department::Legal).await?; + } + + // Create 5 users in Finance department + for i in 1..=5 { + let username = format!("finance{}", i); + let password = format!("finance{}pass", i); + debug!(username, "Creating Finance department user"); + self.create_user(&username, &password, Department::Finance).await?; + } + + // Setup document signing requirements + self.setup_signing_requirements().await?; + + info!("Successfully created 10 users in hierarchical structure!"); + Ok(()) + } + + // Configure document signing requirements + #[instrument(skip(self))] + async fn setup_signing_requirements(&self) -> Result<()> { + info!("Setting up document signing requirements"); + + let url = format!("{}/v1/documents/data/config/signing_requirements", self.addr); + let payload = json!({ + "data": { + "total_required": 3, + "departments": [ + { + "name": "legal", + "required": 1, + "total_users": 5 + }, + { + "name": "finance", + "required": 1, + "total_users": 5 + } + ], + "description": "Requires 3 signatures total, with at least 1 from Legal and 1 from Finance departments" + } + }); + + let response = self.client + .post(&url) + .header("X-Vault-Token", &self.token) + .json(&payload) + .send() + .await?; + + match response.status() { + StatusCode::NO_CONTENT | StatusCode::OK => { + info!("Successfully configured signing requirements"); + Ok(()) + } + status => { + let error_text = response.text().await?; + error!("Failed to configure signing requirements: {} - {}", status, error_text); + Err(anyhow::anyhow!("Failed to configure signing requirements: {} - {}", status, error_text)) + } + } + } + + // Login a user and get their token + #[instrument(skip(self, password))] + pub async fn login_user(&self, username: &str, password: &str) -> Result { + info!("Attempting user login: {}", username); + + let url = format!("{}/v1/auth/userpass/login/{}", self.addr, username); + let payload = json!({ + "password": password, + }); + + let response = self.client + .post(&url) + .json(&payload) + .send() + .await?; + + match response.status() { + StatusCode::OK => { + let json: serde_json::Value = response.json().await?; + let token = json["auth"]["client_token"] + .as_str() + .context("Failed to extract client token")? + .to_string(); + + info!("User {} successfully logged in", username); + Ok(token) + } + status => { + let error_text = response.text().await?; + error!("Failed to login: {} - {}", status, error_text); + Err(anyhow::anyhow!("Failed to login: {} - {}", status, error_text)) + } + } + } + + // Get user info including department + #[instrument(skip(self))] + pub async fn get_user_info(&self, username: &str) -> Result { + debug!("Getting user info for: {}", username); + + let url = format!("{}/v1/documents/data/users/{}", self.addr, username); + + let response = self.client + .get(&url) + .header("X-Vault-Token", &self.token) + .send() + .await?; + + match response.status() { + StatusCode::OK => { + let json: serde_json::Value = response.json().await?; + let department_str = json["data"]["data"]["department"] + .as_str() + .context("Failed to extract department")?; + + let department = match department_str { + "legal" => Department::Legal, + "finance" => Department::Finance, + _ => { + error!("Unknown department: {}", department_str); + return Err(anyhow::anyhow!("Unknown department: {}", department_str)); + } + }; + + debug!("Retrieved user info for {} in {:?} department", username, department); + Ok(User { + username: username.to_string(), + department, + }) + } + status => { + let error_text = response.text().await?; + error!("Failed to get user info: {} - {}", status, error_text); + Err(anyhow::anyhow!("Failed to get user info: {} - {}", status, error_text)) + } + } + } +} diff --git a/test_docker.sh b/test_docker.sh index 12f4b32..1d8697d 100755 --- a/test_docker.sh +++ b/test_docker.sh @@ -1,4 +1,4 @@ -#!/bin/bash +#!/usr/bin/env bash set -eo pipefail # Colors for terminal output @@ -48,6 +48,12 @@ if ! command -v docker-compose > /dev/null 2>&1; then exit 1 fi +# Check if jq is available +if ! command -v jq > /dev/null 2>&1; then + log "ERROR" "jq command not found. Please install jq (JSON processor)." + exit 1 +fi + # Build the Docker image log "INFO" "Building Docker image..." docker-compose build @@ -99,22 +105,28 @@ wait_for_vault_init() { # Wait for vault-init to complete wait_for_vault_init -# Check if vault-credentials.txt was created -if [ -f "vault-credentials.txt" ]; then - log "INFO" "Credentials file was created successfully" +# Check if vault-credentials.json was created +if [ -f "vault-credentials.json" ]; then + log "INFO" "JSON credentials file was created successfully" else - log "ERROR" "Credentials file was not created" + log "ERROR" "JSON credentials file was not created" exit 1 fi -# Verify the content of vault-credentials.txt -if grep -q "Unseal Keys:" vault-credentials.txt && grep -q "Root Token:" vault-credentials.txt; then - log "INFO" "Credentials file contains expected content" +# Verify the content of vault-credentials.json +if jq -e '.keys_base64 | length' vault-credentials.json >/dev/null && \ + jq -e '.root_token' vault-credentials.json >/dev/null; then + log "INFO" "JSON credentials file contains expected content" else - log "ERROR" "Credentials file doesn't contain expected content" + log "ERROR" "JSON credentials file doesn't contain expected content" exit 1 fi +# Also check for backward compatibility +if [ -f "vault-credentials.txt" ]; then + log "INFO" "Text credentials file was also created (for backward compatibility)" +fi + # Verify Vault is unsealed after initial setup vault_status=$(docker-compose exec -T vault env VAULT_ADDR=http://127.0.0.1:8200 vault status -format=json 2>/dev/null || echo '{"sealed": true}') @@ -157,28 +169,34 @@ else echo $vault_status fi -# Extract keys from credentials file and root token -log "INFO" "Extracting unseal keys and root token from credentials file..." -unseal_keys=$(grep "Base64 Unseal Keys:" -A 3 vault-credentials.txt | grep "Key" | awk '{print $3}') -root_token=$(grep "Root Token:" vault-credentials.txt | awk '{print $3}') +# Extract keys and token from JSON credentials file +log "INFO" "Extracting unseal keys and root token from JSON credentials file..." +# Using jq to extract the token +root_token=$(jq -r '.root_token' vault-credentials.json) # First, try running 'vault operator unseal' directly for a more robust test log "INFO" "Attempting to unseal Vault directly with unseal keys..." -key1=$(echo "$unseal_keys" | head -n 1) -key2=$(echo "$unseal_keys" | head -n 2 | tail -n 1) -key3=$(echo "$unseal_keys" | head -n 3 | tail -n 1) +# Use jq to extract the keys directly into an array - more elegant +readarray -t unseal_keys_array < <(jq -r '.keys_base64[0:3][]' vault-credentials.json) -docker-compose exec -T vault env VAULT_ADDR=http://127.0.0.1:8200 vault operator unseal "$key1" -docker-compose exec -T vault env VAULT_ADDR=http://127.0.0.1:8200 vault operator unseal "$key2" -docker-compose exec -T vault env VAULT_ADDR=http://127.0.0.1:8200 vault operator unseal "$key3" +# Apply each key +for key in "${unseal_keys_array[@]}"; do + if [ -n "$key" ]; then + log "INFO" "Applying unseal key: ${key:0:8}..." # Show only first 8 chars for security + docker-compose exec -T vault env VAULT_ADDR=http://127.0.0.1:8200 vault operator unseal "$key" + fi +done # As a fallback, also try running vault-init with environment variables log "INFO" "Starting vault-init with environment variables..." -docker-compose run -e VAULT_ADDR=http://vault:8200 \ - -e VAULT_UNSEAL_KEY_1=$(echo "$unseal_keys" | head -n 1) \ - -e VAULT_UNSEAL_KEY_2=$(echo "$unseal_keys" | head -n 2 | tail -n 1) \ - -e VAULT_UNSEAL_KEY_3=$(echo "$unseal_keys" | head -n 3 | tail -n 1) \ - --rm vault-init +# Use the array to set environment variables +env_vars="-e VAULT_ADDR=http://vault:8200" +for i in "${!unseal_keys_array[@]}"; do + env_vars="$env_vars -e VAULT_UNSEAL_KEY_$((i+1))=${unseal_keys_array[$i]}" +done + +# Run the command with all environment variables +eval "docker-compose run $env_vars --rm vault-init" # Verify Vault is unsealed now vault_status=$(docker-compose exec -T vault env VAULT_ADDR=http://127.0.0.1:8200 vault status -format=json 2>/dev/null || echo '{"sealed": true}') @@ -197,9 +215,16 @@ fi # Test some basic Vault operations log "INFO" "Testing basic Vault operations..." -# Write a secret +# Write a secret using the root token from JSON credentials token_result=$(docker-compose exec -T vault env VAULT_ADDR=http://127.0.0.1:8200 vault login "$root_token" 2>&1) -log "INFO" "Login result: $(echo "$token_result" | grep "Success")" +login_success=$(echo "$token_result" | grep -c "Success" || echo "0") +if [ "$login_success" -gt 0 ]; then + log "INFO" "Successfully logged in with root token" +else + log "ERROR" "Failed to log in with root token" + echo "$token_result" + exit 1 +fi # Enable KV secrets engine enable_result=$(docker-compose exec -T vault env VAULT_ADDR=http://127.0.0.1:8200 vault secrets enable -path=kv kv 2>&1 || echo "KV already enabled") diff --git a/test_local.sh b/test_local.sh index 903eb64..071f85e 100755 --- a/test_local.sh +++ b/test_local.sh @@ -1,4 +1,4 @@ -#!/bin/bash +#!/usr/bin/env bash set -e # Detect OS and handle accordingly @@ -41,6 +41,7 @@ fi echo "Starting Vault server in non-dev mode..." # Create temporary config file +rm -fr /tmp/vault-test/data /tmp/vault-test/config mkdir -p /tmp/vault-test/data /tmp/vault-test/config cat > /tmp/vault-test/config/vault.hcl << EOF @@ -54,7 +55,7 @@ listener "tcp" { } disable_mlock = true -ui = true +ui = false EOF vault server -config=/tmp/vault-test/config/vault.hcl > ./vault_server.log 2>&1 & @@ -70,16 +71,16 @@ sleep 5 # Check if Vault is up and running for i in {1..10}; do - if curl -fs -m 1 http://127.0.0.1:8200/v1/sys/health?standbyok=true\&sealedok=true\&uninitok=true > /dev/null 2>&1; then + if wget -q -O- --no-check-certificate http://127.0.0.1:8200/v1/sys/health?standbyok=true\\&sealedok=true\\&uninitok=true > /dev/null 2>&1; then echo "Vault is up and running!" break fi - + if [ $i -eq 10 ]; then echo "Timed out waiting for Vault to become available" exit 1 fi - + echo "Vault is unavailable - sleeping (attempt $i/10)" sleep 2 done @@ -94,11 +95,11 @@ if [ -f "vault-credentials.txt" ]; then # Extract the unseal keys for demonstration UNSEAL_KEYS=$(grep "Key" vault-credentials.txt | head -n 3 | awk '{print $3}') ROOT_TOKEN=$(grep "Root Token" vault-credentials.txt | awk '{print $3}') - + echo "Root Token: $ROOT_TOKEN" echo "First 3 Unseal Keys (needed for threshold):" echo "$UNSEAL_KEYS" - + # Clean up temporary files rm -f vault-credentials.txt else @@ -115,4 +116,4 @@ rm "$VAULT_PID_FILE" rm -rf /tmp/vault-test rm -f ./vault_server.log -echo "All cleaned up. Test successful!" \ No newline at end of file +echo "All cleaned up. Test successful!"