diff --git a/.gitignore b/.gitignore index b7370a8..0054e88 100644 --- a/.gitignore +++ b/.gitignore @@ -13,7 +13,6 @@ Cargo.lock # Vault related files vault-credentials.txt -vault-credentials.json vault-config/ # Temporary test files diff --git a/CLAUDE.md b/CLAUDE.md deleted file mode 100644 index fe6b9f5..0000000 --- a/CLAUDE.md +++ /dev/null @@ -1,24 +0,0 @@ -# Vault-Hier Development Guidelines - -## Build & Test Commands -- Build & run: `cargo build && cargo run` -- Run tests: `cargo test` (or `cargo test -- --nocapture` for verbose output) -- Run single test: `cargo test test_name -- --nocapture` -- Docker test: `./test_docker.sh` (includes vault initialization) -- Local test: `./test_local.sh` (sets up local vault) -- Lint: `cargo clippy -- -D warnings` -- Format: `cargo fmt --all` - -## Code Style Guidelines -- **Formatting**: Follow rustfmt conventions (run `cargo fmt` before committing) -- **Imports**: Group by crate (stdlib → external → internal) -- **Error Handling**: Use `anyhow` with descriptive messages; propagate with `?` or `thiserror` for actionable errors -- **Naming**: Snake case for functions/variables, CamelCase for types -- **Async**: Use Tokio for async runtime with structured task management -- **Logging**: Use `tracing` macros for structured logging (`info!`, `debug!`, `error!`, `warn!`, `trace!`) -- **Documentation**: Document public APIs with doc comments (`///`) - -## Architecture Notes -- Modular design with separate services (document, vault, API) -- Hierarchical signing with department validation -- JWT-based authentication using Vault transit backend diff --git a/Cargo.toml b/Cargo.toml index 3c7dfe2..3382a3f 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -9,9 +9,4 @@ tokio = { version = "1.28.0", features = ["full"] } serde = { version = "1.0.160", features = ["derive"] } serde_json = "1.0.96" anyhow = "1.0.70" -axum = { version = "0.6.18", features = ["multipart"] } -uuid = { version = "1.3.0", features = ["v4", "serde"] } -sha2 = "0.10.6" -base64 = "0.21.0" -tracing = "0.1.37" -tracing-subscriber = { version = "0.3.17", features = ["env-filter"] } + diff --git a/README.md b/README.md index 5d5f23e..06b65ad 100644 --- a/README.md +++ b/README.md @@ -1,90 +1,93 @@ -# Hierarchical Document Signing with HashiCorp Vault +# Vault Hierarchical Initializer -This project implements a hierarchical document signing system using HashiCorp Vault. It allows for secure document signing with a requirement of a specific number of signatures from different departmental groups. +A Rust-based utility for initializing and unsealing HashiCorp Vault in non-dev (production) mode. -## Features +## Overview -- **Hierarchical Signing**: Requires 3 of 5 signatures to validate a document, with at least 1 signature from each department -- **Department Structure**: Two departments (Legal and Finance) with 5 users each -- **Document API**: Upload, sign, and verify documents through a RESTful API -- **Vault Integration**: Leverages HashiCorp Vault's Transit engine for cryptographic operations +This project provides a Docker-based solution for: -## System Architecture +1. Running a HashiCorp Vault server in non-dev (production) mode +2. Automatically initializing the Vault instance +3. Unsealing the Vault after initialization +4. Storing unseal keys and root token securely -The system consists of: +## Prerequisites -1. **Vault Server**: Provides secure storage and cryptographic operations -2. **Rust Application**: Initializes Vault and provides a REST API for document operations -3. **User Hierarchy**: 10 users organized into 2 departments -4. **Signature Requirements**: 3 of 5 signatures needed, with at least 1 from each department +- Docker and Docker Compose installed on your system +- Rust (if you want to build the project locally) -## API Endpoints +## Configuration -- **POST /api/login**: Authenticate with username/password and get a token -- **POST /api/documents**: Upload a new document for signing -- **GET /api/documents/:id**: Retrieve document metadata -- **POST /api/documents/:id/sign**: Sign a document with your user credentials -- **GET /api/documents/:id/verify**: Check if a document has sufficient signatures +In production mode, Vault: +- Starts sealed and requires a threshold of unseal keys to unseal +- Stores data persistently in mounted volumes +- Requires explicit initialization +- Needs manual unsealing after restarts -## Getting Started +The implementation uses: +- 5 key shares with a threshold of 3 keys needed for unsealing +- Persistent volume storage for Vault data -### Prerequisites +## Usage -- Docker and Docker Compose -- Rust development environment (if building from source) +### Starting Vault with Docker Compose -### Running with Docker +```bash +docker-compose up -d +``` -1. Start the Vault server and initialization program: - ``` - docker-compose up -d - ``` +This will: +1. Start a Vault server in production mode +2. Run the vault-hier utility to initialize Vault if needed +3. Automatically unseal Vault using the threshold number of keys +4. Save the unseal keys and root token to `vault-credentials.txt` in the mounted volume -2. The service will automatically: - - Initialize Vault (if needed) - - Unseal Vault - - Create 10 users in a hierarchical structure - - Start the API server on port 3000 +### Getting Vault Credentials -3. User credentials: - - Legal department: legal1/legal1pass through legal5/legal5pass - - Finance department: finance1/finance1pass through finance5/finance5pass +After initialization, you can find the unseal keys and root token in: -### API Usage Examples +``` +./vault-credentials.txt +``` -1. **Login**: - ```bash - curl -X POST http://localhost:3000/api/login \ - -H "Content-Type: application/json" \ - -d '{"username":"legal1","password":"legal1pass"}' - ``` +Keep these credentials safe! They provide full access to your Vault instance. -2. **Upload Document**: - ```bash - curl -X POST http://localhost:3000/api/documents \ - -F "name=Contract" \ - -F "file=@/path/to/document.pdf" - ``` +### Restarting a Sealed Vault -3. **Sign Document**: - ```bash - curl -X POST http://localhost:3000/api/documents/DOCUMENT_ID/sign \ - -H "Content-Type: application/json" \ - -d '{"username":"legal1","token":"USER_TOKEN"}' - ``` +If your Vault instance restarts, it will start in a sealed state. To unseal it automatically: -4. **Verify Document**: - ```bash - curl -X GET http://localhost:3000/api/documents/DOCUMENT_ID/verify - ``` +```bash +# Set the unseal keys as environment variables +export VAULT_UNSEAL_KEY_1="your-first-key" +export VAULT_UNSEAL_KEY_2="your-second-key" +export VAULT_UNSEAL_KEY_3="your-third-key" + +# Restart the vault-init container to trigger unsealing +docker-compose restart vault-init +``` + +## Development + +### Building the Project Locally + +```bash +cargo build --release +``` + +### Running Tests + +```bash +cargo test +``` + +### Custom Configuration + +To modify the key sharing threshold: +1. Edit the `init_req` struct in `src/main.rs` +2. Rebuild the Docker image ## Security Considerations -- All cryptographic operations are performed by Vault's Transit engine -- Each user has their own signing key -- Root token should be secured in production environments -- Consider adding TLS for production deployments - -## License - -MIT +- In a production environment, never store unseal keys on the same machine as Vault +- Consider using a key management solution like Shamir's Secret Sharing +- Rotate root tokens regularly and use appropriate authentication methods \ No newline at end of file diff --git a/docker-compose.yml b/docker-compose.yml index b62ab73..a01ee30 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -5,7 +5,7 @@ services: ports: - "8200:8200" environment: - - 'VAULT_LOCAL_CONFIG={"storage": {"file": {"path": "/vault/file"}}, "listener": {"tcp": {"address": "0.0.0.0:8200", "tls_disable": true}}, "ui": false, "disable_mlock": true}' + - 'VAULT_LOCAL_CONFIG={"storage": {"file": {"path": "/vault/file"}}, "listener": {"tcp": {"address": "0.0.0.0:8200", "tls_disable": true}}, "ui": true, "disable_mlock": true}' cap_add: - IPC_LOCK volumes: @@ -45,4 +45,4 @@ volumes: networks: vault-net: - driver: bridge + driver: bridge \ No newline at end of file diff --git a/src/api.rs b/src/api.rs deleted file mode 100644 index f632b54..0000000 --- a/src/api.rs +++ /dev/null @@ -1,241 +0,0 @@ -use anyhow::Result; -use axum::{ - extract::{Multipart, Path, State}, - http::StatusCode, - response::{IntoResponse, Response}, - routing::{get, post}, - Json, Router, - Server, -}; -use serde::{Deserialize, Serialize}; -use std::sync::Arc; -use tracing::{info, error, debug, instrument}; - -use crate::document_service::{Document, DocumentService, SignatureVerification}; -use crate::vault_setup::VaultClient; - -// API state containing services -#[derive(Clone)] -pub struct ApiState { - document_service: Arc, - vault_client: Arc, -} - -// API error -#[derive(Debug)] -pub struct ApiError(anyhow::Error); - -// Login request -#[derive(Debug, Deserialize)] -pub struct LoginRequest { - username: String, - password: String, -} - -// Login response -#[derive(Debug, Serialize)] -pub struct LoginResponse { - token: String, -} - -// Sign document request -#[derive(Debug, Deserialize)] -pub struct SignDocumentRequest { - username: String, - token: String, -} - -// API response implementations -impl IntoResponse for ApiError { - fn into_response(self) -> Response { - error!("API error: {}", self.0); - ( - StatusCode::INTERNAL_SERVER_ERROR, - format!("Error: {}", self.0), - ) - .into_response() - } -} - -impl From for ApiError -where - E: Into, -{ - fn from(err: E) -> Self { - Self(err.into()) - } -} - -// Start the API server -#[instrument(skip(vault_addr, root_token))] -pub async fn start_api( - vault_addr: &str, - root_token: &str, - api_port: u16, -) -> Result<()> { - info!("Starting API server on port {}...", api_port); - - // Initialize Vault client - let vault_client = VaultClient::new(vault_addr, root_token); - - // Setup required secrets engines and auth methods - vault_client.setup_secrets_engines().await?; - - // Setup 10 users in hierarchical structure - vault_client.setup_hierarchical_users().await?; - - // Initialize document service - let document_service = DocumentService::new(vault_client.clone()); - - // Create API state - let state = ApiState { - document_service: Arc::new(document_service), - vault_client: Arc::new(vault_client), - }; - - // Setup router - let app = Router::new() - .route("/health", get(health_check)) - .route("/api/login", post(login)) - .route("/api/documents", post(upload_document)) - .route("/api/documents/:id", get(get_document)) - .route("/api/documents/:id/sign", post(sign_document)) - .route("/api/documents/:id/verify", get(verify_document)) - .with_state(state); - - info!("API routes configured"); - - // Get the socket address - let addr = std::net::SocketAddr::from(([0, 0, 0, 0], api_port)); - - // Bind and serve - info!("Serving API at {}", addr); - Server::bind(&addr) - .serve(app.into_make_service()) - .await?; - - Ok(()) -} - -// Health check endpoint -#[instrument] -async fn health_check() -> &'static str { - debug!("Health check endpoint called"); - "OK" -} - -// Login endpoint -#[instrument(skip(state, request), fields(username = %request.username))] -async fn login( - State(state): State, - Json(request): Json, -) -> Result, ApiError> { - info!("Login attempt for user: {}", request.username); - - let token = state.vault_client - .login_user(&request.username, &request.password) - .await?; - - info!("User {} successfully authenticated", request.username); - Ok(Json(LoginResponse { token })) -} - -// Upload document endpoint -#[instrument(skip(state, multipart))] -async fn upload_document( - State(state): State, - mut multipart: Multipart, -) -> Result, ApiError> { - info!("Document upload request received"); - - let mut document_name = String::new(); - let mut document_content = Vec::new(); - - // Process multipart form - while let Some(field) = multipart.next_field().await? { - let name = field.name().unwrap_or("").to_string(); - - if name == "name" { - document_name = field.text().await?; - debug!("Received document name: {}", document_name); - } else if name == "file" { - document_content = field.bytes().await?.to_vec(); - debug!("Received document content: {} bytes", document_content.len()); - } - } - - if document_name.is_empty() || document_content.is_empty() { - error!("Missing document name or content"); - return Err(anyhow::anyhow!("Missing document name or content").into()); - } - - // Upload document - let document_id = state.document_service - .upload_document(&document_name, &document_content) - .await?; - - // Return document metadata - let document = state.document_service - .get_document(&document_id) - .await?; - - info!("Document uploaded successfully with ID: {}", document_id); - Ok(Json(document)) -} - -// Get document endpoint -#[instrument(skip(state))] -async fn get_document( - State(state): State, - Path(document_id): Path, -) -> Result, ApiError> { - info!("Fetching document: {}", document_id); - - let document = state.document_service - .get_document(&document_id) - .await?; - - debug!("Retrieved document {} with {} signatures", - document.id, document.signatures.len()); - - Ok(Json(document)) -} - -// Sign document endpoint -#[instrument(skip(state, request), fields(document_id = %document_id, username = %request.username))] -async fn sign_document( - State(state): State, - Path(document_id): Path, - Json(request): Json, -) -> Result, ApiError> { - info!("Signing request for document {} by user {}", document_id, request.username); - - state.document_service - .sign_document(&document_id, &request.username, &request.token) - .await?; - - let document = state.document_service - .get_document(&document_id) - .await?; - - info!("Document {} successfully signed by {}", document_id, request.username); - Ok(Json(document)) -} - -// Verify document endpoint -#[instrument(skip(state))] -async fn verify_document( - State(state): State, - Path(document_id): Path, -) -> Result, ApiError> { - info!("Verifying document signatures: {}", document_id); - - let verification = state.document_service - .verify_document_signatures(&document_id) - .await?; - - info!("Document {} verification result: {}", - document_id, if verification.is_verified { "VERIFIED" } else { "PENDING" }); - - Ok(Json(verification)) -} diff --git a/src/document_service.rs b/src/document_service.rs deleted file mode 100644 index 41f357f..0000000 --- a/src/document_service.rs +++ /dev/null @@ -1,485 +0,0 @@ -use anyhow::{Context, Result}; -use reqwest::StatusCode; -use serde::{Deserialize, Serialize}; -use serde_json::json; -use sha2::{Sha256, Digest}; -use std::collections::HashMap; -use uuid::Uuid; -use base64::{Engine as _, engine::general_purpose::STANDARD as BASE64}; -use tracing::{info, error, debug, instrument}; - -use crate::vault_setup::{Department, User, VaultClient}; - -// Document status -#[derive(Debug, Clone, Serialize, Deserialize)] -pub enum DocumentStatus { - #[serde(rename = "pending")] - Pending, - #[serde(rename = "verified")] - Verified, -} - -// Document metadata -#[derive(Debug, Clone, Serialize, Deserialize)] -pub struct Document { - pub id: String, - pub name: String, - pub hash: String, - pub status: DocumentStatus, - pub signatures: HashMap, // username -> signature -} - -// Signature verification response -#[derive(Debug, Clone, Serialize, Deserialize)] -pub struct SignatureVerification { - pub document_id: String, - pub is_verified: bool, - pub signatures_count: usize, - pub legal_signatures: usize, - pub finance_signatures: usize, - pub required_signatures: usize, - pub required_legal: usize, - pub required_finance: usize, -} - -// Document service to handle document operations -pub struct DocumentService { - vault_client: VaultClient, -} - -impl DocumentService { - pub fn new(vault_client: VaultClient) -> Self { - DocumentService { vault_client } - } - - // Upload a new document and store its metadata - #[instrument(skip(self, content), fields(document_name = %name))] - pub async fn upload_document(&self, name: &str, content: &[u8]) -> Result { - info!("Uploading new document: {}", name); - - // Generate a unique ID - let id = Uuid::new_v4().to_string(); - debug!("Generated document ID: {}", id); - - // Calculate document hash - let mut hasher = Sha256::new(); - hasher.update(content); - let hash = format!("{:x}", hasher.finalize()); - debug!("Document hash: {}", hash); - - // Create document metadata - let document = Document { - id: id.clone(), - name: name.to_string(), - hash, - status: DocumentStatus::Pending, - signatures: HashMap::new(), - }; - - // Store document metadata in Vault - self.store_document_metadata(&document).await?; - - info!("Document uploaded with ID: {}", id); - Ok(id) - } - - // Store document metadata in Vault - #[instrument(skip(self, document), fields(document_id = %document.id))] - async fn store_document_metadata(&self, document: &Document) -> Result<()> { - debug!("Storing document metadata for {}", document.id); - - let url = format!("{}/v1/documents/data/docs/{}", - self.vault_client.addr, document.id); - - let payload = json!({ - "data": { - "id": document.id, - "name": document.name, - "hash": document.hash, - "status": match document.status { - DocumentStatus::Pending => "pending", - DocumentStatus::Verified => "verified", - }, - "signatures": document.signatures, - } - }); - - let response = self.vault_client.client - .post(&url) - .header("X-Vault-Token", &self.vault_client.token) - .json(&payload) - .send() - .await?; - - match response.status() { - StatusCode::OK | StatusCode::NO_CONTENT => { - info!("Successfully stored document metadata for {}", document.id); - Ok(()) - } - status => { - let error_text = response.text().await?; - error!("Failed to store document metadata: {} - {}", status, error_text); - Err(anyhow::anyhow!("Failed to store document metadata: {} - {}", status, error_text)) - } - } - } - - // Get document metadata - #[instrument(skip(self))] - pub async fn get_document(&self, document_id: &str) -> Result { - debug!("Getting document metadata for {}", document_id); - - let url = format!("{}/v1/documents/data/docs/{}", - self.vault_client.addr, document_id); - - let response = self.vault_client.client - .get(&url) - .header("X-Vault-Token", &self.vault_client.token) - .send() - .await?; - - match response.status() { - StatusCode::OK => { - let json: serde_json::Value = response.json().await?; - - // Extract status - let status_str = json["data"]["data"]["status"] - .as_str() - .context("Failed to extract status")?; - - let status = match status_str { - "pending" => DocumentStatus::Pending, - "verified" => DocumentStatus::Verified, - _ => { - error!("Unknown document status: {}", status_str); - return Err(anyhow::anyhow!("Unknown status: {}", status_str)); - } - }; - - // Extract signatures - let signatures_value = &json["data"]["data"]["signatures"]; - let mut signatures = HashMap::new(); - - if let Some(obj) = signatures_value.as_object() { - for (username, sig) in obj { - if let Some(sig_str) = sig.as_str() { - signatures.insert(username.clone(), sig_str.to_string()); - } - } - } - - let document = Document { - id: json["data"]["data"]["id"].as_str().context("Missing id")?.to_string(), - name: json["data"]["data"]["name"].as_str().context("Missing name")?.to_string(), - hash: json["data"]["data"]["hash"].as_str().context("Missing hash")?.to_string(), - status, - signatures, - }; - - debug!("Retrieved document: {} with {} signatures", document.id, document.signatures.len()); - Ok(document) - } - status => { - let error_text = response.text().await?; - error!("Failed to get document: {} - {}", status, error_text); - Err(anyhow::anyhow!("Failed to get document: {} - {}", status, error_text)) - } - } - } - - // Sign a document with the user's key - #[instrument(skip(self, user_token), fields(document_id = %document_id, username = %username))] - pub async fn sign_document(&self, document_id: &str, username: &str, user_token: &str) -> Result<()> { - info!("Signing document {} by user {}", document_id, username); - - // Get document metadata - let document = self.get_document(document_id).await?; - - // Get user info to verify department - let user = self.vault_client.get_user_info(username).await?; - - // Sign the document hash with user's key - let url = format!("{}/v1/transit/sign/{}", - self.vault_client.addr, username); - - let payload = json!({ - "input": BASE64.encode(document.hash.as_bytes()), - }); - - let response = self.vault_client.client - .post(&url) - .header("X-Vault-Token", user_token) - .json(&payload) - .send() - .await?; - - match response.status() { - StatusCode::OK => { - let json: serde_json::Value = response.json().await?; - let signature = json["data"]["signature"] - .as_str() - .context("Failed to extract signature")? - .to_string(); - - debug!("Generated signature for document {}", document_id); - - // Update document with signature - self.add_signature(document_id, username, &signature).await?; - - // Update department signature record - self.record_department_signature(document_id, &user).await?; - - // Check if document now has enough signatures - self.update_document_status(document_id).await?; - - info!("Document {} signed by {}", document_id, username); - Ok(()) - } - status => { - let error_text = response.text().await?; - error!("Failed to sign document: {} - {}", status, error_text); - Err(anyhow::anyhow!("Failed to sign document: {} - {}", status, error_text)) - } - } - } - - // Add a signature to a document - #[instrument(skip(self, signature), fields(document_id = %document_id, username = %username))] - async fn add_signature(&self, document_id: &str, username: &str, signature: &str) -> Result<()> { - debug!("Adding signature from {} to document {}", username, document_id); - - // Get current document - let mut document = self.get_document(document_id).await?; - - // Add signature - document.signatures.insert(username.to_string(), signature.to_string()); - - // Store updated document - self.store_document_metadata(&document).await?; - - debug!("Added signature from {} to document {}", username, document_id); - Ok(()) - } - - // Record department signature - #[instrument(skip(self), fields(document_id = %document_id, department = ?user.department, username = %user.username))] - async fn record_department_signature(&self, document_id: &str, user: &User) -> Result<()> { - let dept_str = match user.department { - Department::Legal => "legal", - Department::Finance => "finance", - }; - - debug!("Recording {} department signature for document {}", dept_str, document_id); - - let url = format!("{}/v1/documents/data/dept/{}/signatures/{}", - self.vault_client.addr, dept_str, document_id); - - // Check if department signatures already exist - let response = self.vault_client.client - .get(&url) - .header("X-Vault-Token", &self.vault_client.token) - .send() - .await; - - let mut signatures = Vec::new(); - - // If record exists, get current signatures - if let Ok(resp) = response { - if resp.status() == StatusCode::OK { - let json: serde_json::Value = resp.json().await?; - if let Some(array) = json["data"]["data"]["signatures"].as_array() { - for sig in array { - if let Some(sig_str) = sig.as_str() { - signatures.push(sig_str.to_string()); - } - } - } - } - } - - // Add user to signatures if not already present - if !signatures.contains(&user.username) { - signatures.push(user.username.clone()); - } - - // Store updated signatures - let payload = json!({ - "data": { - "signatures": signatures, - } - }); - - let response = self.vault_client.client - .post(&url) - .header("X-Vault-Token", &self.vault_client.token) - .json(&payload) - .send() - .await?; - - match response.status() { - StatusCode::OK | StatusCode::NO_CONTENT => { - info!("Recorded signature for {} in {} department", user.username, dept_str); - Ok(()) - } - status => { - let error_text = response.text().await?; - error!("Failed to record department signature: {} - {}", status, error_text); - Err(anyhow::anyhow!("Failed to record department signature: {} - {}", status, error_text)) - } - } - } - - // Update document status if it has enough signatures - #[instrument(skip(self))] - async fn update_document_status(&self, document_id: &str) -> Result<()> { - debug!("Checking if document {} has enough signatures", document_id); - - // Verify signatures - let verification = self.verify_document_signatures(document_id).await?; - - if verification.is_verified { - // Get current document - let mut document = self.get_document(document_id).await?; - - // Update status - document.status = DocumentStatus::Verified; - - // Store updated document - self.store_document_metadata(&document).await?; - - info!("Document {} marked as verified", document_id); - } else { - debug!( - "Document {} not yet verified. Has {}/{} signatures ({} legal, {} finance)", - document_id, - verification.signatures_count, - verification.required_signatures, - verification.legal_signatures, - verification.required_finance - ); - } - - Ok(()) - } - - // Verify document signatures - #[instrument(skip(self))] - pub async fn verify_document_signatures(&self, document_id: &str) -> Result { - info!("Verifying signatures for document {}", document_id); - - // Get document - let document = self.get_document(document_id).await?; - - // Get signing requirements - let url = format!("{}/v1/documents/data/config/signing_requirements", - self.vault_client.addr); - - let response = self.vault_client.client - .get(&url) - .header("X-Vault-Token", &self.vault_client.token) - .send() - .await?; - - let config = match response.status() { - StatusCode::OK => response.json::().await?, - status => { - let error_text = response.text().await?; - error!("Failed to get signing requirements: {} - {}", status, error_text); - return Err(anyhow::anyhow!("Failed to get signing requirements: {} - {}", status, error_text)); - } - }; - - // Get required signatures - let required_signatures = config["data"]["data"]["total_required"] - .as_u64() - .context("Missing total_required")? as usize; - - // Get department requirements - let mut required_legal = 0; - let mut required_finance = 0; - - if let Some(departments) = config["data"]["data"]["departments"].as_array() { - for dept in departments { - let name = dept["name"].as_str().context("Missing department name")?; - let required = dept["required"].as_u64().context("Missing required")? as usize; - - match name { - "legal" => required_legal = required, - "finance" => required_finance = required, - _ => (), - } - } - } - - // Get department signatures - let legal_signatures = self.get_department_signatures(document_id, "legal").await?; - let finance_signatures = self.get_department_signatures(document_id, "finance").await?; - - // Check if requirements are met - let total_signatures = document.signatures.len(); - let is_verified = total_signatures >= required_signatures && - legal_signatures.len() >= required_legal && - finance_signatures.len() >= required_finance; - - let verification = SignatureVerification { - document_id: document_id.to_string(), - is_verified, - signatures_count: total_signatures, - legal_signatures: legal_signatures.len(), - finance_signatures: finance_signatures.len(), - required_signatures, - required_legal, - required_finance, - }; - - info!( - "Verification result for document {}: verified={}, signatures={}/{}, legal={}/{}, finance={}/{}", - document_id, - verification.is_verified, - verification.signatures_count, - verification.required_signatures, - verification.legal_signatures, - verification.required_legal, - verification.finance_signatures, - verification.required_finance - ); - - Ok(verification) - } - - // Get department signatures for a document - #[instrument(skip(self))] - async fn get_department_signatures(&self, document_id: &str, department: &str) -> Result> { - debug!("Getting {} department signatures for document {}", department, document_id); - - let url = format!("{}/v1/documents/data/dept/{}/signatures/{}", - self.vault_client.addr, department, document_id); - - let response = self.vault_client.client - .get(&url) - .header("X-Vault-Token", &self.vault_client.token) - .send() - .await; - - let mut signatures = Vec::new(); - - // If record exists, get signatures - if let Ok(resp) = response { - if resp.status() == StatusCode::OK { - let json: serde_json::Value = resp.json().await?; - if let Some(array) = json["data"]["data"]["signatures"].as_array() { - for sig in array { - if let Some(sig_str) = sig.as_str() { - signatures.push(sig_str.to_string()); - } - } - } - } - } - - debug!("Found {} signatures for {} department on document {}", - signatures.len(), department, document_id); - - Ok(signatures) - } -} diff --git a/src/lib.rs b/src/lib.rs deleted file mode 100644 index 7ce0e6f..0000000 --- a/src/lib.rs +++ /dev/null @@ -1,11 +0,0 @@ -// Modules that implement our hierarchical signing system -pub mod vault_setup; -pub mod vault_init; -pub mod document_service; -pub mod api; - -// Re-export main components for easier access -pub use vault_setup::VaultClient; -pub use vault_init::initialize_vault; -pub use document_service::DocumentService; -pub use api::start_api; diff --git a/src/main.rs b/src/main.rs index 2efb99b..2460556 100644 --- a/src/main.rs +++ b/src/main.rs @@ -1,40 +1,412 @@ -use anyhow::Result; -use std::env; -use tracing::{info}; -use tracing_subscriber::{fmt, EnvFilter}; +use anyhow::{Context, Result}; +use reqwest::{Client, StatusCode}; +use serde::{Deserialize, Serialize}; +use std::{ + env, + fs::File, + io::Write, + path::Path, + process::Command, + time::Duration, +}; +use tokio::time::sleep; -// Import our library -use vault_hier::{start_api, initialize_vault}; +// Vault API response structures +#[derive(Debug, Deserialize)] +struct InitResponse { + keys: Vec, + keys_base64: Vec, + root_token: String, +} + +#[derive(Debug, Deserialize)] +struct SealStatusResponse { + sealed: bool, + t: u8, + n: u8, + progress: u8, +} + +#[derive(Debug, Serialize)] +struct InitRequest { + secret_shares: u8, + secret_threshold: u8, +} + +#[derive(Debug, Serialize)] +struct UnsealRequest { + key: String, +} + +// Function to save Vault credentials to a file +fn save_credentials(response: &InitResponse, file_path: &str) -> Result<()> { + let mut file = File::create(Path::new(file_path))?; + writeln!(file, "Unseal Keys:")?; + for (i, key) in response.keys.iter().enumerate() { + writeln!(file, "Key {}: {}", i + 1, key)?; + } + writeln!(file, "Base64 Unseal Keys:")?; + for (i, key) in response.keys_base64.iter().enumerate() { + writeln!(file, "Key {}: {}", i + 1, key)?; + } + writeln!(file)?; + writeln!(file, "Root Token: {}", response.root_token)?; + + println!("Credentials saved to {}", file_path); + Ok(()) +} + +// Wait for Vault to become available +async fn wait_for_vault(addr: &str) -> Result<()> { + println!("Waiting for Vault to be ready..."); + + let client = Client::new(); + + for i in 1..=30 { + let health_url = format!("{}/v1/sys/health?standbyok=true&sealedok=true&uninitok=true", addr); + + match client.get(&health_url).timeout(Duration::from_secs(1)).send().await { + Ok(response) => { + let status = response.status().as_u16(); + // Accept any of these status codes as "available" + if matches!(status, 200 | 429 | 472 | 473 | 501 | 503) { + println!("Vault is available! Status code: {}", status); + return Ok(()); + } + + println!("Vault returned unexpected status code: {}", status); + }, + Err(e) => { + println!("Error connecting to Vault: {}", e); + } + } + + if i == 30 { + return Err(anyhow::anyhow!("Timed out waiting for Vault to become available")); + } + + println!("Vault is unavailable - sleeping (attempt {}/30)", i); + sleep(Duration::from_secs(2)).await; + } + + Ok(()) +} + +// Function to copy credentials to a mounted volume if available +fn copy_credentials_to_volume(src_path: &str) -> Result<()> { + println!("Searching for credentials file..."); + + if let Ok(metadata) = std::fs::metadata(src_path) { + if metadata.is_file() { + println!("Found credentials at {}, copying...", src_path); + + // Create the data directory if it doesn't exist + if let Err(e) = std::fs::create_dir_all("/app/data") { + println!("Warning: Couldn't create /app/data directory: {}", e); + } else { + let dest_path = "/app/data/vault-credentials.txt"; + + // Check if source and destination are the same + if src_path == dest_path { + println!("Source and destination are the same file, skipping copy"); + } else { + match std::fs::copy(src_path, dest_path) { + Ok(_) => println!("Credentials saved to {}", dest_path), + Err(e) => println!("Failed to copy credentials: {}", e), + } + } + } + } + } else { + // If the file doesn't exist in the current directory, search for it + let output = Command::new("find") + .args(["/", "-name", "vault-credentials.txt", "-type", "f"]) + .output(); + + match output { + Ok(output) => { + let files = String::from_utf8_lossy(&output.stdout); + let files: Vec<&str> = files.split('\n').filter(|s| !s.is_empty()).collect(); + + if !files.is_empty() { + println!("Found credentials at {}, copying...", files[0]); + + // Create the data directory if it doesn't exist + if let Err(e) = std::fs::create_dir_all("/app/data") { + println!("Warning: Couldn't create /app/data directory: {}", e); + } else { + let dest_path = "/app/data/vault-credentials.txt"; + + // Check if source and destination are the same + if files[0] == dest_path { + println!("Source and destination are the same file, skipping copy"); + } else { + match std::fs::copy(files[0], dest_path) { + Ok(_) => println!("Credentials saved to {}", dest_path), + Err(e) => println!("Failed to copy credentials: {}", e), + } + } + } + } else { + println!("Could not find credentials file"); + } + }, + Err(e) => println!("Failed to search for credentials: {}", e), + } + } + + Ok(()) +} + +async fn check_init_status(client: &Client, addr: &str) -> Result { + println!("Checking if Vault is already initialized..."); + + let response = client + .get(format!("{}/v1/sys/init", addr)) + .send() + .await?; + + if response.status().is_success() { + let status = response.json::().await?; + if let Some(initialized) = status.get("initialized").and_then(|v| v.as_bool()) { + return Ok(initialized); + } + } + + // If we couldn't determine, assume not initialized + Ok(false) +} + +async fn check_seal_status(client: &Client, addr: &str) -> Result { + println!("Checking Vault seal status..."); + + let response = client + .get(format!("{}/v1/sys/seal-status", addr)) + .send() + .await?; + + if response.status().is_success() { + let status = response.json::().await?; + println!("Seal status: sealed={}, threshold={}, shares={}, progress={}", + status.sealed, status.t, status.n, status.progress); + return Ok(status); + } else { + let error_text = response.text().await?; + anyhow::bail!("Failed to get seal status: {}", error_text); + } +} + +async fn init_vault(client: &Client, addr: &str) -> Result { + // First check if already initialized + let initialized = check_init_status(client, addr).await?; + + if initialized { + anyhow::bail!("Vault is already initialized. Cannot re-initialize."); + } + + println!("Initializing Vault..."); + + // Configure with 5 key shares and a threshold of 3 + // This is a standard production configuration, requiring 3 out of 5 keys to unseal + let init_req = InitRequest { + secret_shares: 5, + secret_threshold: 3, + }; + + let response = client + .put(format!("{}/v1/sys/init", addr)) + .json(&init_req) + .send() + .await?; + + match response.status() { + StatusCode::OK => { + let init_response = response.json::().await?; + println!("Vault initialized successfully!"); + Ok(init_response) + } + status => { + let error_text = response.text().await?; + anyhow::bail!("Failed to initialize Vault: {} - {}", status, error_text); + } + } +} + +async fn unseal_vault(client: &Client, addr: &str, unseal_keys: &[String]) -> Result<()> { + // First check the current seal status + let mut seal_status = check_seal_status(client, addr).await?; + + if !seal_status.sealed { + println!("Vault is already unsealed!"); + return Ok(()); + } + + println!("Unsealing Vault..."); + + // We need to provide enough keys to meet the threshold + // The threshold is in seal_status.t + let required_keys = seal_status.t as usize; + + if unseal_keys.len() < required_keys { + anyhow::bail!( + "Not enough unseal keys provided. Need {} keys, but only have {}", + required_keys, + unseal_keys.len() + ); + } + + // Apply each key one at a time until unsealed + for (i, key) in unseal_keys.iter().take(required_keys).enumerate() { + println!("Applying unseal key {}/{}...", i + 1, required_keys); + + let unseal_req = UnsealRequest { + key: key.clone(), + }; + + let response = client + .put(format!("{}/v1/sys/unseal", addr)) + .json(&unseal_req) + .send() + .await?; + + if !response.status().is_success() { + let error_text = response.text().await?; + anyhow::bail!("Failed to apply unseal key: {}", error_text); + } + + // Check the updated seal status + seal_status = check_seal_status(client, addr).await?; + + if !seal_status.sealed { + println!("Vault unsealed successfully after applying {} keys!", i + 1); + return Ok(()); + } + } + + // If we get here, we've applied all keys but Vault is still sealed + if seal_status.sealed { + anyhow::bail!("Applied all available unseal keys, but Vault is still sealed"); + } + + Ok(()) +} #[tokio::main] async fn main() -> Result<()> { - // Initialize tracing - fmt() - .with_env_filter(EnvFilter::from_default_env().add_directive("vault_hier=info".parse()?)) - .with_target(false) - .init(); - // Get Vault address from env var or use default let vault_addr = env::var("VAULT_ADDR").unwrap_or_else(|_| "http://127.0.0.1:8200".to_string()); - - // Get API port from env var or use default - let api_port = env::var("API_PORT") - .unwrap_or_else(|_| "3000".to_string()) - .parse::() - .unwrap_or(3000); - - info!("Vault address: {}", vault_addr); - info!("Connecting to Vault at: {}", vault_addr); - - // Initialize and unseal Vault, get the root token - let root_token = initialize_vault(&vault_addr).await?; - - info!("Starting hierarchical document signing API..."); - - // Start the hierarchical signing API - start_api(&vault_addr, &root_token, api_port).await?; - - info!("API server shutdown. Exiting."); - + let client = Client::new(); + + println!("Vault address: {}", vault_addr); + println!("Connecting to Vault at: {}", vault_addr); + + // Wait for Vault to be available + wait_for_vault(&vault_addr).await?; + + // Get Vault status to display + let health_url = format!("{}/v1/sys/health?standbyok=true&sealedok=true&uninitok=true", vault_addr); + match client.get(&health_url).send().await { + Ok(response) => { + if response.status().is_success() { + let status_text = response.text().await?; + println!("Vault status: {}", status_text); + } + }, + Err(e) => println!("Error getting Vault status: {}", e), + } + + // First check if Vault is already initialized + let initialized = check_init_status(&client, &vault_addr).await?; + + if initialized { + println!("Vault is already initialized."); + + // Check if Vault is sealed + let seal_status = check_seal_status(&client, &vault_addr).await?; + + if seal_status.sealed { + println!("Vault is sealed. Looking for unseal keys..."); + + // Try to load unseal keys from environment variables + let mut unseal_keys = Vec::new(); + for i in 1..=5 { + match env::var(format!("VAULT_UNSEAL_KEY_{}", i)) { + Ok(key) => { + println!("Found unseal key {} from environment", i); + unseal_keys.push(key); + }, + Err(_) => { + println!("Unseal key {} not found in environment", i); + } + } + } + + // If we have unseal keys, try to unseal + if !unseal_keys.is_empty() { + println!("Found {} unseal keys. Attempting to unseal...", unseal_keys.len()); + unseal_vault(&client, &vault_addr, &unseal_keys).await?; + } else { + println!("No unseal keys found. Vault remains sealed."); + println!("To unseal, set VAULT_UNSEAL_KEY_1, VAULT_UNSEAL_KEY_2, etc. environment variables."); + } + } else { + println!("Vault is already unsealed."); + } + } else { + // Initialize Vault + println!("Vault is not initialized. Proceeding with initialization..."); + let init_response = init_vault(&client, &vault_addr).await?; + + // Save credentials to files + println!("Saving credentials to file..."); + let current_dir = std::env::current_dir().context("Failed to get current directory")?; + let cred_path = current_dir.join("vault-credentials.txt"); + save_credentials(&init_response, cred_path.to_str().unwrap())?; + println!("Credentials saved to: {}", cred_path.display()); + + // Also save to /app/data as a backup for Docker volume mounting + if let Ok(()) = std::fs::create_dir_all("/app/data") { + let docker_path = "/app/data/vault-credentials.txt"; + save_credentials(&init_response, docker_path)?; + println!("Backup credentials saved to Docker volume at: {}", docker_path); + } + + println!("========================================="); + println!("IMPORTANT: SAVE THESE CREDENTIALS SECURELY"); + println!("========================================="); + println!("Root Token: {}", init_response.root_token); + println!("Unseal Keys (first 3 of 5 needed to unseal):"); + for (i, key) in init_response.keys_base64.iter().enumerate() { + println!("Key {}: {}", i + 1, key); + } + println!("========================================="); + + // Unseal Vault using the first three keys + let unseal_keys = init_response.keys_base64.iter() + .take(3) // We only need threshold number of keys (3) + .cloned() + .collect::>(); + + unseal_vault(&client, &vault_addr, &unseal_keys).await?; + + println!("Vault is now initialized and unsealed"); + + // Store the root token and unseal keys in environment variables + // Using unsafe block as set_var is now considered unsafe in recent Rust + unsafe { + env::set_var("VAULT_TOKEN", &init_response.root_token); + for (i, key) in init_response.keys_base64.iter().enumerate() { + env::set_var(format!("VAULT_UNSEAL_KEY_{}", i + 1), key); + } + } + + println!("Vault initialization and unseal complete!"); + } + + // Copy credentials to the mounted volume (former docker-entrypoint.sh functionality) + copy_credentials_to_volume("vault-credentials.txt")?; + + println!("Operation complete!"); + Ok(()) -} +} \ No newline at end of file diff --git a/src/vault_init.rs b/src/vault_init.rs deleted file mode 100644 index 9d489e1..0000000 --- a/src/vault_init.rs +++ /dev/null @@ -1,173 +0,0 @@ -use anyhow::{Context, Result}; -use reqwest::Client; -use std::{ - env, - fs, -}; -use tracing::{info, warn, error, debug}; - -use crate::vault_setup::VaultClient; - -/// Initialize and unseal the Vault, returning the root token for further operations -pub async fn initialize_vault(vault_addr: &str) -> Result { - let client = Client::new(); - - // Wait for Vault to be available - VaultClient::wait_for_vault(vault_addr).await?; - - // Display Vault health status - let health_url = format!("{}/v1/sys/health?standbyok=true&sealedok=true&uninitok=true", vault_addr); - match client.get(&health_url).send().await { - Ok(response) => { - if response.status().is_success() { - let status_text = response.text().await?; - info!("Vault status: {}", status_text); - } - }, - Err(e) => warn!("Error getting Vault status: {}", e), - } - - // First check if Vault is already initialized - let initialized = VaultClient::check_init_status(&client, vault_addr).await?; - let mut root_token = String::new(); - - if initialized { - info!("Vault is already initialized."); - - // Check if Vault is sealed - let seal_status = VaultClient::check_seal_status(&client, vault_addr).await?; - - if seal_status.sealed { - info!("Vault is sealed. Looking for unseal keys..."); - - // Try to load unseal keys from environment variables - let mut unseal_keys = Vec::new(); - for i in 1..=5 { - match env::var(format!("VAULT_UNSEAL_KEY_{}", i)) { - Ok(key) => { - info!("Found unseal key {} from environment", i); - unseal_keys.push(key); - }, - Err(_) => { - debug!("Unseal key {} not found in environment", i); - } - } - } - - // If we have unseal keys, try to unseal - if !unseal_keys.is_empty() { - info!("Found {} unseal keys. Attempting to unseal...", unseal_keys.len()); - VaultClient::unseal_vault(&client, vault_addr, &unseal_keys).await?; - } else { - warn!("No unseal keys found. Vault remains sealed."); - info!("To unseal, set VAULT_UNSEAL_KEY_1, VAULT_UNSEAL_KEY_2, etc. environment variables."); - } - } else { - info!("Vault is already unsealed."); - } - - // Try to load root token from environment or credentials file - match env::var("VAULT_TOKEN") { - Ok(token) => { - info!("Found root token from environment"); - root_token = token; - }, - Err(_) => { - // Try to load from credentials file - if let Ok(contents) = fs::read_to_string("vault-credentials.json") { - if let Ok(creds) = serde_json::from_str::(&contents) { - if let Some(token) = creds["root_token"].as_str() { - info!("Found root token from credentials file"); - root_token = token.to_string(); - } - } - } - } - } - - if root_token.is_empty() { - error!("Unable to find root token. Please set VAULT_TOKEN environment variable or provide vault-credentials.json file."); - anyhow::bail!("Unable to find root token. Please set VAULT_TOKEN environment variable or provide vault-credentials.json file."); - } - } else { - // Initialize Vault - info!("Vault is not initialized. Proceeding with initialization..."); - let init_response = VaultClient::init_vault(&client, vault_addr).await?; - - // Save credentials to files - info!("Saving credentials to files..."); - let current_dir = std::env::current_dir().context("Failed to get current directory")?; - - // Save as JSON (new format) - let json_path = current_dir.join("vault-credentials.json"); - VaultClient::save_credentials(&init_response, json_path.to_str().unwrap())?; - info!("JSON credentials saved to: {}", json_path.display()); - - // Save as text (for backward compatibility) - let text_path = current_dir.join("vault-credentials.txt"); - VaultClient::save_credentials(&init_response, text_path.to_str().unwrap())?; - info!("Text credentials saved to: {}", text_path.display()); - - // Also save to /app/data as a backup for Docker volume mounting - if let Ok(()) = std::fs::create_dir_all("/app/data") { - let docker_json_path = "/app/data/vault-credentials.json"; - VaultClient::save_credentials(&init_response, docker_json_path)?; - info!("Backup JSON credentials saved to Docker volume at: {}", docker_json_path); - - let docker_text_path = "/app/data/vault-credentials.txt"; - VaultClient::save_credentials(&init_response, docker_text_path)?; - info!("Backup text credentials saved to Docker volume at: {}", docker_text_path); - } - - info!("========================================="); - info!("IMPORTANT: SAVE THESE CREDENTIALS SECURELY"); - info!("========================================="); - info!("Root Token: {}", init_response.root_token); - info!("Unseal Keys (first 3 of 5 needed to unseal):"); - for (i, key) in init_response.keys_base64.iter().enumerate() { - info!("Key {}: {}", i + 1, key); - } - info!("========================================="); - - // Unseal Vault using the first three keys - let unseal_keys = init_response.keys_base64.iter() - .take(3) // We only need threshold number of keys (3) - .cloned() - .collect::>(); - - VaultClient::unseal_vault(&client, vault_addr, &unseal_keys).await?; - - info!("Vault initialization and unseal complete!"); - - // Set root token - root_token = init_response.root_token; - } - - // Look for any existing credentials and copy them to the mounted volume - if let Ok(metadata) = std::fs::metadata("vault-credentials.json") { - if metadata.is_file() { - info!("Found JSON credentials file, ensuring it's saved to Docker volume..."); - if let Ok(()) = std::fs::create_dir_all("/app/data") { - match std::fs::copy("vault-credentials.json", "/app/data/vault-credentials.json") { - Ok(_) => info!("JSON credentials saved to Docker volume"), - Err(e) => warn!("Failed to copy JSON credentials: {}", e), - } - } - } - } - - if let Ok(metadata) = std::fs::metadata("vault-credentials.txt") { - if metadata.is_file() { - info!("Found text credentials file, ensuring it's saved to Docker volume..."); - if let Ok(()) = std::fs::create_dir_all("/app/data") { - match std::fs::copy("vault-credentials.txt", "/app/data/vault-credentials.txt") { - Ok(_) => info!("Text credentials saved to Docker volume"), - Err(e) => warn!("Failed to copy text credentials: {}", e), - } - } - } - } - - info!("Vault setup complete!"); - Ok(root_token) -} diff --git a/src/vault_setup.rs b/src/vault_setup.rs deleted file mode 100644 index 771b3cf..0000000 --- a/src/vault_setup.rs +++ /dev/null @@ -1,707 +0,0 @@ -use anyhow::{Context, Result}; -use reqwest::{Client, StatusCode}; -use serde::{Deserialize, Serialize}; -use serde_json::json; -use std::{ - fs::File, - io::Write, - path::Path, - time::Duration, -}; -use tokio::time::sleep; -use tracing::{info, error, debug, instrument}; - -// Vault API response structures -#[derive(Debug, Deserialize)] -pub struct InitResponse { - pub keys: Vec, - pub keys_base64: Vec, - pub root_token: String, -} - -#[derive(Debug, Deserialize)] -pub struct SealStatusResponse { - pub sealed: bool, - pub t: u8, - pub n: u8, - pub progress: u8, -} - -#[derive(Debug, Serialize)] -pub struct InitRequest { - pub secret_shares: u8, - pub secret_threshold: u8, -} - -#[derive(Debug, Serialize)] -pub struct UnsealRequest { - pub key: String, -} - -// Department types for organizational structure -#[derive(Debug, Clone, Serialize, Deserialize)] -pub enum Department { - Legal, - Finance, -} - -// User representation with department info -#[derive(Debug, Clone, Serialize, Deserialize)] -pub struct User { - pub username: String, - pub department: Department, -} - -// VaultClient to interact with Vault API -#[derive(Clone)] -pub struct VaultClient { - pub client: Client, - pub addr: String, - pub token: String, -} - -impl VaultClient { - pub fn new(addr: &str, token: &str) -> Self { - VaultClient { - client: Client::new(), - addr: addr.to_string(), - token: token.to_string(), - } - } - - // Wait for Vault to become available - #[instrument(skip(addr))] - pub async fn wait_for_vault(addr: &str) -> Result<()> { - info!("Waiting for Vault to be ready..."); - - let client = Client::new(); - - for i in 1..=30 { - let health_url = format!("{}/v1/sys/health?standbyok=true&sealedok=true&uninitok=true", addr); - - match client.get(&health_url).timeout(Duration::from_secs(1)).send().await { - Ok(response) => { - let status = response.status().as_u16(); - // Accept any of these status codes as "available" - if matches!(status, 200 | 429 | 472 | 473 | 501 | 503) { - info!("Vault is available! Status code: {}", status); - return Ok(()); - } - - debug!("Vault returned unexpected status code: {}", status); - }, - Err(e) => { - debug!("Error connecting to Vault: {}", e); - } - } - - if i == 30 { - error!("Timed out waiting for Vault to become available"); - return Err(anyhow::anyhow!("Timed out waiting for Vault to become available")); - } - - info!("Vault is unavailable - sleeping (attempt {}/30)", i); - sleep(Duration::from_secs(2)).await; - } - - Ok(()) - } - - // Function to check if Vault is initialized - #[instrument(skip(client, addr))] - pub async fn check_init_status(client: &Client, addr: &str) -> Result { - info!("Checking if Vault is already initialized..."); - - let response = client - .get(format!("{}/v1/sys/init", addr)) - .send() - .await?; - - if response.status().is_success() { - let status = response.json::().await?; - if let Some(initialized) = status.get("initialized").and_then(|v| v.as_bool()) { - return Ok(initialized); - } - } - - // If we couldn't determine, assume not initialized - Ok(false) - } - - // Function to check Vault seal status - #[instrument(skip(client, addr))] - pub async fn check_seal_status(client: &Client, addr: &str) -> Result { - info!("Checking Vault seal status..."); - - let response = client - .get(format!("{}/v1/sys/seal-status", addr)) - .send() - .await?; - - if response.status().is_success() { - let status = response.json::().await?; - info!("Seal status: sealed={}, threshold={}, shares={}, progress={}", - status.sealed, status.t, status.n, status.progress); - return Ok(status); - } else { - let error_text = response.text().await?; - error!("Failed to get seal status: {}", error_text); - anyhow::bail!("Failed to get seal status: {}", error_text); - } - } - - // Function to initialize Vault - #[instrument(skip(client, addr))] - pub async fn init_vault(client: &Client, addr: &str) -> Result { - // First check if already initialized - let initialized = Self::check_init_status(client, addr).await?; - - if initialized { - error!("Vault is already initialized. Cannot re-initialize."); - anyhow::bail!("Vault is already initialized. Cannot re-initialize."); - } - - info!("Initializing Vault..."); - - // Configure with 5 key shares and a threshold of 3 - // This is a standard production configuration, requiring 3 out of 5 keys to unseal - let init_req = InitRequest { - secret_shares: 5, - secret_threshold: 3, - }; - - let response = client - .put(format!("{}/v1/sys/init", addr)) - .json(&init_req) - .send() - .await?; - - match response.status() { - StatusCode::OK => { - let init_response = response.json::().await?; - info!("Vault initialized successfully!"); - Ok(init_response) - } - status => { - let error_text = response.text().await?; - error!("Failed to initialize Vault: {} - {}", status, error_text); - anyhow::bail!("Failed to initialize Vault: {} - {}", status, error_text); - } - } - } - - // Function to unseal Vault - #[instrument(skip(client, addr, unseal_keys))] - pub async fn unseal_vault(client: &Client, addr: &str, unseal_keys: &[String]) -> Result<()> { - // First check the current seal status - let mut seal_status = Self::check_seal_status(client, addr).await?; - - if !seal_status.sealed { - info!("Vault is already unsealed!"); - return Ok(()); - } - - info!("Unsealing Vault..."); - - // We need to provide enough keys to meet the threshold - // The threshold is in seal_status.t - let required_keys = seal_status.t as usize; - - if unseal_keys.len() < required_keys { - error!( - "Not enough unseal keys provided. Need {} keys, but only have {}", - required_keys, - unseal_keys.len() - ); - anyhow::bail!( - "Not enough unseal keys provided. Need {} keys, but only have {}", - required_keys, - unseal_keys.len() - ); - } - - // Apply each key one at a time until unsealed - for (i, key) in unseal_keys.iter().take(required_keys).enumerate() { - info!("Applying unseal key {}/{}...", i + 1, required_keys); - - let unseal_req = UnsealRequest { - key: key.clone(), - }; - - let response = client - .put(format!("{}/v1/sys/unseal", addr)) - .json(&unseal_req) - .send() - .await?; - - if !response.status().is_success() { - let error_text = response.text().await?; - error!("Failed to apply unseal key: {}", error_text); - anyhow::bail!("Failed to apply unseal key: {}", error_text); - } - - // Check the updated seal status - seal_status = Self::check_seal_status(client, addr).await?; - - if !seal_status.sealed { - info!("Vault unsealed successfully after applying {} keys!", i + 1); - return Ok(()); - } - } - - // If we get here, we've applied all keys but Vault is still sealed - if seal_status.sealed { - error!("Applied all available unseal keys, but Vault is still sealed"); - anyhow::bail!("Applied all available unseal keys, but Vault is still sealed"); - } - - Ok(()) - } - - // Function to save Vault credentials to a file - pub fn save_credentials(response: &InitResponse, file_path: &str) -> Result<()> { - // For JSON output - if file_path.ends_with(".json") { - let json = serde_json::json!({ - "keys": response.keys, - "keys_base64": response.keys_base64, - "root_token": response.root_token - }); - - let mut file = File::create(Path::new(file_path))?; - file.write_all(serde_json::to_string_pretty(&json)?.as_bytes())?; - info!("Credentials saved to JSON file: {}", file_path); - return Ok(()); - } - - // For plaintext output (legacy format) - let mut file = File::create(Path::new(file_path))?; - writeln!(file, "Unseal Keys:")?; - for (i, key) in response.keys.iter().enumerate() { - writeln!(file, "Key {}: {}", i + 1, key)?; - } - writeln!(file, "Base64 Unseal Keys:")?; - for (i, key) in response.keys_base64.iter().enumerate() { - writeln!(file, "Key {}: {}", i + 1, key)?; - } - writeln!(file)?; - writeln!(file, "Root Token: {}", response.root_token)?; - - info!("Credentials saved to {}", file_path); - Ok(()) - } - - // Enable required secrets engines - #[instrument(skip(self))] - pub async fn setup_secrets_engines(&self) -> Result<()> { - info!("Setting up Vault secrets engines..."); - - // Enable Transit for document signing - self.enable_secrets_engine("transit", "transit").await?; - - // Enable KV v2 for document storage - self.enable_secrets_engine("kv-v2", "documents").await?; - - // Enable userpass for authentication - self.enable_auth_method("userpass").await?; - - info!("Secrets engines setup complete!"); - Ok(()) - } - - // Enable a secrets engine - #[instrument(skip(self))] - async fn enable_secrets_engine(&self, engine_type: &str, path: &str) -> Result<()> { - info!("Enabling {} secrets engine at {}", engine_type, path); - - let url = format!("{}/v1/sys/mounts/{}", self.addr, path); - let payload = json!({ - "type": engine_type, - }); - - let response = self.client - .post(&url) - .header("X-Vault-Token", &self.token) - .json(&payload) - .send() - .await?; - - match response.status() { - StatusCode::NO_CONTENT | StatusCode::OK => { - info!("Successfully enabled {} engine at {}", engine_type, path); - Ok(()) - } - StatusCode::BAD_REQUEST => { - // Check if already exists - let error_text = response.text().await?; - if error_text.contains("path is already in use") { - info!("Secrets engine already enabled at {}", path); - Ok(()) - } else { - error!("Failed to enable secrets engine: {}", error_text); - Err(anyhow::anyhow!("Failed to enable secrets engine: {}", error_text)) - } - } - status => { - let error_text = response.text().await?; - error!("Failed to enable secrets engine: {} - {}", status, error_text); - Err(anyhow::anyhow!("Failed to enable secrets engine: {} - {}", status, error_text)) - } - } - } - - // Enable an auth method - #[instrument(skip(self))] - async fn enable_auth_method(&self, method: &str) -> Result<()> { - info!("Enabling {} auth method", method); - - let url = format!("{}/v1/sys/auth/{}", self.addr, method); - let payload = json!({ - "type": method, - }); - - let response = self.client - .post(&url) - .header("X-Vault-Token", &self.token) - .json(&payload) - .send() - .await?; - - match response.status() { - StatusCode::NO_CONTENT | StatusCode::OK => { - info!("Successfully enabled {} auth method", method); - Ok(()) - } - StatusCode::BAD_REQUEST => { - // Check if already exists - let error_text = response.text().await?; - if error_text.contains("path is already in use") { - info!("Auth method already enabled at {}", method); - Ok(()) - } else { - error!("Failed to enable auth method: {}", error_text); - Err(anyhow::anyhow!("Failed to enable auth method: {}", error_text)) - } - } - status => { - let error_text = response.text().await?; - error!("Failed to enable auth method: {} - {}", status, error_text); - Err(anyhow::anyhow!("Failed to enable auth method: {} - {}", status, error_text)) - } - } - } - - // Create a new user in Vault and associate with department - #[instrument(skip(self, password))] - pub async fn create_user(&self, username: &str, password: &str, department: Department) -> Result<()> { - info!("Creating user {} in department {:?}", username, department); - - // Step 1: Create a policy for the user - let policy_name = format!("{}-policy", username); - self.create_signing_policy(&policy_name, department.clone()).await?; - - // Step 2: Create the user with userpass auth - let url = format!("{}/v1/auth/userpass/users/{}", self.addr, username); - let payload = json!({ - "password": password, - "policies": [policy_name], - }); - - let response = self.client - .post(&url) - .header("X-Vault-Token", &self.token) - .json(&payload) - .send() - .await?; - - match response.status() { - StatusCode::NO_CONTENT | StatusCode::OK => { - info!("Successfully created user {}", username); - - // Step 3: Create a signing key for the user - self.create_signing_key(username).await?; - - // Step 4: Store user metadata in KV store - self.store_user_metadata(username, department).await?; - - Ok(()) - } - status => { - let error_text = response.text().await?; - error!("Failed to create user: {} - {}", status, error_text); - Err(anyhow::anyhow!("Failed to create user: {} - {}", status, error_text)) - } - } - } - - // Create a signing policy for a user based on their department - #[instrument(skip(self))] - async fn create_signing_policy(&self, policy_name: &str, department: Department) -> Result<()> { - let dept_name = match department { - Department::Legal => "legal", - Department::Finance => "finance", - }; - - // Policy content with specific paths for the department - let policy = format!(r#" - # Allow reading document metadata - path "documents/data/docs/*" {{ - capabilities = ["read"] - }} - - # Allow signing with user's key - path "transit/sign/{{{{identity.entity.name}}}}" {{ - capabilities = ["update"] - }} - - # Allow signature verification - path "transit/verify/*" {{ - capabilities = ["update"] - }} - - # Department-specific path - path "documents/data/dept/{}/signatures/*" {{ - capabilities = ["create", "read", "update"] - }} - "#, dept_name); - - let url = format!("{}/v1/sys/policies/acl/{}", self.addr, policy_name); - let payload = json!({ - "policy": policy, - }); - - let response = self.client - .put(&url) - .header("X-Vault-Token", &self.token) - .json(&payload) - .send() - .await?; - - match response.status() { - StatusCode::NO_CONTENT | StatusCode::OK => { - info!("Successfully created policy {}", policy_name); - Ok(()) - } - status => { - let error_text = response.text().await?; - error!("Failed to create policy: {} - {}", status, error_text); - Err(anyhow::anyhow!("Failed to create policy: {} - {}", status, error_text)) - } - } - } - - // Create a signing key for a user in the Transit engine - #[instrument(skip(self))] - async fn create_signing_key(&self, username: &str) -> Result<()> { - let url = format!("{}/v1/transit/keys/{}", self.addr, username); - let payload = json!({ - "type": "ed25519", - }); - - let response = self.client - .post(&url) - .header("X-Vault-Token", &self.token) - .json(&payload) - .send() - .await?; - - match response.status() { - StatusCode::NO_CONTENT | StatusCode::OK => { - info!("Successfully created signing key for {}", username); - Ok(()) - } - status => { - let error_text = response.text().await?; - error!("Failed to create signing key: {} - {}", status, error_text); - Err(anyhow::anyhow!("Failed to create signing key: {} - {}", status, error_text)) - } - } - } - - // Store user metadata in the KV store - #[instrument(skip(self))] - async fn store_user_metadata(&self, username: &str, department: Department) -> Result<()> { - let dept_str = match department { - Department::Legal => "legal", - Department::Finance => "finance", - }; - - let url = format!("{}/v1/documents/data/users/{}", self.addr, username); - let payload = json!({ - "data": { - "username": username, - "department": dept_str, - } - }); - - let response = self.client - .post(&url) - .header("X-Vault-Token", &self.token) - .json(&payload) - .send() - .await?; - - match response.status() { - StatusCode::NO_CONTENT | StatusCode::OK => { - info!("Successfully stored metadata for user {}", username); - Ok(()) - } - status => { - let error_text = response.text().await?; - error!("Failed to store user metadata: {} - {}", status, error_text); - Err(anyhow::anyhow!("Failed to store user metadata: {} - {}", status, error_text)) - } - } - } - - // Create 10 users with departmental hierarchy - 5 in each department - #[instrument(skip(self))] - pub async fn setup_hierarchical_users(&self) -> Result<()> { - info!("Setting up hierarchical user structure"); - - // Create 5 users in Legal department - for i in 1..=5 { - let username = format!("legal{}", i); - let password = format!("legal{}pass", i); - debug!(username, "Creating Legal department user"); - self.create_user(&username, &password, Department::Legal).await?; - } - - // Create 5 users in Finance department - for i in 1..=5 { - let username = format!("finance{}", i); - let password = format!("finance{}pass", i); - debug!(username, "Creating Finance department user"); - self.create_user(&username, &password, Department::Finance).await?; - } - - // Setup document signing requirements - self.setup_signing_requirements().await?; - - info!("Successfully created 10 users in hierarchical structure!"); - Ok(()) - } - - // Configure document signing requirements - #[instrument(skip(self))] - async fn setup_signing_requirements(&self) -> Result<()> { - info!("Setting up document signing requirements"); - - let url = format!("{}/v1/documents/data/config/signing_requirements", self.addr); - let payload = json!({ - "data": { - "total_required": 3, - "departments": [ - { - "name": "legal", - "required": 1, - "total_users": 5 - }, - { - "name": "finance", - "required": 1, - "total_users": 5 - } - ], - "description": "Requires 3 signatures total, with at least 1 from Legal and 1 from Finance departments" - } - }); - - let response = self.client - .post(&url) - .header("X-Vault-Token", &self.token) - .json(&payload) - .send() - .await?; - - match response.status() { - StatusCode::NO_CONTENT | StatusCode::OK => { - info!("Successfully configured signing requirements"); - Ok(()) - } - status => { - let error_text = response.text().await?; - error!("Failed to configure signing requirements: {} - {}", status, error_text); - Err(anyhow::anyhow!("Failed to configure signing requirements: {} - {}", status, error_text)) - } - } - } - - // Login a user and get their token - #[instrument(skip(self, password))] - pub async fn login_user(&self, username: &str, password: &str) -> Result { - info!("Attempting user login: {}", username); - - let url = format!("{}/v1/auth/userpass/login/{}", self.addr, username); - let payload = json!({ - "password": password, - }); - - let response = self.client - .post(&url) - .json(&payload) - .send() - .await?; - - match response.status() { - StatusCode::OK => { - let json: serde_json::Value = response.json().await?; - let token = json["auth"]["client_token"] - .as_str() - .context("Failed to extract client token")? - .to_string(); - - info!("User {} successfully logged in", username); - Ok(token) - } - status => { - let error_text = response.text().await?; - error!("Failed to login: {} - {}", status, error_text); - Err(anyhow::anyhow!("Failed to login: {} - {}", status, error_text)) - } - } - } - - // Get user info including department - #[instrument(skip(self))] - pub async fn get_user_info(&self, username: &str) -> Result { - debug!("Getting user info for: {}", username); - - let url = format!("{}/v1/documents/data/users/{}", self.addr, username); - - let response = self.client - .get(&url) - .header("X-Vault-Token", &self.token) - .send() - .await?; - - match response.status() { - StatusCode::OK => { - let json: serde_json::Value = response.json().await?; - let department_str = json["data"]["data"]["department"] - .as_str() - .context("Failed to extract department")?; - - let department = match department_str { - "legal" => Department::Legal, - "finance" => Department::Finance, - _ => { - error!("Unknown department: {}", department_str); - return Err(anyhow::anyhow!("Unknown department: {}", department_str)); - } - }; - - debug!("Retrieved user info for {} in {:?} department", username, department); - Ok(User { - username: username.to_string(), - department, - }) - } - status => { - let error_text = response.text().await?; - error!("Failed to get user info: {} - {}", status, error_text); - Err(anyhow::anyhow!("Failed to get user info: {} - {}", status, error_text)) - } - } - } -} diff --git a/test_docker.sh b/test_docker.sh index 1d8697d..12f4b32 100755 --- a/test_docker.sh +++ b/test_docker.sh @@ -1,4 +1,4 @@ -#!/usr/bin/env bash +#!/bin/bash set -eo pipefail # Colors for terminal output @@ -48,12 +48,6 @@ if ! command -v docker-compose > /dev/null 2>&1; then exit 1 fi -# Check if jq is available -if ! command -v jq > /dev/null 2>&1; then - log "ERROR" "jq command not found. Please install jq (JSON processor)." - exit 1 -fi - # Build the Docker image log "INFO" "Building Docker image..." docker-compose build @@ -105,26 +99,20 @@ wait_for_vault_init() { # Wait for vault-init to complete wait_for_vault_init -# Check if vault-credentials.json was created -if [ -f "vault-credentials.json" ]; then - log "INFO" "JSON credentials file was created successfully" -else - log "ERROR" "JSON credentials file was not created" - exit 1 -fi - -# Verify the content of vault-credentials.json -if jq -e '.keys_base64 | length' vault-credentials.json >/dev/null && \ - jq -e '.root_token' vault-credentials.json >/dev/null; then - log "INFO" "JSON credentials file contains expected content" -else - log "ERROR" "JSON credentials file doesn't contain expected content" - exit 1 -fi - -# Also check for backward compatibility +# Check if vault-credentials.txt was created if [ -f "vault-credentials.txt" ]; then - log "INFO" "Text credentials file was also created (for backward compatibility)" + log "INFO" "Credentials file was created successfully" +else + log "ERROR" "Credentials file was not created" + exit 1 +fi + +# Verify the content of vault-credentials.txt +if grep -q "Unseal Keys:" vault-credentials.txt && grep -q "Root Token:" vault-credentials.txt; then + log "INFO" "Credentials file contains expected content" +else + log "ERROR" "Credentials file doesn't contain expected content" + exit 1 fi # Verify Vault is unsealed after initial setup @@ -169,34 +157,28 @@ else echo $vault_status fi -# Extract keys and token from JSON credentials file -log "INFO" "Extracting unseal keys and root token from JSON credentials file..." -# Using jq to extract the token -root_token=$(jq -r '.root_token' vault-credentials.json) +# Extract keys from credentials file and root token +log "INFO" "Extracting unseal keys and root token from credentials file..." +unseal_keys=$(grep "Base64 Unseal Keys:" -A 3 vault-credentials.txt | grep "Key" | awk '{print $3}') +root_token=$(grep "Root Token:" vault-credentials.txt | awk '{print $3}') # First, try running 'vault operator unseal' directly for a more robust test log "INFO" "Attempting to unseal Vault directly with unseal keys..." -# Use jq to extract the keys directly into an array - more elegant -readarray -t unseal_keys_array < <(jq -r '.keys_base64[0:3][]' vault-credentials.json) +key1=$(echo "$unseal_keys" | head -n 1) +key2=$(echo "$unseal_keys" | head -n 2 | tail -n 1) +key3=$(echo "$unseal_keys" | head -n 3 | tail -n 1) -# Apply each key -for key in "${unseal_keys_array[@]}"; do - if [ -n "$key" ]; then - log "INFO" "Applying unseal key: ${key:0:8}..." # Show only first 8 chars for security - docker-compose exec -T vault env VAULT_ADDR=http://127.0.0.1:8200 vault operator unseal "$key" - fi -done +docker-compose exec -T vault env VAULT_ADDR=http://127.0.0.1:8200 vault operator unseal "$key1" +docker-compose exec -T vault env VAULT_ADDR=http://127.0.0.1:8200 vault operator unseal "$key2" +docker-compose exec -T vault env VAULT_ADDR=http://127.0.0.1:8200 vault operator unseal "$key3" # As a fallback, also try running vault-init with environment variables log "INFO" "Starting vault-init with environment variables..." -# Use the array to set environment variables -env_vars="-e VAULT_ADDR=http://vault:8200" -for i in "${!unseal_keys_array[@]}"; do - env_vars="$env_vars -e VAULT_UNSEAL_KEY_$((i+1))=${unseal_keys_array[$i]}" -done - -# Run the command with all environment variables -eval "docker-compose run $env_vars --rm vault-init" +docker-compose run -e VAULT_ADDR=http://vault:8200 \ + -e VAULT_UNSEAL_KEY_1=$(echo "$unseal_keys" | head -n 1) \ + -e VAULT_UNSEAL_KEY_2=$(echo "$unseal_keys" | head -n 2 | tail -n 1) \ + -e VAULT_UNSEAL_KEY_3=$(echo "$unseal_keys" | head -n 3 | tail -n 1) \ + --rm vault-init # Verify Vault is unsealed now vault_status=$(docker-compose exec -T vault env VAULT_ADDR=http://127.0.0.1:8200 vault status -format=json 2>/dev/null || echo '{"sealed": true}') @@ -215,16 +197,9 @@ fi # Test some basic Vault operations log "INFO" "Testing basic Vault operations..." -# Write a secret using the root token from JSON credentials +# Write a secret token_result=$(docker-compose exec -T vault env VAULT_ADDR=http://127.0.0.1:8200 vault login "$root_token" 2>&1) -login_success=$(echo "$token_result" | grep -c "Success" || echo "0") -if [ "$login_success" -gt 0 ]; then - log "INFO" "Successfully logged in with root token" -else - log "ERROR" "Failed to log in with root token" - echo "$token_result" - exit 1 -fi +log "INFO" "Login result: $(echo "$token_result" | grep "Success")" # Enable KV secrets engine enable_result=$(docker-compose exec -T vault env VAULT_ADDR=http://127.0.0.1:8200 vault secrets enable -path=kv kv 2>&1 || echo "KV already enabled") diff --git a/test_local.sh b/test_local.sh index 071f85e..903eb64 100755 --- a/test_local.sh +++ b/test_local.sh @@ -1,4 +1,4 @@ -#!/usr/bin/env bash +#!/bin/bash set -e # Detect OS and handle accordingly @@ -41,7 +41,6 @@ fi echo "Starting Vault server in non-dev mode..." # Create temporary config file -rm -fr /tmp/vault-test/data /tmp/vault-test/config mkdir -p /tmp/vault-test/data /tmp/vault-test/config cat > /tmp/vault-test/config/vault.hcl << EOF @@ -55,7 +54,7 @@ listener "tcp" { } disable_mlock = true -ui = false +ui = true EOF vault server -config=/tmp/vault-test/config/vault.hcl > ./vault_server.log 2>&1 & @@ -71,16 +70,16 @@ sleep 5 # Check if Vault is up and running for i in {1..10}; do - if wget -q -O- --no-check-certificate http://127.0.0.1:8200/v1/sys/health?standbyok=true\\&sealedok=true\\&uninitok=true > /dev/null 2>&1; then + if curl -fs -m 1 http://127.0.0.1:8200/v1/sys/health?standbyok=true\&sealedok=true\&uninitok=true > /dev/null 2>&1; then echo "Vault is up and running!" break fi - + if [ $i -eq 10 ]; then echo "Timed out waiting for Vault to become available" exit 1 fi - + echo "Vault is unavailable - sleeping (attempt $i/10)" sleep 2 done @@ -95,11 +94,11 @@ if [ -f "vault-credentials.txt" ]; then # Extract the unseal keys for demonstration UNSEAL_KEYS=$(grep "Key" vault-credentials.txt | head -n 3 | awk '{print $3}') ROOT_TOKEN=$(grep "Root Token" vault-credentials.txt | awk '{print $3}') - + echo "Root Token: $ROOT_TOKEN" echo "First 3 Unseal Keys (needed for threshold):" echo "$UNSEAL_KEYS" - + # Clean up temporary files rm -f vault-credentials.txt else @@ -116,4 +115,4 @@ rm "$VAULT_PID_FILE" rm -rf /tmp/vault-test rm -f ./vault_server.log -echo "All cleaned up. Test successful!" +echo "All cleaned up. Test successful!" \ No newline at end of file