Module storage

Storage Service

This module provides the storage abstraction layer for the Search Engine Backend. It handles all interactions with Azure CosmosDB for storing web pages, crawl queues, and search statistics.

Key Components

• StorageService: Main service for CosmosDB operations
• WebPage: Document structure for indexed web pages
• CrawlQueue: Queue management for crawling operations
• SearchStatistic: Analytics data for search operations

Usage

use search_engine_backend::{Config, StorageService};
use std::sync::Arc;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let config = Arc::new(Config::from_env()?);
    let storage = StorageService::new(config).await?;
     
    // Storage service is now ready for use
    Ok(())
}

Structs

CrawlQueue

Represents a crawl queue entry for managing web crawling operations.

GLOBAL_LOG_BUFFER

Global log buffer instance

LogBuffer

Thread-safe log buffer for capturing application logs

LogEntry

Structure to hold captured log entries for display in the dashboard

SearchStatistic

Represents search analytics and statistics data.

StorageService

Storage service for Azure CosmosDB operations.

WebPage

Represents a web page document stored in the database.

Enums

CrawlStatus

Constants

COSMOS_API_VERSION 🔒

Azure Cosmos DB REST API version

MAX_LOG_ENTRIES 🔒

Maximum number of log entries to keep in memory