ArchiveFlow – Open Source Data Archiving Kit

Self-hosted email archiving platform that sits between your mail servers and storage — ingest from Google Workspace, Microsoft 365, IMAP, or EML/Mbox uploads, then search everything instantly with retention policies and legal holds for compliance.

Stack

• SvelteKit — Frontend UI with Tailwind CSS and Shadcn components, server-rendered on its own dev port.
• Node.js + Express — Backend API and Drizzle ORM data layer, run as a separate service from the frontend.
• PostgreSQL + Drizzle ORM — Type-safe schema and migrations for archived messages and metadata.
• Meilisearch — Full-text search across message bodies, subjects, senders, and attachments.
• BullMQ + Valkey/Redis — Distributed task queue for parallel email ingestion and indexing.

Features

• Universal mailbox connectivity — Native ingestion from Google Workspace, Microsoft 365, standard IMAP servers, or EML/Mbox file uploads.
• Full-text search — Meilisearch indexes bodies, subjects, senders, and attachments — find any message in milliseconds.
• Compliance tooling — Retention policies, legal holds, and role-based access control for teams that need to prove what they kept and why.
• Distributed ingestion — BullMQ task queues parallelize large mailbox imports across worker processes instead of blocking on one at a time.

Install

git clone https://github.com/yashthakur1/OpenArchiver_6.git
cd OpenArchiver_6
npm install
cp .env.example .env

Environment

DATABASE_URL=postgresql://user:pass@localhost:5432/archiveflow
MEILI_HOST=http://localhost:7700
MEILI_MASTER_KEY=your-master-key-here
REDIS_HOST=localhost
REDIS_PORT=6379
JWT_SECRET=your-secure-random-string

Walkthrough

1. Clone the repository

Clone the monorepo and move into the project directory.

git clone https://github.com/yashthakur1/OpenArchiver_6.git
cd OpenArchiver_6

2. Install dependencies

Install all monorepo dependencies using npm workspaces — this covers the frontend, backend API, and workers in one pass.

npm install

3. Configure environment variables

Copy .env.example to .env and fill in your database, search, and queue credentials — DATABASE_URL, MEILI_HOST/MEILI_MASTER_KEY, REDIS_HOST/REDIS_PORT, and a JWT_SECRET for auth.

cp .env.example .env

4. Run database migrations

Generate and apply Drizzle migrations against your PostgreSQL database.

npm run db:generate
npm run db:migrate

5. Start the development environment

One command concurrently runs the SvelteKit frontend (http://localhost:3000), the backend API (http://localhost:4000), and the background workers that handle mailbox ingestion and indexing.

npm run dev:oss

6. Build and deploy

Build the production bundles, then start the app server and the background workers as separate long-running processes (e.g. under pm2 or systemd).

npm run build:oss
npm run start:oss

# in a separate terminal or service
npm run start:workers

Gotchas

• The dev/build/start scripts are the :oss variants (dev:oss, build:oss, start:oss) — the plain npm run dev/build won't start the full stack in this monorepo.
• Workers run as a separate process from the app server — forgetting to start npm run start:workers in production means ingestion silently stops even though the UI still loads.
• Large mailbox imports (100k+ messages) run through BullMQ in the background and can take hours on first sync.
• Meilisearch needs at least 1GB RAM for indexing large archives — bump it for bigger imports.
• Retention/legal-hold policies are enforced app-side — deleting rows directly in Postgres bypasses them, so always go through the API.

Why not just build it yourself?

If you need real-time collaboration, a full email client with sending, or compliance certifications (HIPAA, SOC2) out of the box, this isn't it. ArchiveFlow is read-only archival and search, not a mail client or a certified compliance platform.

Links

GitHub

License: MIT