Heelix: Privacy-First Task Mining on the Desktop with Tauri, Rust, and Local LLMs

# Heelix: Privacy-First Task Mining on the Desktop with Tauri, Rust, and Local LLMs

Most productivity tools ask you to log what you're doing. Heelix figures it out for you — analyzing your work patterns locally, surfacing project context automatically, and tracking progress without a single byte leaving your machine. Here's the architecture behind it.

What Is Task Mining?

Task mining captures how work actually happens — not how process documents say it should happen. For knowledge workers, that means understanding:

• Which applications and files are in use, and when
• How time distributes across projects and tasks
• When context-switching fragments focus
• What work is happening versus what was planned

Traditional task mining tools are enterprise, cloud-based, and invasive. Heelix takes the opposite approach: everything stays local.

Why Tauri + Rust

Tauri: Lightweight Desktop Shell

Electron ships a full Chromium browser. Tauri uses the operating system's native webview — on macOS, that's WebKit; on Windows, WebView2; on Linux, WebKitGTK. The difference in binary size and memory consumption is dramatic:

For an app meant to run quietly in the background all day, every megabyte matters. Heelix's Tauri shell uses less than 50 MB of RAM at rest — invisible alongside the browser tabs and IDEs it's monitoring.

Rust: The Performance Core

Tauri's backend is Rust, and we put it to work. Heelix's Rust layer handles:

• Window and input monitoring — Low-level OS APIs via platform-specific crates (xcb/x11 on Linux, CGWindowList on macOS, Windows-rs on Windows)
• File system watching — notify crate for real-time project file change detection
• Text extraction — OCR of window titles and active document content via platform accessibility APIs
• Embedding inference — Running sentence-transformers for text embeddings using candle (HuggingFace's Rust ML framework)
• LLM inference — Running quantized local models via llama.cpp bindings, with GPU acceleration through Metal (macOS) or Vulkan (cross-platform)
• SQLite storage — All activity data in a local encrypted database, never transmitted

┌─────────────────────────────────────┐
│           Tauri Shell                │
│  ┌──────────────────────────────┐   │
│  │   React Frontend (TypeScript) │   │
│  │   - Timeline visualization    │   │
│  │   - Project dashboards        │   │
│  │   - Context cards             │   │
│  └──────────┬───────────────────┘   │
│             │ IPC (JSON)             │
│  ┌──────────▼───────────────────┐   │
│  │      Rust Backend             │   │
│  │  ┌────────────────────────┐  │   │
│  │  │ Activity Monitor        │  │   │
│  │  │ (window tracking,       │  │   │
│  │  │  filesystem events)     │  │   │
│  │  └────────┬───────────────┘  │   │
│  │           ▼                   │   │
│  │  ┌────────────────────────┐  │   │
│  │  │ Processing Pipeline     │  │   │
│  │  │ - Text extraction        │  │   │
│  │  │ - Embedding generation   │  │   │
│  │  │ - Clustering & labeling  │  │   │
│  │  └────────┬───────────────┘  │   │
│  │           ▼                   │   │
│  │  ┌────────────────────────┐  │   │
│  │  │ Local LLM Inference     │  │   │
│  │  │ - Task classification    │  │   │
│  │  │ - Context summarization  │  │   │
│  │  │ - Progress estimation    │  │   │
│  │  └────────────────────────┘  │   │
│  └──────────────────────────────┘   │
└─────────────────────────────────────┘

Local LLM Processing: Intelligence Without the Cloud

The core insight behind Heelix is that modern small language models are good enough for task understanding — and they run on consumer hardware.

Model Selection

We ship with Phi-3-mini (3.8B parameters, Q4_K_M quantization, ~2.2 GB) as the default model. Users can swap in any GGUF-compatible model. On an M2 MacBook Air, Phi-3-mini generates at ~25 tokens/second — more than fast enough for batch processing of activity summaries.

What the LLM Actually Does

The LLM runs on a schedule (every 5–15 minutes, configurable) and processes:

1. Task Classification — Given a window of activity (application, file path, window title, active duration), classify the task: "Writing documentation for auth module", "Reviewing PR #342", "Debugging payment webhook timeout"

1. Context Summarization — When the user switches to a project, generate a concise summary of recent activity: "You were working on the payment integration. Last change was in src/billing/webhook.ts — you added retry logic. Tests are passing."

1. Progress Estimation — Compare current activity against project milestones (pulled from git branches, issue trackers, or manually defined goals) and provide a rough progress estimate: "Auth module appears ~70% complete based on commit activity and issue resolution."

Prompt engineering for these tasks is tuned for the small model size:

System: You are a task classifier for a productivity tool.
Given window activity data, output a concise task label.
Be specific. Include file names when relevant.
Respond in JSON: {"task": "...", "project": "...", "confidence": 0.0-1.0}

User:
- Application: VS Code
- Window title: webhook.ts — payment-service — Heelix
- Active duration: 23 minutes
- Recent keystrokes (sampled): retry logic, exponential backoff...

Privacy by Architecture

Because everything runs locally:

• No data leaves the device — Window titles, file paths, keystroke samples never touch a server
• Encrypted at rest — SQLite database encrypted with SQLCipher, key derived from OS keychain
• User controls granularity — Exclude specific applications, file paths, or time windows
• Opt-in sharing — If the user wants team visibility, they explicitly enable and configure it

This privacy-first approach makes Heelix viable for industries (finance, healthcare, legal) where cloud-based monitoring tools are non-starters.

The Frontend: React + D3

The UI is a React app rendered in the native webview, communicating with the Rust backend via Tauri's typed IPC bridge. Key views:

• Timeline — Scrollable activity timeline with task color-coding, built with a custom D3.js timeline component
• Project Dashboard — Per-project cards showing recent activity, progress estimates, and context summaries
• Focus Analytics — Context-switch frequency, deep-work blocks, distraction patterns
• Settings — Model selection, exclusion rules, privacy controls

The frontend subscribes to real-time updates from the Rust backend via Tauri events, so the timeline populates as you work.

Project Context: Understanding Progress

Heelix connects to local git repositories to enrich activity data with project context:

• Branch detection — Which feature branch is active
• Commit velocity — Commits per day, lines changed
• Issue linking — Parses branch names and commit messages for issue references
• File churn — Which files get the most attention

Combined with LLM-classified task labels, this gives a rich picture of what's actually happening in a project — without manual status updates.

Lessons from Building It

1. Small models punch above their weight — Phi-3-mini on Q4 quantization handles task classification with >85% accuracy, measured against manual labels from our dogfooding team
2. Rust's ecosystem for ML is maturing fast — candle and llama.cpp bindings cover 90% of what we need; the remaining 10% (sentence-transformers tokenization quirks) required custom work
3. Tauri's IPC is surprisingly capable — Passing embedding vectors as JSON between Rust and JS works fine up to ~1000 dimensions; beyond that, we use shared memory
4. Privacy is a feature, not a constraint — Users who would never install a cloud-based monitoring tool are enthusiastic about a local one

---

Interested in Heelix or building privacy-first desktop AI? Let's talk.