All projectsProject 15 of 22 · Build Agents That Remember

Concentration

CONCENTRATION, named after the memory card game, finds the matching cards in a month of screenpipe history.

The project

The problem: To decide what to automate, you first need to know which tasks repeat and how much time they take. The starter's repeated-task finder ranks tasks by observed minutes, but only from records where someone typed Task:, Run: and Minutes: lines by hand. Nobody annotates a month of screen history, so in practice that ranking stays empty.

What we built: CONCENTRATION, named after the memory card game, finds the matching cards in a month of screenpipe history. It proposes those annotations itself, and a person confirms them before anything is ranked.

How it works:

  1. Ingest: screenpipe's OCR and accessibility text is grouped into sessions (one app and window, no pause over 5 minutes). Source IDs are kept all the way through, and memory and input (keystrokes) are refused.
  2. Embed and match: a local static embedding model (VTX-JEV-2, CPU only), with mean-centering, decides when two sessions are "the same step".
  3. Find tasks: step types that recur on 3 or more days, close together and in a consistent order, become a task, and each day's occurrence becomes a run. Sessions that break the usual pattern are flagged for review.
  4. Describe: a local LLM proposes a name and steps. Every quote is checked word for word against the screen text, and the LLM never decides what counts as a task.
  5. Review: a local web page where a person confirms or rejects tasks, removes noise, splits merged runs, and marks exceptions and troubleshooting.
  6. Export: confirmed runs feed the starter's ranking and its review desk.

On a synthetic month, it found 4 of 5 hidden tasks completely and 3 of 4 runs of the fifth. We show that miss on purpose: minutes are observed, never estimated. Everything runs locally.

How it’s built

Technologies: Python 3.12 (standard library, NumPy, SQLite); VTX-JEV-2, a static Model2Vec embedding model that runs on CPU; Ollama with qwen3:4b-instruct for local descriptions, with JSON-schema output; plain HTML and JavaScript for the review page; Bun and the screenpipe hackathon starter for ranking (bun run repetition) and the review desk; Make to run the pipeline.

screenpipe integration:

Input: our ingest calls screenpipe's local /search API (content_type=ocr,accessibility), page by page until it reaches pagination.total, with the starter's API-key headers. Captures become sessions in SQLite.
Privacy by design: we refuse memory (AI summaries, not exact text) and input (keystrokes can hold passwords).

Traceability: every capture keeps its screenpipe ID (ocr:<frame_id>, ui:<id>), from the sessions to the review page and into the starter's report, so every claim can be traced back to a screen.

Output: confirmed runs are exported in the starter's own Task: / Run: / Minutes: format, and one run becomes a review-desk draft with exact quotes and screenpipe://frame links.

Testing: a fake screenpipe serves synthetic data with the same /search API, so the whole pipeline runs end to end with an answer key.

🇫🇷 made with love <3 by @nass