Déjà-vu — show once, run forever. A mail agent built on screenpipe that doesn’t guess: it replays what it saw you do. Handle a client’s request once (“can you resend the quote?”) and screenpipe learns the workflow: open the mail, find the right PDF in File Explorer, drag it into Gmail, reply in your style. From then on, Déjà-vu does it for you with the real mouse and keyboard. When a mail asks for something you saw on screen (a price, a delivery date, your availability), it finds it in screenpipe and cites the exact frame it came from. Every reply comes with a confidence score and its reasons. It only sends on its own above 80/100, and it never touches money or legal mails.
Déjà-vu is a Windows desktop agent built with Python and a Next.js 15 / TypeScript / Tailwind interface. It drives the real machine:
Selenium attaches to Brave through the DevTools port to read Gmail;
pyautogui moves the mouse and types the reply;
pywinauto controls File Explorer for the PDF drag-and-drop.
screenpipe is the agent’s memory and teacher. We recorded about 80 real reply sessions. extract_playbooks.py then queries screenpipe’s local REST API (/search with input and accessibility events) around each session to rebuild the actual gesture: open the mail, open File Explorer, select the file, drag it into Gmail, reply. It turns that into one playbook per request type. screenpipe’s own workflow pipes also learned the routine natively: it appears in its workflow catalogue (/workflows) and in its memories.
At run time, the agent searches past input events on the same file and shows the matching screenshot (/frames/{id}) as proof. It uses OCR search to find prices or dates the user saw on screen and cites them in the reply. It computes a confidence score: it sends alone only above 80/100, and never on money or legal mails. The Show me once button stars a session (/starred-sessions) to teach a new workflow. A screenpipe pipe (mail-autopilot) runs the agent every 15 minutes.