Application · Desktop assistant · v0.9.1

AbstractAssistant

The desktop assistant of AbstractFramework: a compact chat palette one click away (on its menu-bar or system-tray icon, or ⌘⇧Space from any app), a hands-free voice conversation mode and a small CLI. It is a thin client of AbstractGateway: the gateway runs the workflows, holds the provider connections and executes the tools, and the app keeps only your local preferences and caches. It is OS-agnostic: the same Python app runs on macOS, Linux and Windows desktops that have a system tray, and has been tested most on macOS (see Platforms).

Get started GitHub PyPI Docs llms.txt llms-full.txt

pip install "abstractassistant[voice]"

# the desktop app (menu bar or system tray)
assistant
# one turn in the terminal
assistant run --prompt "Summarize my notes"
# name the gateway and sign in
assistant --gateway-url http://127.0.0.1:8080 \
  --gateway-token <token>

Overview

AbstractAssistant — One Click Away

AbstractGateway owns workflow discovery, durable runs, provider connections, tool execution and policy, workspace policy and multimodal defaults. The Assistant keeps your preferences, window placement, downloads, a rebuildable cache of your sessions, and the overrides you choose for this app.

Tray / Palette / CLI  ──▸  AbstractGateway  ──▸  AbstractRuntime  ──▸  AbstractCore  ──▸  Providers

The AbstractAssistant palette in the dark theme: header with the chat name and the new chat, tools, schedule, speaker and settings buttons and a green connection dot; a question about steeping tea, the answer with its statistics chips (10 in, 5 out, 0 tools, 0 files, 37 ms), and the composer with attach, microphone and voice conversation buttons — The palette with a short conversation.

The palette, a thin client of your gateway

Click the menu-bar (or system-tray) icon, or press the summon shortcut (⌘⇧Space by default), and the palette opens. Its title says what the app is doing: Running, Reconnecting, Listening, Speaking or Not sent. The header names the current chat, which opens the switcher, and holds the buttons for a new chat, tool permissions, Schedule this conversation…, spoken replies and Settings. The dot on the right shows whether the gateway is reachable.

Each reply ends with its statistics: tokens, tools, files, duration and model, with detail views
While a run works, an activity card shows the current step, the elapsed time and recent tool calls, with pause, resume and stop
With Stream replies on, the answer appears while the model writes it, its reasoning folded under Thinking
Attach files with the button or drag and drop; typed text during a run steers it

The workflow each turn runs

Each turn runs the gateway's default workflow for the Assistant (abstractassistant.agent.v1). When the gateway sets none, the app runs its built-in orchestrator, abstractassistant-orchestrator, which it publishes to your tenant catalog. You can pick another one in Settings → Models & reasoning → Workflow.

Providers stay on the gateway

You connect local engines and cloud providers, and keep API keys, on the gateway (for example in the console). The Assistant needs no API key or model file. It reads the gateway's providers and models, and you can pin a different one for this app, per route.

Your sessions, from every device

The switcher lists every chat on the gateway, whichever client started it: the Assistant on any computer, AbstractCode or the Observer. Sessions live on the gateway, so every device shows the same list.

Platforms

One app on macOS, Linux and Windows

AbstractAssistant is one Python app (Qt, with a Qt system-tray icon and a global shortcut). It is OS-agnostic: on macOS, Linux and Windows it installs with pip or from the gateway console and runs with the same palette, voice mode and CLI. It has been tested most on macOS, which also has a packaged form, the menu-bar app bundle.

	macOS	Linux	Windows
Tray icon	Menu-bar icon; click opens the palette	System-tray icon where the desktop has a tray host (GNOME needs the AppIndicator extension); otherwise the app opens as a window and says why	System-tray icon; click opens the palette, right-click shows its menu
Global shortcut	`⌘⇧Space`, once macOS grants Accessibility (and Input Monitoring)	The same default, with Super for `⌘`	The same default, with the Windows key for `⌘`
Voice	Microphone capture and streaming playback in the base install on every platform; speech-to-text and text-to-speech run on the gateway. Playback uses `afplay` on macOS, `winsound` on Windows, and `paplay`, `aplay`, `ffplay` or `mpg123` on Linux.
Notifications	Tray notifications (the icon's message balloon) on every platform with a tray
Install	From the gateway console's Apps page, or `pip install abstractassistant`; the `abstractframework` Python package includes it
App bundle	`build-macos-app` builds `AbstractAssistant.app` (unsigned, menu-bar only)	None: run the `assistant` command

When the shortcut cannot be registered, the icon's tooltip says so and the icon still opens the palette. The CLI (assistant run --prompt) needs no tray at all.

Install

Open it from the console, or install it with pip

You need Python 3.10 or later and an AbstractGateway you can reach. The console and pip installs work on every platform the app runs on; the app bundle is macOS only.

# Gateway console > Apps > Assistant card: Install (when it is missing), then Open
# The same from a terminal on the gateway's computer:
abstractgateway apps install assistant
abstractgateway apps launch assistant

Install adds abstractassistant to the gateway's own Python environment, with every abstract* package the gateway runs pinned to its current version, so installing the Assistant never changes the gateway. The abstractframework Python package already includes it.
Open starts the Assistant on the gateway's computer, already signed in as you, and the app remembers that sign-in. The console can sign in only an Assistant it launches, so quit a running Assistant before you click Open. From another computer, the card says the Assistant runs on the gateway's computer.
The gateway's own tray icon has a Launch Assistant entry that finds the app the same way.

pip install "abstractassistant[voice]"   # [voice]: adds local speech-to-text

assistant                                          # launch the desktop app
assistant run --prompt "What changed in the repo today?"
assistant run --prompt "Explain this error" --stream on
assistant --gateway-url http://127.0.0.1:8080 --gateway-token <token>

The base install covers text chat, spoken replies, gateway-backed media and microphone capture for dictation and voice conversation; the voice extra adds local speech-to-text. Speech-to-text and text-to-speech run on the gateway either way.
assistant run prints the final answer once on stdout; --stream on|off overrides the saved Stream replies choice for that turn. The command is also installed as abstractassistant.
--gateway-url alone is enough once you have signed in to that gateway: the app reuses the sign-in saved in ~/.abstractassistant/gateway_connection.json.

# From a checkout of the repository
pip install -e ".[macos-app]"
build-macos-app                  # builds AbstractAssistant.app and installs it into /Applications
build-macos-app --skip-install   # leaves it in dist/macos/AbstractAssistant.app

The bundle is unsigned and not notarized: an app you build yourself opens normally, but a copy sent to someone else is blocked by Gatekeeper. It is menu-bar only, so look for its icon in the menu bar, not the Dock. build-macos-app runs on macOS only; there is no Linux or Windows package.

How the Assistant finds your gateway. The first match wins: --gateway-url; the sign-in saved in Settings → Connection; this computer's gateway (with AbstractGateway in the same Python environment, the gateway's own rule; otherwise, as in the app bundle, the local pointer file ~/.abstractframework/gateway.json that the installer and abstractgateway serve write); then http://127.0.0.1:8080. So a gateway that the installer moved off a busy port 8080 is still found. The pointer file never holds a token.

Voice

Talk to it, hands-free

Microphone capture and playback happen on your computer; speech-to-text and text-to-speech run on the gateway (the installer gives it local voice through AbstractVoice). Every reply has a speaker button, the composer's microphone dictates into the message, and ⌘⇧V starts a voice conversation.

The palette during a voice conversation: the title reads Listening in green, and a strip above the composer shows a microphone glyph, a live level meter, the text Listening… and the pause and end buttons — Voice conversation mode: the strip above the composer.

Listen, send, speak, listen again

Start with ⌘⇧V or the waveform button next to send. A strip above the composer shows a level meter and the state: Listening…, Heard: …, Thinking… (mic paused), Speaking…. Each utterance becomes a turn after a short pause, and spoken replies stream from the gateway, so long answers start playing within about a second.

The microphone pauses while the assistant works, so side conversation never steers a run
Short, spoken-style replies by default; typing still works during a conversation
Barge-in: keep the mic open with headphones and say "stop" to interrupt a reply
Replies play on the computer's current output device, or the one you pick in Settings → Voice

Keyboard shortcuts

Shown with macOS keys. On Linux and Windows, ⌘ in the global shortcut is the Super / Windows key, and the palette's own shortcuts use Ctrl where macOS uses ⌘.

Keys	Action
`⌘⇧Space`	Summon the palette from any app (the default; change it in Settings → Appearance). It works once the app may read the keyboard: System Settings → Privacy & Security → Accessibility / Input Monitoring.
`⌘⇧V`	Start or end a voice conversation
`⌘N` · `⌘,`	New chat · Settings
`⌘.`	Stop the running reply
`⌘1` · `⌘2`	Switcher tabs: Sessions · Automations
`Return` · `Shift`+`Return`	Send · new line

Sessions and automations

Your chats and your automations, side by side

Click the chat name in the header. The switcher has two tabs, Sessions | Automations, both fed by the gateway. An automation runs a task on the gateway on a fixed UTC schedule; it keeps running while the Assistant is closed or the computer sleeps.

The switcher on its Automations tab: Show archived and + New automation at the top, then four automations. Inbox triage, every 8 hours, Active; Downloads clean-up, every 7 days, marked waiting for you, Active; Weekly dependency check, every 7 days, next —, Paused; Morning research digest, every 24 hours, run #2, Active — The switcher's Automations tab.

One card per automation

Each card shows when the automation last ran, its last result, the schedule, the next run, the run count and the workspace folder, which opens in your file manager (Finder on macOS) when it is on this computer. A chip says "waiting for you" or "failed" only when it applies. The button on the right says the automation's state in words: Active ▶ in green, Paused ⏸ in amber; click it to pause or resume.

The same automations as AbstractCode and the Observer, whichever app created them
+ New automation for a new task; Show archived for the history
The tab label and Automations… in the tray icon's menu count what needs you

An automation opened in the palette: ← Chat, the title Morning research digest, the line every 24 hours (UTC) · Active · next in 23 h · Growing · 2 runs, the controls Pause, Resume, Run now with its play-in-a-circle icon, Stop current, Edit and Archive, then run #1 as a chat turn with its task and the answer listing three papers — An automation's runs read as a chat.

Open an automation

Click a card and the palette shows the automation in place of the chat; ← Chat returns. Each run reads as a pair of chat turns: what fired it and the task, then the answer. Quiet runs are dimmed, notable results carry a Notified badge, and failures give the gateway's reason and the number of attempts.

Control	What it does
Pause / Resume	No scheduled run until you resume; resuming continues at the next scheduled time
Run now	One run immediately; the schedule does not move
Stop current	Stops the run in progress
Edit	Title, interval and context mode, from the next run
Archive	Nothing runs any more; the history is kept (confirmed in the palette)

Run now, in the words of its tooltip (the same hint as in the web clients): “Run it once now, without waiting for the schedule. The schedule does not move: the next scheduled run keeps its time, or starts right after this run if its time comes first. Does not count toward a run limit. Works while paused; it stays paused. Not available while a run is in progress.” The tooltip then adds the next scheduled time and, for a Growing automation, “Growing context: later runs see this run in their history.” A disabled control's tooltip gives the reason first.

Schedule this conversation

The clock button in the header opens Schedule this conversation…, prefilled with the conversation's workflow and its last question, which you can edit. + New automation opens the same window empty.

When (UTC): every 5 minutes, 30 minutes, hour, 8 hours, 24 hours or 7 days, every N minutes, hours or days, or once at a date and time
Context: Independent (each run starts fresh) or Growing (each run sees the previous runs)
Tools: Tools run without asking (creating the automation is your approval) or Ask each time

The Downloads clean-up automation with a run waiting for you: the banner 1 occurrence(s) waiting for you, the task List my old downloads and suggest what to archive, and a highlighted answer turn that reads The run asks to use these tools, listing an execute_command call with its arguments, with Approve and Deny buttons — A run that waits for your approval.

Answer a run, discuss a result

A run can stop and wait for you. It shows what it waits for: a question with its choices and a free-text answer, a tool approval with Approve / Deny (for automations set to Ask each time), or an event payload typed as JSON.

Discuss under a finished run opens an ordinary chat seeded with the automation's history up to that run. It has its own writable workspace, with the automation's files mounted read-only, and it appears among your chats with the badge about automation <title>.

Tray notifications

You get a notification once for each result the workflow marks as notable, each run that failed after all its retries, and each run waiting for you ("<title> is waiting for you", or "<title> needs your approval"). Ordinary results never notify, and nothing is repeated after a relaunch.

When it checks

The Assistant reads the automation list every 60 seconds while the palette is visible, and every 5 minutes while it is hidden, so a notification reaches you while you work in other apps. Opening an automation marks its new items as seen on every device.

The workspace folder

Every session and automation card has a folder icon for the workspace the gateway reports. It opens in your file manager when the folder is on this computer; until the gateway reports the folder, the icon is disabled and says so.

Tools and settings

You decide what runs, and what this app overrides

Tool approvals

When a workflow wants a tool you have not pre-approved, a sheet opens beside the palette. It lists each call with the gateway's risk tier and the command, path or query it will use. Allow once (Return) runs this batch, Deny (⌘D) refuses it, and Decide later (Esc) keeps the run waiting. The palette stays usable while the sheet is open.

"Always allow … on this Mac" stops the questions for one tool
"Allow all enabled tools in this chat" trusts a single chat
Per-tool Off / Auto / Ask in Settings → Tools & permissions

The Settings window on Tools & permissions: the sidebar lists Connection, Models, Voice, Workspace, Tools, Appearance and About; the page shows the gateway's tool mode (approval), a tool filter, and the files toolset with analyze_code (Reads only, Auto), analyze_media (Reads only, Ask) and edit_file (Makes changes, Ask), and the buttons Use gateway defaults and Save — Settings → Tools & permissions.

Seven Settings sections

Connection, Models & reasoning (workflow, reasoning effort, reply streaming), Voice, Workspace (root and allowed folders), Tools & permissions, Appearance & window and About. Every value says whether it is the gateway default or this app's override.

Each route the Assistant drives (chat model, voice output and input, music, sound effects, image generation, edit and upscale, video, image to video) has its own Provider and Model lists. The first item is always Gateway default; any other choice is kept on this computer and sent with each request, and the gateway's shared defaults never change.

Examples

A few things to try

Ask a question without typing

Press ⌘⇧V. The strip reads Listening….
Ask your question and pause; the strip shows Heard: …, then Thinking… (mic paused).
The answer is spoken (Speaking…) and the loop listens again for a follow-up.
Press ⌘⇧V or the strip's ✕ to end.

Turn a conversation into a daily digest

Ask the question once, for example "Summarise the most discussed machine-learning papers of the last 24 hours."
Click the clock button: Schedule this conversation… opens with that question as the task.
Under When (UTC), pick every 24 hours; under Context, Growing.
Click Schedule. The automation opens in the palette, and its first run is due at once unless you set a first-run time.

Approve a run from the notification

A notification says "Downloads clean-up needs your approval".
Choose Automations… in the tray icon's menu (or ⌘2 in the switcher).
Click the card marked "waiting for you".
Read the tool call and its arguments, then click Approve or Deny.

Discuss a result

Open the automation and scroll to the run you want.
Click Discuss under its answer and type your question in "What do you want to discuss about this result?".
Click Start. A new chat opens, seeded with the runs up to that one, and stays in your Sessions tab with the badge about automation <title>.

Use another workflow or model for this app

Press ⌘, and open Models & reasoning.
Under Workflow, pick another assistant workflow; it applies from the next turn.
Under the Chat model route, choose a Provider and a Model. Pick Gateway default to follow the gateway again.

Local files

What stays on your computer

The gateway is the source of truth for workflows, provider and model defaults, tools and their approval defaults, workspace policy, run history, sessions, automations and artifacts. The Assistant stores the rest under ~/.abstractassistant/.

File	Holds
`preferences.json`	This app's overrides and preferences: model routes, reasoning effort, reply streaming, workspace grant, voice options, tool modes, hotkey, window size
`gateway_connection.json`	Gateway URL and sign-in state
`session_cache.json`, `sessions/`	A rebuildable cache of the gateway's sessions; the sessions themselves live on the gateway
`automations_notified.json`	Which automation notifications were already shown
downloads, cached audio	Files fetched from the gateway and spoken replies

Guides in the repository: Getting started · Voice · Automations · Settings · Installation · Troubleshooting · Changelog

Where to next

Keep going

User journeyInstall, pick a model, and say hello from the palette. Install AbstractFrameworkOne download or one line sets up the gateway and its console. Web & terminal consoleThe Apps page that installs and opens the Assistant. AutomationsHow scheduled conversations run, from the Assistant, Code or the Observer. AbstractGatewayThe control plane the Assistant talks to.