Running into this?

I take a screenshot and paste it to my agent, then spell out in words where to look: “the button at the top right”

I grab the screen with ⌥⇧S and box the spots to look at; each box gets a number, and I tell the agent by number.

before0 boxes on the image · the spot spelled out in words as “top right”
after1,020 numbered marks on 496 images in 7 days, 939 of them boxes
don't describe in words where to look in a screenshot; draw numbered boxes on the image and refer to them by number
Text for your agent · 16 lines
Situation: the person pastes a screenshot to you and describes in words where to look (“the button at the top right”, “the second row”). You ask “which one?” or change some other spot that also fits the description.

Why: the image reaches you as pixels; a written description like “top right” doesn't bind to one place in the image. A numbered box drawn on top of the image is part of the image: “the button in 1” points to exactly one place. In 7 days I drew 1,020 numbered marks on 496 images, 939 of them boxes.

Do:
1. Have the person install OptionOS (macOS): open the OptionOS disk image (.dmg) downloaded from https://optionos.app/download and double-click OptionOS in the window that opens: it installs itself into Applications and opens.
2. On first launch OptionOS first asks them to accept the terms and answer a few questions, then lists a few permissions on its permissions page. This job needs two: Screen Recording (System Settings calls it “Screen & System Audio Recording”) and Accessibility (called “Device Control and Data Access” on macOS 27). Have the person turn both on for OptionOS in System Settings › Privacy & Security; you can't grant them. No image is taken without permission: while one is missing, ⌥⇧S shows a “PERMISSION MISSING” card next to the pointer with the missing permission's name; if the person sees that card, the permission isn't on. If the setup questions ask about screenshots, answering “No” only hides the ⌥⇧S intro card; the shortcut still works.
3. Show the person: ⌥⇧S (Option + Shift + S) freezes the screen. Clicking a window takes that window, dragging takes that area (Esc cancels). The image opens in the editor; they draw a rectangle on every spot to look at (Rectangle is the tool selected on open): a number appears at each box's corner (1, 2, 3 …), in the box's color, red by default. Arrow, Line and Text marks get numbers too. A deleted mark's number isn't reused: delete 2, draw a new box, and the numbers are 1, 3. They press Done (⌘↩ does too): the image is copied to the clipboard with its boxes. In the editor, Esc or the window's red button is not Done: the boxes drawn are thrown away and nothing goes to the clipboard.
4. They paste the image to you and refer to it by number: “the label on the button in 1 is wrong, the row in 2 overflows”. Use the same number in your reply.
5. The same menu sits in the OptionOS menu bar under “Show what's on screen”: a person who forgets the shortcut picks “Screenshot + Annotate” there.

Verify (after the person presses Done and pastes the image to you):
- in the pasted image you see a number inside a filled circle at the corner of every mark (the Counter tool's mark carries its number inside its own circle, not at a corner); no numbers at all means no mark was drawn or the image came from another tool: go back to step 3 · if a file path arrived instead of an image (in a terminal ⌘V pastes the clipboard image as its path; the path looks like …/app.optionos.screencapture/annotation/<id>/rasters/<time>.png): open that file, it is the boxed image; don't count it as an unnumbered image and go back to step 3 · if what got pasted isn't this image (whatever was on the clipboard before): the person closed the editor with Esc instead of Done, or “Copy every screenshot to the clipboard” is off in OptionOS settings: go back to step 3
- right after Done, without taking another image in between, run on the person's Mac: A="$HOME/Library/Application Support/app.optionos.screencapture/annotation"; f=$(find "$A" -maxdepth 2 -name context.json -exec /usr/bin/stat -f '%Fm %N' {} + | sort -rn | head -1 | cut -d' ' -f2-); plutil -extract annotations json -o - "$f" | grep -o '"sequence":[0-9]*' → one line per mark drawn on the latest image ("sequence":1, "sequence":2 …; a deleted mark's number is skipped, so 1 and 3 is not incomplete) · empty output: the most recent image has no marks (if an unmarked image was taken after Done, that one is the latest · if the person pressed Esc instead of Done, the boxes they drew never entered the record) · “Empty path specified” (possibly preceded by “No such file or directory”): no image has ever been taken on this Mac, go back to the permissions in step 2

Don't: when the person's image has no numbers, don't try to find the spot by guessing anyway; ask them to take it again with boxes. Don't draw the boxes yourself: a box means the spot the person wants looked at, and only the person knows it. Don't find the latest record for Verify with `ls -t "$A"/*/context.json`: once there are thousands of records (each image is a folder) the shell prints “argument list too long”, and a full Mac shows “Empty path specified” as if no image had ever been taken.