Asking a vision model what and where in the same call makes it worse at both
This story is from 2026-09-25. It is preserved in the archive; the latest stories are on the live feed.
Handrail takes a screenshot of whatever you are stuck in, answers your question about it, and then draws an arrow on the real control you need to touch. The obvious way to build that is one call: here is the screen, here is the question, give me the answer and the coordinates. That is how I built i…
Read the full story at DEV Community — AI ↗
Timeline · 1 report
- 2026-09-25 12:23 · DEV Community — AI
Asking a vision model what and where in the same call makes it worse at both