Deepmind reimagines mouse cursor as context variable

deepmin researchers adrien baranes and rob marchant proposed 'pointer engineering' to shift ai interaction from precise prompts to context-aware cursor inputs. the method treats the cursor as a variable, capturing surrounding visual data to execute commands like 'fix this' or 'move that' with minimal user effort. pixels are interpreted as structured entities, allowing a handwritten note to become a to-do list or a video frame to become a booking link. this functionality is live in gemini for chrome and will debut as 'magic pointer' on the googlebook, streamlining casual interactions without replacing prompt engineering for complex tasks.