ππ£π§-π°π© based agent // can work on your computer instead of you
We used GPT-4Vision to create a self-operating computer.
It can look at any UI and determine what clicks and keystrokes are needed to accomplish a task.
In this demo, we ask it to write a poem in Apple Notes.
GPT-4 viewed what was on the screen, found the open note, and then started clicking and typing the poem inside it.
Weβre just beginning to scratch the surface of whatβs possible by combining language, vision, and automation in a single AI system.
The future is here and weβre building it.
Shoutout to our teamβs new dad Josh Bickett, pushing boundaries with this demo β all while balancing family time with his newborn at home! [link]
