OpenAI has recently announced an exciting new feature for its ChatGPT desktop app, specifically designed for macOS users. This feature allows ChatGPT to interact directly with several popular coding applications, such as VS Code, Xcode, TextEdit, Terminal, and iTerm2.
What Does This Mean for Developers?
In the past, developers often had to copy and paste their code into ChatGPT to get assistance or suggestions. This process was a bit cumbersome and time-consuming. Now, with the new feature called “Work with Apps,” developers can allow ChatGPT to automatically access the code they are currently working on. When this feature is turned on, ChatGPT will receive the relevant section of code along with any questions or prompts the developer has. This means less hassle and a smoother workflow.
However, it’s important to note that, unlike some other AI coding tools like Cursor or GitHub Copilot, ChatGPT currently cannot write code directly into the coding apps. Instead, developers will still need to copy ChatGPT’s suggestions and paste them back into their coding environment.
How Does It Work?
In a demonstration, an OpenAI employee showcased how this feature works using Xcode, a popular coding tool. The employee was working on a project that modeled the solar system but had forgotten to include Earth. By selecting the Xcode tab in the ChatGPT app and asking the AI to “add the missing planets,” ChatGPT was able to generate the necessary code for Earth, which matched the existing format. However, the employee still had to manually paste this code back into the Xcode project.
To make this interaction possible, OpenAI is utilizing macOS’s accessibility features, which allow ChatGPT to read text from other applications. This accessibility technology has been available for nearly 20 years and is generally reliable for most common applications. However, it does have limitations; for example, it can only read text and cannot interpret visual elements like images or videos.
What’s Next?
For some applications, like Microsoft’s VS Code, users may need to install a special extension to allow ChatGPT to access the code. The “Work with Apps” feature will send the last 200 lines of code to ChatGPT along with each prompt for certain applications. For other apps, it will use all the code visible in the active window. Users can also highlight specific sections of code to help ChatGPT focus on the relevant parts of their project.
Read Also: Musk’s Updated Lawsuit Against OpenAI Includes Microsoft As A Defendant
Looking ahead, OpenAI plans to expand this feature to more applications, especially those related to writing tasks. While the current focus is on coding apps, OpenAI envisions a future where ChatGPT can understand and work with a wider range of software.
Future Developments
This new capability is seen as a stepping stone towards creating more advanced AI systems that can operate like agents, understanding and interacting with various content on your computer. OpenAI’s product lead, Alexander Embiricos, mentioned that while this feature isn’t a fully-fledged AI agent yet, it is an important step in that direction.
There are also reports that OpenAI is working on a more general-purpose AI agent, codenamed “Operator,” which is expected to be released in early 2025. This tool aims to compete with other AI agents being developed by companies like Anthropic and Google.
Currently, this feature is only available for macOS users, and there are plans for integration with Apple products in December. However, it remains unclear when similar functionalities will be available for Windows users.
In summary, OpenAI’s new feature for ChatGPT is set to make coding easier for developers by allowing the AI to read and understand their code directly within their coding applications, paving the way for more sophisticated AI interactions in the future.
