- Joined
- Aug 19, 2017
- Messages
- 2,579 (0.97/day)
The age of automation is upon us. Anthropic, the company behind top-performing Claude large language models that compete directly with OpenAI GPT, has today announced updates to its models and a new feature—computer use. The computer use allows Claude 3.5 Sonnet model to access the user's system by looking at the screen, moving the cursor, typing text, and clicking buttons. While only being experimental for now, the system is prone to errors and creating "dumb" mistakes. However, it allows for one very important feature: driving the operating system designed for humans using artificial intelligence.
There is a benchmark that evaluates AI model's ability to use computers like a human does on human-centered operating system. Called OSWorld, the Claude 3.5 Sonnet model has managed to score 14.9% in screenshot-only category, and 22.0% in some other tasks that require more steps. A typical human scores around 72.36% in this testing, which proves to be difficult even for natural intelligence. However, this is only the beginning as these models advance rapidly. Usually, these models work with other types of data, like text and static images, where they process it and do computation based on it. Working on computers designed for human interaction first is a great leap in the capabilities of AI models.
Finally, Anthropic pushed an update to its Claude 3.5 Sonnet and Hauiku models, which are now more capable than ever for various tasks. Below are the company's evaluations comparing it to the older models, as well as OpenAI and Google's state-of-the-art AI designs.
The comparison lacks OpenAI o1 models, as those are based on different techniques compare to the LLMs.
View at TechPowerUp Main Site | Source
There is a benchmark that evaluates AI model's ability to use computers like a human does on human-centered operating system. Called OSWorld, the Claude 3.5 Sonnet model has managed to score 14.9% in screenshot-only category, and 22.0% in some other tasks that require more steps. A typical human scores around 72.36% in this testing, which proves to be difficult even for natural intelligence. However, this is only the beginning as these models advance rapidly. Usually, these models work with other types of data, like text and static images, where they process it and do computation based on it. Working on computers designed for human interaction first is a great leap in the capabilities of AI models.
Finally, Anthropic pushed an update to its Claude 3.5 Sonnet and Hauiku models, which are now more capable than ever for various tasks. Below are the company's evaluations comparing it to the older models, as well as OpenAI and Google's state-of-the-art AI designs.
The comparison lacks OpenAI o1 models, as those are based on different techniques compare to the LLMs.
View at TechPowerUp Main Site | Source