How can learning agents 'learn to use the web'?
One of the most compelling individual examples of progress with machine learning (ML) and artificial intelligence (AI) involves digital “learning agents” that work on ML algorithms to actually navigate the web, and to use specific in-page functionality in much the same ways that humans do.
Through the power of more sophisticated machine learning processes, computers have become able to “see” images and decipher what they mean. Engineers have been able to program AI technologies with an astonishing level of granularity – in the sense that computers can now “read” text off of a visual page with a high degree of literacy. This requires a remarkable amount of resources – to use raw pixel inputs to let the technology perceive shapes of letters, numbers and text characters – and then to use natural language processing to string those characters together, and come up with commands and responses.
However, another of the primary pathways to learning agent improvement is iteration. The programs are essentially “trained” to do the right thing from a human perspective, and refine their capabilities according to training sets.
An excellent example of all of this progress can be found at OpenAI's “Mini World of Bits” page that talks about reinforcement learning agents that perceive sets of raw pixels in a small webpage and can “produce keyboard and mouse actions.”
Web users can see the technologies outputting those keyboard and mouse events with user-like movements on small webpages: to operate drop-down list boxes, check boxes with logic, respond to text inputs, choose colors, and much more. OpenAI states that “one may utilize an unrestricted amount of pretraining on the training environments.”
All of this shows that artificial intelligence and machine learning are progressing rapidly, and that this will require human responses to keep up. The types of rote technology built into webpages to prove that a user is “not a robot” may have to be significantly upgraded in order to be effective as artificial intelligence essentially escapes some of the pens we've created for it. At the same time, there is an exciting set of applications for AI agents being able to use the web in a meaningful way – for a while now, people have talked about using artificial intelligence to improve recommendation engines, or go surf the web for results. Now, these same artificial intelligence agents could be used to work with controls on the web as well.
Tags
Written by Justin Stoltzfus | Contributor, Reviewer

Justin Stoltzfus is a freelance writer for various Web and print publications. His work has appeared in online magazines including Preservation Online, a project of the National Historic Trust, and many other venues.
More Q&As from our experts
- How does machine learning support better supply chain management?
- What is the difference between little endian and big endian?
- How can unstructured data benefit your business's bottom line?
Related Terms
- AI-Complete
- Machine Learning
- Deep Learning
- Chatbot
- Artificial Intelligence
- Random Access Memory
- Local Area Network
- Network Diagram
- Network Science
- Network Theory
Related Articles

The Awakening of Ambient Intelligence

Philosophy and Technology?

Machine Learning 101

Computer Programming: From Machine Language to Artificial Intelligence
Tech moves fast! Stay ahead of the curve with Techopedia!
Join nearly 200,000 subscribers who receive actionable tech insights from Techopedia.
- The CIO Guide to Information Security
- Robotic Process Automation: What You Need to Know
- Data Governance Is Everyone's Business
- Key Applications for AI in the Supply Chain
- Service Mesh for Mere Mortals - Free 100+ page eBook
- Do You Need a Head of Remote?
- Web Data Collection in 2022 - Everything you need to know