Skip to main content
AgentQL tools provides web interaction and structured data extraction from any web page using an AgentQL query or a Natural Language prompt. AgentQL can be used across multiple languages and web pages without breaking over time and change.

Overview

AgentQL provides the following three tools:
  • ExtractWebDataTool extracts structured data as JSON from a web page given a URL using either an AgentQL query or a Natural Language description of the data.
The following two tools are also bundled as AgentQLBrowserToolkit and must be used with a Playwright browser or a remote browser instance via Chrome DevTools Protocol (CDP):
  • ExtractWebDataBrowserTool extracts structured data as JSON from the active web page in a browser using either an AgentQL query or a Natural Language description.
  • GetWebElementBrowserTool finds a web element on the active web page in a browser using a Natural Language description and returns its CSS selector for further interaction.

Integration details

Tool features

Setup

To run this notebook, install Playwright browser and configure Jupyter Notebook’s asyncio loop.

Credentials

To use the AgentQL tools, you will need to get your own API key from the AgentQL Dev Portal and set the AgentQL environment variable.

Instantiation

ExtractWebDataTool

You can instantiate ExtractWebDataTool with the following params:
  • api_key: Your AgentQL API key from dev.agentql.com. Optional.
  • timeout: The number of seconds to wait for a request before timing out. Increase if data extraction times out. Defaults to 900.
  • is_stealth_mode_enabled: Whether to enable experimental anti-bot evasion strategies. This feature may not work for all websites at all times. Data extraction may take longer to complete with this mode enabled. Defaults to False.
  • wait_for: The number of seconds to wait for the page to load before extracting data. Defaults to 0.
  • is_scroll_to_bottom_enabled: Whether to scroll to bottom of the page before extracting data. Defaults to False.
  • mode: "standard" uses deep data analysis, while "fast" trades some depth of analysis for speed and is adequate for most usecases. Learn more about the modes in this guide. Defaults to "fast".
  • is_screenshot_enabled: Whether to take a screenshot before extracting data. Returned in ‘metadata’ as a Base64 string. Defaults to False.
ExtractWebDataTool is implemented with AgentQL’s REST API, you can view more details about the parameters in the API Reference docs.

ExtractWebDataBrowserTool

To instantiate ExtractWebDataBrowserTool, you need to connect the tool with a browser instance. You can set the following params:
  • timeout: The number of seconds to wait for a request before timing out. Increase if data extraction times out. Defaults to 900.
  • wait_for_network_idle: Whether to wait until the network reaches a full idle state before executing. Defaults to True.
  • include_hidden: Whether to take into account visually hidden elements on the page. Defaults to True.
  • mode: "standard" uses deep data analysis, while "fast" trades some depth of analysis for speed and is adequate for most usecases. Learn more about the modes in this guide. Defaults to "fast".
ExtractWebDataBrowserTool is implemented with AgentQL’s SDK. You can find more details about the parameters and the functions in AgentQL’s API References.

GetWebElementBrowserTool

To instantiate GetWebElementBrowserTool, you need to connect the tool with a browser instance. You can set the following params:
  • timeout: The number of seconds to wait for a request before timing out. Increase if data extraction times out. Defaults to 900.
  • wait_for_network_idle: Whether to wait until the network reaches a full idle state before executing. Defaults to True.
  • include_hidden: Whether to take into account visually hidden elements on the page. Defaults to False.
  • mode: "standard" uses deep data analysis, while "fast" trades some depth of analysis for speed and is adequate for most usecases. Learn more about the modes in this guide. Defaults to "fast".
GetWebElementBrowserTool is implemented with AgentQL’s SDK. You can find more details about the parameters and the functions in AgentQL’s API References.`

Invocation

ExtractWebDataTool

This tool uses AgentQL’s REST API under the hood, sending the publicly available web page’s URL to AgentQL’s endpoint. This will not work with private pages or logged in sessions. Use ExtractWebDataBrowserTool for those usecases.
  • url: The URL of the web page you want to extract data from.
  • query: The AgentQL query to execute. Use AgentQL query if you want to extract precisely structured data. Learn more about how to write an AgentQL query in the docs or test one out in the AgentQL Playground.
  • prompt: A Natural Language description of the data to extract from the page. AgentQL will infer the data’s structure from your prompt. Use prompt if you want to extract data defined by free-form language without defining a particular structure.
Note: You must define either a query or a prompt to use AgentQL.

ExtractWebDataBrowserTool

  • query: The AgentQL query to execute. Use AgentQL query if you want to extract precisely structured data. Learn more about how to write an AgentQL query in the docs or test one out in the AgentQL Playground.
  • prompt: A Natural Language description of the data to extract from the page. AgentQL will infer the data’s structure from your prompt. Use prompt if you want to extract data defined by free-form language without defining a particular structure.
Note: You must define either a query or a prompt to use AgentQL. To extract data, first you must navigate to a web page using LangChain’s Playwright tool.
The langchain-community package is no longer maintained. Examples that import from langchain_community may be outdated or broken. Use with caution.

GetWebElementBrowserTool

  • prompt: A Natural Language description of the web element to find on the page.
The langchain-community package is no longer maintained. Examples that import from langchain_community may be outdated or broken. Use with caution.

Chaining

You can use AgentQL tools in a chain by first binding one to a tool-calling model and then calling it:

Instantiate LLM

Execute tool chain

Use within an agent

You can use AgentQL tools with an AI Agent using the AgentQLBrowserToolkit . This toolkit includes ExtractDataBrowserTool and GetWebElementBrowserTool. Here’s an example of agentic browser actions that combine AgentQL’s toolkit with the Playwright tools.

Instantiate toolkit

The langchain-community package is no longer maintained. Examples that import from langchain_community may be outdated or broken. Use with caution.

Use with a ReAct Agent


API reference

For more information on how to use this integration, please refer to the git repo or the langchain integration documentation