Automating Form Submission with Puppeteer: Step-by-Step Guide

In this blog post, we’ll explore how to use Puppeteer to automate the process of filling out a form and taking a screenshot of the results.

Puppeteer provides a powerful API for controlling headless (or full) Chrome or Chromium browsers, which is extremely useful for tasks such as web scraping, automated testing, and browser automation.

Prerequisites

Before starting, ensure you have Node.js installed on your system. You can download it from the official website.

Additionally, you need to install Puppeteer, which can be done using npm:

npm install puppeteer

Step-by-Step Breakdown

1. Importing Puppeteer

We start by importing the Puppeteer library, which allows us to control the browser.

const puppeteer = require("puppeteer");

2. Asynchronous Function to Enter Form Data

This function takes two parameters:

• url: The URL of the webpage where the form is located.
• searchQuery: The query string to be entered into the form.

async function enterFormData(url, searchQuery) {
    try {
        const browser = await puppeteer.launch({ headless: false });
        const page = await browser.newPage();
        
        await page.goto(url);
        
        await page.focus('input[name="p"]');
        await page.keyboard.type(searchQuery);

        await page.keyboard.press('Enter');

        await page.waitForNavigation({ waitUntil: 'networkidle2' });

        await page.screenshot({ path: 'query-results.png' });

        await browser.close();

        console.log("Form Data Submitted Successfully");
    } catch (error) {
        console.log(error);
    }
}

Below are the details of the steps executed.

Launch Browser: We launch a new browser instance.

The headless: false option runs the browser in non-headless mode so that we can see the browser actions. You can set it to true for headless mode.Open New Page: We open a new tab in the browser.

Navigate to URL: We navigate to the specified URL using page.goto(url).

Focus on Input Field: We focus on the input field where the form data should be entered. Here, the input field is identified by input[name="p"].Type Search Query: We type the searchQuery into the focused input field using page.keyboard.type(searchQuery).

Submit Form: We simulate pressing the ‘Enter’ key to submit the form.Wait for Navigation: We wait for the page to load after the form submission. The waitForNavigation method ensures the script waits until the network is idle, indicating that the page has fully loaded.

Take Screenshot: We take a screenshot of the results page and save it as query-results.png.

Close Browser: Finally, we close the browser and log a success message. If an error occurs, it is caught and logged.

3. Running the Function

Here, we define the URL and the search query, then call the enterFormData function with these parameters.

const url = "https://yahoo.com";
const query = "sunrise";

enterFormData(url, query);

The complete Script is given below

const puppeteer = require("puppeteer");

async function enterFormData(url, searchQuery) {
    try {
        const browser = await puppeteer.launch({ headless: false });
        const page = await browser.newPage();
        
        await page.goto(url);
        
        await page.focus('input[name="p"]');
        await page.keyboard.type(searchQuery);

        await page.keyboard.press('Enter');

        await page.waitForNavigation({ waitUntil: 'networkidle2' });

        await page.screenshot({ path: 'query-results.png' });

        await browser.close();

        console.log("Form Data Submitted Successfully");
    } catch (error) {
        console.log(error);
    }
}

const url = "https://yahoo.com";
const query = "sunrise";

enterFormData(url, query);

This script demonstrates how to use Puppeteer to automate the process of filling out a form, submitting it, and taking a screenshot of the results.

Puppeteer’s powerful API makes it a versatile tool for web automation tasks, including web scraping, automated testing, and much more.

Feel free to customize the script to suit your specific needs, such as handling different input fields or performing more complex interactions with the page.

Puppeteer’s documentation provides extensive information on its capabilities and usage.

Happy automating!