How to Find Broken Links using Selenium WebDriver

Here I will be going to explain you that how to find out the broken links in your web page using Selenium WebDriver. I mean to find out the links which throws : 404 error (Page not found error) when user clicks on it. As you know that it would be very difficult for a manual tester to validate each and every hyper links in a web page if a page has 100+ hyper links and also it is time consuming too if you have to test web pages those are having 100+ links after every build and there are chances that the person who is testing all these links may miss few by mistake. Its better to automate it to save time as well as to increase the testing accuracy.

Look at the example below :

Scenario :

1) Launch URL – http://automate-apps.com/contents/
2) Fetch all the hyper links that has “HREF” attribute and save it into a list.
3) Validate all each hyper links one by one by making HTTP request to web server to make sure that the hyper links are not throwing any server error like 404(Page not found error).
4) Print the result of each URL.

Code to fetch all the hyper links and store it in a list

 public List<WebElement> fetchAllLinks(WebDriver driver) {

		  List<WebElement> list = new ArrayList<WebElement>();

		  //Find all elements with anchor tag and store it into a list
		  list = driver.findElements(By.tagName("a"));

		  List<WebElement> finalList = new ArrayList<WebElement>(); ;

		  //Now scan all hyper links in order to filter out the ones those are without href attribute
		  for (int i = 0 ; i<list.size(); i++) {

			  if(list.get(i).getAttribute("href") != null) {

				  finalList.add(list.get(i));
			  }
		  }	

		  return finalList;

	  }

Code to validate URL’s using HTTP connection

public String isLinkBroken(URL url) throws Exception {
	  
			String response = " ";
	 
			//Create an instance of HTTP URL connection
			HttpURLConnection connection = (HttpURLConnection) url.openConnection();
	 
			//connect to the URL
			connection.connect();
				
			// get the response
			response = connection.getResponseMessage();	        
	 
			// Disconnect the connection
			connection.disconnect();
	 
			return response;
	 
			} 

Complete Program

import java.util.List;
import java.net.HttpURLConnection;
import java.net.URL;
import java.util.ArrayList;
import org.junit.After;
import org.junit.Before;
import org.junit.Test;
import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.firefox.FirefoxDriver;

public class FindBrokenLinks {

    WebDriver driver;

    @Before
    public void setUp() {
        driver = new FirefoxDriver();

        driver.get("http://automate-apps.com/contents/");
    }

    @Test
    public void testLinks() throws Exception {

        //Fetch all hyper links those contains HREF attributes
        List<WebElement> links = fetchAllLinks(driver);

        System.out.println("Total number of hyper links on webpage : " + links.size());

        // Validate all URLS one by one using For loop
        for(int i = 0 ; i < links.size(); i++) {
            try {
                System.out.println("URL: " + links.get(i).getAttribute("href") + " - Result : " + isLinkBroken(new URL(links.get(i).getAttribute("href"))));

                } catch(Exception e) {

                    System.out.println("Error at " + links.get(i).getAttribute("innerHTML") + " Exception occured : " + e.getMessage());
                }
            }
        }

    @After
    public void tearDown() {
        driver.quit();
    }

      public List<WebElement> fetchAllLinks(WebDriver driver) {

          List<WebElement> list = new ArrayList<WebElement>();

          //Find all elements with anchor tag and store it into a list
          list = driver.findElements(By.tagName("a"));

          List<WebElement> finalList = new ArrayList<WebElement>(); ;

          //Now scan all hyper links in order to filter the ones those are without href attribute and save elements into FinalList
          for (int i = 0 ; i<list.size(); i++) {

              if(list.get(i).getAttribute("href") != null) {

                  finalList.add(list.get(i));
              }
          }

          return finalList;

      }

        public String isLinkBroken(URL url) throws Exception {

            String response = " ";

            //Create an instance of HTTP URL connection
            HttpURLConnection connection = (HttpURLConnection) url.openConnection();

            //connect to the URL
            connection.connect();

            // get the response
            response = connection.getResponseMessage();

            // Disconnect the connection
            connection.disconnect();

            return response;

            }

    }

Program Output would be like this, if all hyper links are working:

Look at the Program Output if some of the links are not working or throwing page not found error :

If you really like the information provided above, please don’t forget to like us on Facebook, you can also leave the comment.