Building a Site Connectivity checker in Python

A site connectivity checker is a tool that helps you monitor the status of your internet connection. It checks if the website you are trying to access is available and accessible. If the website is down, the checker will provide a message indicating that the website is currently unavailable. On the other hand, if the website is up and running, the checker will return a message indicating that the website is accessible.

Nowadays, the internet has become a fundamental aspect of our daily lives. From communication to business operations, we rely on the internet to carry out numerous tasks. As a result, having a reliable and stable internet connection has become crucial. At times, internet connectivity issues can cause frustration and loss of productivity. To overcome this, building a site connectivity checker using Python can come in handy.

In this article, we will discuss how to build a site connectivity checker using Python. The Requests library is a popular library for making HTTP requests in Python and is easy to use. We will use the Requests library in Python to make HTTP requests to a website and check its status.

Once we have installed the Requests library, we can start importing it into our Python script. Then, we can write a simple function to make an HTTP request to a website and check its status. The function will use the GET method to request and return a status code indicating the website's status.

We can then write a loop that will continuously make HTTP requests to the website and check its status. If the status code is 200, we can display a message indicating that the website is accessible. If the status code is 404, we can display a message indicating that the website is currently unavailable.

The site connectivity checker can be expanded in several ways to make it functional and user-friendly. Some of the ideas are:-

GUI Interface: To make the site connectivity checker more accessible, a GUI interface can be added to the script. The GUI interface can display the website's status in a visually appealing way and allow the user to interact with the checker more easily.
Logging: The site connectivity checker can be updated to include a logging feature to keep track of the website's status. The log can keep a record of when the website was checked and what the status was. This can be useful for troubleshooting connectivity issues and tracking the reliability of the website.
Email Notifications: To stay informed of any connectivity issues, the site connectivity checker can be updated to send email notifications. The checker can be programmed to send an email if the website is unavailable, alerting the user that there may be an issue with their internet connection.
Customizable Settings: To make the site connectivity checker more flexible, customizable settings can be added. For example, the user can choose the frequency of the check, the website to check, and the email address to send notifications to.
Graphical Representation of Connectivity: The site connectivity checker can also be updated to provide a graphical representation of the website's connectivity. The graph can display the website's status over time, making it easier to identify patterns and trends in its reliability.

Prerequisites required

To build a site connectivity checker using Python, you should have a basic understanding of the following concepts:-

Python programming: A fundamental understanding of Python programming language is necessary to build the site connectivity checker. You should know the syntax, variables, functions, and loops.
HTTP Requests: The site connectivity checker will make HTTP requests to websites, so it's essential to understand HTTP requests and responses.
Python Requests Library: You should understand the Requests library in Python and how to use it to make HTTP requests.
Error Handling: Error handling is crucial when building a site connectivity checker, as you must catch any exceptions or errors that may occur during the request.
Basic knowledge of sockets: Sockets are used in Python to create connections between the client and the server. A basic understanding of sockets can help you better understand how the site connectivity checker works.
GUI programming: If you want to build a graphical user interface for the site connectivity checker, you should understand GUI programming in Python.

Building a Site Connectivity checker in Python

Setting up site connectivity checker

To set up the site connectivity checker, the first step is that we need to create a virtual environment for site connectivity.

By creating a virtual environment for your site connectivity checker project in Python, you can isolate your project dependencies and avoid conflicts with other projects on your system. This helps you maintain a clean and organized environment for your project and makes it easier to share with others.

Another advantage of virtual environments is having different versions of the same package installed in different virtual environments. For example, you can have one virtual environment with version 2.0 of the Requests library and another with version 3.0 of the same library. This helps you test your project with different versions of the required packages and ensure compatibility with different systems.

Creating a virtual environment creates a separate directory for your project, and all the packages you install will be stored in that directory. This means that when you deactivate the virtual environment, the packages will not be available in your system, and you won't accidentally use them in another project.

Steps to create a virtual environment for your site connectivity checker project in Python:

Install virtualenv: To create a virtual environment in Python, you first need to install the virtualenv package. You can install it using the pip command in your terminal:

Create a virtual environment: You need to create a virtual environment for your site connectivity checker project. To create a virtual environment, use the following command in your terminal:

php
virtualenv <environment_name>

Replace <environment_name> with the name you want to give to your virtual environment. For example, if you want to name your virtual environment site_checker, you would use the following Command:

Activate the virtual environment: Once you've created the virtual environment, you need to activate it before you can use it. To activate the virtual environment, use the following command in your terminal:

javascript
source <environment_name>/bin/activate

Replace <environment_name> with the name you gave to your virtual environment.

Install required packages: With the virtual environment activated, you can install the required packages for your site connectivity checker project. To install the Requests library, use the following command in your terminal:

Deactivate the virtual environment: When you're finished working on your project, you can deactivate the virtual environment by using the following command in your terminal:

Organizing the Site Connectivity Checker

Organizing your code is essential for making it easier to maintain and update in the future. A well-organized project makes it easier for others to understand and contribute to the project and reduces the risk of bugs and errors. Here are some additional tips for organizing your site connectivity checker project in Python:

Use descriptive variable names: Make sure your variable names are descriptive and explain what they are storing. For example, instead of using -x" as a variable name, use -site_url" to describe what it is storing.
Use modules and packages: If your project grows, consider breaking it up into modules or packages. This makes it easier to manage and maintain the code, and it also makes it easier to reuse code between projects.
Document your code: Make sure to document your code by adding docstrings to your functions and classes. This helps others understand how to use your code, and it also helps you remember how your code works in the future.
Use a consistent coding style: Make sure to use a consistent coding style throughout your project. This makes it easier for others to read and understand your code and helps you keep your code clean and organized.
Use logging: Use logging to keep track of what your code is doing. This makes it easier to debug your code and track down errors.
Test your code: Test your code thoroughly to ensure it is working as expected. This helps reduce the risk of bugs and errors in your code.

Checking website connectivity

To check the connectivity of a website, you can use the requests library in Python. The requests library provides a simple way to make HTTP requests in Python, which can be used to check the connectivity of a website.

By checking the connectivity of a website, you can determine whether a website is online and available to users. This is important because it helps you to ensure that your website is always accessible to your users, and it also helps you to monitor the performance of your website.

There are several ways to check the connectivity of a website, including using the requests library in Python, as discussed above. The requests library provides a simple and easy-to-use interface for making HTTP requests, and it can be used to check the connectivity of a website in just a few lines of code.

In addition to checking the connectivity of a website, you can also use the requests library to retrieve information about the website, such as the response status code, headers, and content. For example, you can use the response.status_code attribute to retrieve the status code returned by the server, and you can use the response.text attribute to retrieve the content of the website.

To make the site connectivity checker even more powerful, you can also check the connectivity of multiple websites at once. This can be useful if you need to monitor the connectivity of multiple websites or if you need to check the connectivity of multiple websites in a loop.

Implement a connectivity checker function.

Example for implementing a connectivity checker function using Python and the requests library:

Step 1: Import the necessary modules

Python
import requests
import socket

Step 2: Define the function that checks connectivity to a given URL

Python
def check_connectivity(url):
    try:
        # Request the URL and check the status code
        response = requests.get(url)
        if response.status_code == 200:
            print(f"Successfully connected to {url}")
        else:
            print(f"Failed to connect to {url}. Status code: {response.status_code}")
    except requests.exceptions.RequestException as e:
        print(f"Failed to connect to {url}. Exception: {str(e)}")
    except socket.gaierror as e:
        print(f"Failed to resolve hostname for {url}. Exception: {str(e)}")

This function uses the requests module to make a GET request to the URL and checks the status code of the response. Suppose the status code is 200 (i.e., OK), and the function prints a success message. The function prints an error message if there is an exception while making the request.

Step 3: Test the function

Python
check_connectivity("https://www.google.com")

This will call the check_connectivity() function with the URL "https://www.google.com". If the function is working correctly, it should print a success message.

You can use this function to check the connectivity of multiple URLs by calling them with different URLs as arguments.

Running first connectivity checks

Once you have implemented the connectivity checker function, you can start running your first connectivity checks.

You can also use the connectivity checker function in your projects to ensure that your website is accessible and performing well.

check_connectivity() function that we defined earlier:

Step 1: Open a new Python file in your preferred editor.

Step 2: Copy and paste the same code that is used for implementing the connectivity checker with the imports and the below code into the file:-

Step 3: Save the file with a name like connectivity_checker.py.

Step 4: Open a terminal or command prompt and navigate to the directory where you saved the file.

Step 5: Run the following command to execute the file:

This will call the check_connectivity() function with the URL "https://www.google.com" and print a success message if the connection is successful.

You can modify the URL passed to check_connectivity() to test the connectivity of different websites. You can also modify the function to add more advanced functionality, such as checking for specific content on the website or logging the results to a file.

Creating Websites Connectivity Checkers CLI

Creating a Command Line Interface (CLI) for your website connectivity checker can provide a more user-friendly and efficient way to run connectivity checks. A CLI allows you to run the connectivity checker from the terminal without needing a user interface.

A CLI allows users to interact with your tool directly from the command line without writing code or using a graphical user interface. This can be especially useful for developers who need to quickly check the connectivity of multiple websites or for system administrators who need to automate the monitoring website performance.

To create a CLI for your website connectivity checker, you can use a library such as argparse in Python. Argparse is a standard library that provides a convenient way to write user-friendly command-line interfaces.

This code creates a Command Line Interface (CLI) for a website connectivity checker using the argparse library in Python.

Importing libraries:

Python
import argparse
import sys

In this section, we import the argparse library to create the CLI and the sys library to exit the program with a status code.

Defining the check_connectivity function:

Python
def check_connectivity(website):
    # Implementation of the connectivity checker function
    # ...

In this section, we define the check_connectivity function that takes the website URL as an argument and returns the result of the connectivity check. The implementation of the connectivity checker function is omitted in this example.

Parsing Website URL at Command Line

The code is parsing the website URL at the command line. This is done by using the argparse module, which is a standard library in Python. The argparse module provides an easy way to parse command-line arguments and options.

Step 1: Import the necessary modules

Python
import argparse
import requests
import socket

Step 2: Define the function that checks connectivity to a given URL

Python
def check_connectivity(url):
    try:
        # Request the URL and check the status code
        response = requests.get(url)
        if response.status_code == 200:
            print(f"Successfully connected to {url}")
        else:
            print(f"Failed to connect to {url}. Status code: {response.status_code}")
    except requests.exceptions.RequestException as e:
        print(f"Failed to connect to {url}. Exception: {str(e)}")
    except socket.gaierror as e:
        print(f"Failed to resolve hostname for {url}. Exception: {str(e)}")

Step 3: Parse the URL from the command line using argparse

parser = argparse.ArgumentParser(description='Check the connectivity of a website')
parser.add_argument('url', type=str, help='The URL to check')
args = parser.parse_args()

check_connectivity(args.url)

This code creates an ArgumentParser object and adds a positional argument url to it. It then parses the command line arguments using parse_args(). The url argument is stored in the args object, which is then passed to the check_connectivity() function.

Step 4: Test the code

Save the code above to a file called connectivity_checker.py, then run the following command in the terminal:

javascript
python connectivity_checker.py https://www.google.com

This will call the check_connectivity() function with the URL "https://www.google.com". If the function is working correctly, it should print a success message.

website = args.website
result = check_connectivity(website)

website = args.website retrieves the value of the "website" argument from the args object and stores it in the website variable. This is the URL of the website to be checked.

result = check_connectivity(website) calls the check_connectivity function and passes the website URL as an argument. The function returns a Boolean value indicating whether the website is accessible or not, which is stored in the result variable.

    if result:
        print(f"{website} is accessible")
    else:
        print(f"{website} is not accessible")
        sys.exit(1)

Finally, we use an if-else statement to print the result of the connectivity check and exit with a status code of 1 in case of failure. The sys.exit(1) statement indicates an error exit status, which can be useful when running the CLI in scripts or other automated processes.

Running the main function:

markdown

if __name__ == "__main__":
   main()

In this section, we run the main function when the script is executed directly. This allows us to run the CLI by simply executing the script, without the need to call the main function explicitly.

Load Website URL from files

To load website URLs from a file, we can modify the code to read the file's contents and store the URLs in a list. Then, we can iterate through the list and perform a connectivity check on each website.

Here's how the code can be modified to load website URLs from a file:

Python
def main():
    parser = argparse.ArgumentParser(description="Website Connectivity Checker")
    parser.add_argument("file", type=str, help="File containing website URLs to check")
    args = parser.parse_args()

These lines work the same as they mentioned in the parsing process of the Websites URL command lines.

    file = args.file
    with open(file, "r") as f:
        websites = f.readlines()

This code adds the argument "file" to the argparse.ArgumentParser object to specify the file containing the website URLs to check. The open function is used to open the file, and the readlines method reads the file's contents and stores them in a list.

    for website in websites:
        result = check_connectivity(website.strip())
        if result:
            print(f"{website.strip()} is accessible")
        else:
            print(f"{website.strip()} is not accessible")
            sys.exit(1)

Next, the code iterates through the list of websites and performs a connectivity check on each website using the check_connectivity function. If the website is accessible, a message is printed to the console indicating that the website is accessible.

If the website is not accessible, a message is printed to the console indicating that the website is not accessible, and the program exits with a non-zero status code. The strip method removes any leading or trailing whitespaces from the website URL.

Putting Everything Together in the App's Main Script

The main script is the final puzzle in building a website connectivity checker in Python. We can create a complete and functional application by combining all of the code snippets from the previous steps.

The script will also utilize the argparse library to create a command-line interface for the user. This allows the user to specify the website URL or the file containing the website URLs as a command-line argument.

The script will process the user input, either perform the connectivity check on a single URL or loop through a list of URLs in a file and print the results to be accessible. The program will exit with a non-zero status code, indicating an error.

Create applications entry point script.

To create the application's entry point script, we will first import all necessary modules, such as argparse and requests. Then, we will define the main function that will act as the starting point for our application. Within this function, we will use the argparse module to define and parse the arguments passed to the script, including the website URL to be checked.

parser = argparse.ArgumentParser(description="Website Connectivity Checker") parser.add_argument("-u", "--urls", type=str, help="File containing list of websites to check", required=True)
args = parser.parse_args(user_args)

The first part of the code creates a parser object and adds a single argument --urls which is the path to a file containing a list of websites to check. The required attribute is set to True, so the script will only run if the --urls argument is provided.

with open(args.urls, 'r') as f:
      urls = f.read().splitlines()

The second part of the code opens the file specified by the --urls argument and reads its contents into a list of urls.

results = [] 
for website in urls: result = check_connectivity(website) 
results.append(result) 
if result: 
print(f"{website} is accessible") 
else: 
print(f"{website} is not accessible") sys.exit(1)

The third part of the code iterates through the list of websites, checks each website's connectivity using the check_connectivity function, and appends the result to the results list. If the website is accessible, the message {website} is accessible is printed to the console. Otherwise, {website} is not accessible is printed, and the script exits with a non-zero status code of 1.

Check the connectivity of multiple websites

def check_website_status(url):
  try:
         response = requests.get(url)

The code inside the try block makes a GET request to the website using the requests.get method and stores the response in the response variable.

   # Check if the response has a status code of 200 (OK) #
    except requests.exceptions.RequestException as e:
   # If there is any exception, print the error message and website URL #
    print(f"{url} is down. Reason: {e}")

The requests.exceptions.RequestException exception is caught here, which covers any exceptions that might occur while making a request using the requests library.

The check_website_status function can be used to check the connectivity of any website by passing its URL as an argument.= requests.get(url)

Running connectivity checks from the command line

Running the Python script from the command line is a way to execute the script and see its output without running it within an IDE or text editor. This allows you to automate the connectivity check and schedule it to run regularly without any manual intervention.

Here are the quick steps to run the script from the command line:

Open the terminal or command line: On Windows, you can open the command line by clicking on the Start button, typing "cmd" in the search box, and clicking on the Command Prompt app. You can open the terminal on macOS or Linux by searching for it in Spotlight (macOS) or using the shortcut Ctrl + Alt + T (Linux).
Navigate to the directory where the script is located: You can use the cd command to change the current working directory to the directory where the script is located. For example, if the script is located in the "scripts" directory on your Desktop, you would type cd ~/Desktop/scripts and press Enter.
Execute the script: Type python filename.py (replace filename.py with the actual name of your script) and press Enter. This will run the script and display the output in the terminal.
By running the script from the command line, you can easily automate the connectivity checks and schedule them to run at regular intervals using tools like cron on Linux or Task Scheduler on Windows. This allows you to continuously monitor multiple websites' connectivity and proactively address any connectivity issues that might arise.

Checking website connectivity asynchronously

Checking the connectivity of multiple websites synchronously (one after the other) can take a long time if the number of websites is large or if any of the websites could be faster to respond. To overcome this, you can check the connectivity of websites asynchronously.

In Python, you can use the asyncio library to perform asynchronous tasks, including checking the connectivity of websites. The asyncio library provides a way to run multiple coroutines (small, single-purpose functions) concurrently and manage the flow of execution between them.

This makes it possible to check the connectivity of multiple websites in parallel and improve the performance of the script.

Implementing an asynchronous checker function

1. Importing the asyncio and aiohttp Libraries:

import asyncio
import aiohttp

This step imports the asyncio and aiohttp libraries, which perform asynchronous tasks and make HTTP requests.

2. Writing a Coroutine to Check the Connectivity of a Single Website:

async def check_website_connectivity(url):
    async with aiohttp.ClientSession() as session:
        async with session.get(url) as resp:
            return resp.status

The coroutine uses an async with a statement to create an asynchronous context for a ClientSession object, which is used to make the HTTP request. The ClientSession object is created as an asynchronous context manager, automatically closing when the async with the block is exited.

The session.get method is used to make the HTTP GET request and return a ClientResponse object. The ClientResponse object is then used to obtain the response's status code using the resp.status property.

3. Writing a Main Coroutine that Creates Tasks for Each Website and Runs Them Concurrently:

async def main():
    websites = [
        "https://www.google.com",
        "https://www.facebook.com",
        "https://www.amazon.com"
    ]

    tasks = [check_website_connectivity(website) for website in websites]
    results = await asyncio.gather(*tasks)

    for website, result in zip(websites, results):
        print(f"{website} returned status code {result}")

It defines a list of websites to be checked, creates a task for each website using the check_website_connectivity coroutine, and runs the tasks concurrently using the asyncio.gather function.

The asyncio.gather function collects the results of all tasks and returns them as a list.

4. Running the Main Coroutine:

if __name__ == "__main__":
    asyncio.run(main())

This code calls the asyncio.run function to run the main coroutine and check the connectivity of all websites. The if __name__ == "__main__": block ensures that the code is executed only if the script is run directly, not imported as a module.

To add an asynchronous option to the CLI of the website connectivity checker application, you could use a library such as click to define the CLI interface and options:-

Install the click library:

Import the click library in your Python script:

Python
import click

Add a new option to the CLI interface using the click.option decorator:

less
@click.command()
@click.option("--async/--sync", default=False, help="Run the connectivity checks asynchronously or synchronously")
def main(async):

This option, named --async, allows the user to specify whether the connectivity checks should be performed asynchronously or synchronously. The default value is False, so the connectivity checks will be performed synchronously unless the --async option is specified.

Modify the main code to run the connectivity checks asynchronously or synchronously based on the async option:

css
if async:
    asyncio.run(check_websites_async())
else:
    check_websites_sync()

This code checks the value of the async option and calls either the asyncio.run function to run the check_websites_async coroutine or the check_websites_sync function to run the synchronous version of the connectivity checks.

Check the Connectivity of Multiple Websites Asynchronously

To check the connectivity of multiple websites asynchronously, you can use the asyncio library in Python.

csharp
async def check_websites_async():
    tasks = [check_website_connectivity(url) for url in websites_to_check]
    await asyncio.gather(*tasks)

This coroutine creates a list of tasks, one for each URL that needs to be checked, using a list comprehension. Then, it uses the asyncio.gather function to run all of the tasks concurrently. The await keyword is used to wait for all tasks to complete before moving on to the next step.

Call the check_websites_async coroutine using asyncio.run:

less
asyncio.run(check_websites_async())

This line of code starts the asynchronous event loop and runs the check_websites_async coroutine until it completes.

OUTPUT

$ python connectivity_checker.py https://www.google.com https://www.facebook.com
https://www.google.com is up and running
https://www.facebook.com is up and running

$ python connectivity_checker.py https://www.google.com https://www.facebook.com --async
https://www.google.com is up and running
https://www.facebook.com is up and running

Conclusion

We built a functional site connectivity checker application in Python. Now We know the basics of handling HTTP requests to a given website. We also learned how to create a minimal yet functional command-line interface (CLI) for your application and how to organize a real-world Python project. Additionally, you've tried out Python's asynchronous features.

In this tutorial, you learned how to:-