Proof of Concept for fetching a file from IPFS using local node or Gateway

As decentralized web technology continues to develop, the InterPlanetary File System (IPFS) offers an alternative approach to distributed file storage and sharing. This article presents a proof of concept for building a flexible local client capable of fetching files from IPFS, regardless of whether the user has a local IPFS node or not.

IPFS local nodes vs IPFS gateways

Utilizing a local IPFS client enables users to directly interact with the IPFS network, eliminating the need for intermediaries. This direct interaction allows local clients to fetch files more quickly, as they bypass the reliance on external gateways. Moreover, local clients offer an additional layer of privacy by removing the necessity to expose requests to third-party gateways, which is especially important for users who prioritize data confidentiality. A local IPFS client also empowers users to manage their content more effectively by enabling them to pin files and folders, ensuring their availability to others within the network.

Despite the advantages offered by a local IPFS client, not all users may have one installed or properly configured. In these instances, incorporating a gateway fallback becomes essential for maintaining uninterrupted access to IPFS content. The implementation of a fallback mechanism lowers the barrier to entry for newcomers to the decentralized web, thus promoting wider adoption of decentralized web technologies and enhancing the overall user experience.

Imports and Setup

That’s enough talk for now, let’s get into some code! If you haven’t already installed IPFS and the Kubo CLI, see this blog post for instructions on how to do so.

Python Imports

All we need is the requests library to make HTTP requests to the IPFS gateway and the subprocess library to run the ipfs commands.

import subprocess
import requests

Define Functions

Check Local IPFS Node

The first function, check_local_ipfs_node(), checks whether the user has a local IPFS node installed and running. If this function returns True, the user has a local IPFS node and the ipfs daemon is running. If this function returns False, the user does not have a local IPFS node or the ipfs daemon is not running.

def check_ipfs_installed() -> bool:
    try:
        ipfs_present = subprocess.check_output(["ipfs", "version"]).decode("utf-8")
        print(f"Local IPFS Node Detected! {ipfs_present}")

    except subprocess.CalledProcessError:
        return False  # Return False to indicate that IPFS is not installed

    return True  # Return True to indicate that IPFS is installed and the daemon is running

def check_ipfs_daemon_running() -> bool:
    try:
        subprocess.check_output(["ipfs", "swarm", "peers"]).decode("utf-8") # Check if daemon is running by checking if we can connect to peers
    except subprocess.CalledProcessError:
        return False  # Return False to indicate that the daemon is not running

    return True  # Return True to indicate that the daemon is running

Get Data From Local IPFS Node

The second function, get_data_from_local_ipfs_node(), attempts to fetche the file from your local IPFS node. This function takes the CID as an argument and tries to fetch the file using the local IPFS node. If the file is successfully fetched, the function returns the file contents.

def get_data_from_local_ipfs_node(cid: str) -> bytes:
    """
    Fetches data from local IPFS node. If CID is not found on local node, it will attempt to fetch from IPFS network via local node.
    """
    try: # Check if CID is on local IPFS node
        subprocess.check_output(["ipfs", "pin", "ls", cid])
        print(f"CID: {cid} found in local IPFS node")
        data = subprocess.check_output(["ipfs", "cat", cid])
        return data
    except subprocess.CalledProcessError:
        print(f"CID: {cid} not found in local IPFS node. Attempting to fetch from IPFS network via local node.")
        try:
            if not check_ipfs_daemon_running(): # Check if IPFS daemon is running since we need it to fetch from IPFS network.
                print("IPFS daemon not running. Please start IPFS daemon by running 'ipfs daemon' or installing the IPFS desktop app and starting it.")
                return None
            data = subprocess.check_output(["ipfs", "cat", cid])
            print(f"CID: {cid} found on IPFS network via local IPFS node.")
            print("Pinning CID to local IPFS node")
            print(subprocess.check_output(["ipfs", "pin", "add", cid]))
        except subprocess.CalledProcessError:
            return None
    return data

Get Data From IPFS Network using Gateway

The third function, get_data_from_ipfs_network_using_gateway(), fetches the file from the IPFS network using a public gateway. This function only runs if a local IPFS instance cannot be found. This function takes the CID as an argument and returns the file data. If the data cannot be found on the IPFS network, the the script terminates.

def get_data_from_ipfs_network_using_gateway(cid: str) -> bytes:
    try:
        data = requests.get(f"https://ipfs.io/ipfs/{cid}").content
        print(f"CID: {cid} found on IPFS network via Gateway")
        return data
    except requests.exceptions.RequestException as e:
        return None

Main Function

This is the function that orchastrates the retrieval process. It first checks whether the user has a local IPFS node installed and running. If the user has a local IPFS node, the script will try to fetch the data from the IPFS network using the local node. If the user does not have a local IPFS node, the script will attempt to fetch the data from a public gateway.

cid = 'QmTgttqUf7PvZgdSoe71j3njeEKk1hC3h22n2sQmety3To' # CID of of a Landsat 9 .tif file
def get_data(cid: str) -> bytes:
    if check_ipfs_installed():
        data = get_data_from_local_ipfs_node(cid)

    else:
        print("Local IPFS Node not detected. Please install IPFS and start the daemon.")
        print("Now Attempting to fetch CID from IPFS network via Gateway")
        data = get_data_from_ipfs_network_using_gateway(cid)

    return data


data = get_data(cid)

Local IPFS Node Detected! ipfs version 0.18.1

CID: QmTgttqUf7PvZgdSoe71j3njeEKk1hC3h22n2sQmety3To found in local IPFS node

 82.62 MiB / 82.62 MiB  100.00% 0sK