Mint your NFTs with Stable Diffusion

In the current era of rapid technological evolution, the convergence of AI and web3 presents an intriguing narrative. Rather than being a competition between AI and crypto, it is a story of these two fields synergizing with each other. AI is poised to accelerate the adoption of decentralized networks, expanding the design possibilities of web3 by enabling smart contracts to securely interact with machine learning models.

One remarkable advancement in this field is generative modeling, an aspect of AI that creates new data based on learned patterns from existing information. This process can be likened to an architect designing a unique building inspired by different architectural styles.

One notable innovation in generative modeling is the diffusion model, which can be envisioned as a puzzle master. It takes a jumble of random and meaningless data, referred to as "noise," and skillfully rearranges it into coherent and valuable images. This puzzle master can work independently, generating images without any specific direction (unconditional generation), or it can utilize guiding clues, such as a text prompt, to shape the final output (conditional generation).

In this hands-on tutorial, we will explore the practical application of the convergence between AI and blockchain. Specifically, we will demonstrate how Gelato’s Web3 Functions can be combined with StabilityAI's Stable Diffusion tool to create a captivating NFT creation experience. We'll use Stable Diffusion to generate AI-created images which will be used as metadata for our NFTs.

Web3 Functions are a key part of this process. They connect the 'on-chain' world, which is our smart contract for creating NFTs, with the 'off-chain' world, which is where we can access APIs like the Stable Diffusion tool. Gelato's Web3 Functions bridge these two worlds. They automatically assign the AI-created images to the NFTs. This tutorial will guide you through using these functions to improve your NFT creation process.

Here's what we'll cover:

1. GelatoNft smart contract: You'll learn how to mint new NFTs with this smart contract, maintaining an element of suspense until their final reveal.

2. Stable Diffusion: This AI tool from StabilityAI generates unique images that become the metadata for your NFTs. You'll explore how to integrate AI-driven imagery into your NFTs.

3. Gelato’s Web3 Functions: Serving as a conduit between your NFT smart contract and the Stable Diffusion API off-chain, these functions automate the assignment of AI-created images to your NFTs. By the end of this tutorial, you'll grasp how to use Gelato's Web3 Functions to streamline your NFT creation process.

Get ready for an insightful voyage into the future of digital innovation!

Ensure you have the following before you start:

• Signed up for the Gelato Web3 Functions beta
• Installed Node.js v16+, NPM, and Git
• Set up a Web3 wallet in your browser, such as MetaMask
• Account created on Alchemy, NFT Storage & Stable Diffusion

Once this is set up, we can dive in!

Start by cloning the Gelato Web3 Functions lensgpt Repository: git clone https://github.com/gelatodigital/W3F-NFT-AI

Install the dependencies:

yarn install

Code Structure

Configure Secrets

• Alchemy ID
• Private Key
• A Stable Diffusion Key
• A NFT.Storage Key

Obtain Alchemy ID & Private Key: Locate the .env.template file in your root folder, copy it and rename the copy to .env These keys are only required if deploying your smart contract to testnet/mainnet.

Here are the steps for acquiring these: Acquiring an Alchemy ID

Sign Up for a free Alchemy Account

Creating an account with Alchemy is simple, sign up for free here.

Create an Alchemy App

You need an API key to authenticate your requests. You can create API keys from the dashboard. Navigate to “Create App” and fill in the details to get your new key and paste it in .env.

Acquiring a Private Key

Go to your Metamask wallet and do the following:

• Click on the three dots next to ‘Account Address’
• Then click on ‘Account Details’
• Export the private key
• Enter your Metamask wallet password
• Copy the private key & paste it in .env

Obtain Stable diffusion and NFT.Storage Keys

Find the .env.template file in the web3-functions/stable-diffusion-nft directory. Again, make a copy of it and rename the copy to .env.

Sign in on the NFTStorage website using either your GitHub or email, navigate to API Keys section and generate a new key.

For the Stable Diffusion API Key, visit the website, click on My Profile and get your API Keys from API Keys sections. Once obtained, these keys need to be entered into the newly created .env file.

Now you're all set to get started!

The "GelatoNft" contract represents an NFT on the Polygon blockchain. Users can mint new tokens by calling the mint function, providing a boolean value to indicate if the token represents a night time or sunset. The contract keeps track of minted tokens and their ownership. The revealNft function is used by a specific Gelato message sender to update the token metadata URI.

###GelatoNft Contract

• mint():

The mint function in the "GelatoNft" contract allows users to create and mint a new NFT. It checks if the caller has already minted a token, increments the token ID counter, mints the token to the caller's address, sets the default URI to notRevealedUri

function mint(bool _isNight) external whenNotPaused {
        require(!hasMinted[msg.sender], "Already minted!");
        tokenIds.increment();
        uint256 newItemId = tokenIds.current();
        _mint(msg.sender, newItemId);
        _setTokenURI(newItemId, notRevealedUri);
        hasMinted[msg.sender] = true;
        tokenIdByUser[msg.sender] = newItemId;
        nightTimeByToken[newItemId] = _isNight;
        emit MintEvent(newItemId); }

• revealNft()

The revealNft function is used by Gelato's dedicated msg.sender to update the metadata URI for a specific token.In this contract, the updated URI is generated using the Stable Diffusion API, which creates the NFT metadata image. Once the URI is set, the MetadataUpdate event is emitted to notify listeners about the updated metadata for the token.This is the function automated by Web3 Functions, when user mints the NFT, Web3 Functions will run and call the revealNft() function and update the tokenURI.

    function revealNft(uint256 tokenId, string memory tokenURI) external 
onlyGelatoMsgSender {
        _setTokenURI(tokenId, tokenURI);
        emit MetadataUpdate(tokenId);
    }

GelatoNft Web3 Function

The Web3 function handles the "reveal" process, where initially hidden NFT tokenURI are made public.

• Importing necessary libraries and contract ABIs

These import statements include various utilities necessary for the function to work correctly. Web3Function and Web3FunctionContext are used to define and interact with the Web3 Functions.Contract from the ethers library are used for interacting with smart contracts . NFTStorage is used to store NFT metadata and axios is used for making HTTP requests.

import { Web3Function, Web3FunctionContext } from "@gelatonetwork/web3-functions-sdk";
import { Contract, utils } from "ethers";
import { NFTStorage, File } from "nft.storage";
import axios, { AxiosError } from "axios";

• Validate user arguments and secret In this section, the function fetches user arguments and secrets, which include contract addresses and API keys. The nftAddress is retrieved from user arguments, while the API keys are fetched from secrets. If these values are missing or invalid, the function will throw an error.

  ////// User Arguments const nftAddress = userArgs.nftAddress as string; console.log("nftAddress", nftAddress);

  if (!nftAddress) throw new Error("Missing userArgs.nftAddress please provide");

  ////// User Secrets
  const nftStorageApiKey = await secrets.get("NFT_STORAGE_API_KEY");
  const stableDiffusionApiKey = await secrets.get("STABLE_DIFFUSION_API_KEY");

  if (!nftStorageApiKey || !stableDiffusionApiKey) {
    console.error("Error: Missing secrets");
    return {
      canExec: false,
      message: "Error: Missing Secrets",    };  }

• Fetching and Processing NFTs

The function fetches the current and the last processed token IDs. If there are any new tokens, the function processes them in batches to stay within RPC call limits. The function checks each NFT in sequence and only selects those that have not been revealed yet.

• Generation of Metadata and Image Creation

For each selected NFT, the function generates the NFT properties and description based on whether it's night or day time as per the NFT contract.The description is used as a prompt to generate an image via the Stable Diffusion API. The Stable Diffusion API returns a URL of the generated image, which is then stored for uploading it to IPFS via NFT.Storage. If there is an error during this process, the error is caught and an appropriate message is returned.

const isNight = await nft.nightTimeByToken(tokenId);
const nftProps = generateNftProperties(isNight);
// Generate NFT image with Stable Diffusion 
let imageUrl: string;
const stableDiffusionResponse = await fetch("https://stablediffusionapi.com/api/v3/text2img", {
  method: "POST",
  headers: {
    "Content-Type": "application/json",  },
  body: JSON.stringify({
    key: stableDiffusionApiKey,
    prompt: nftProps.description,
    // Other parameters for the API request...  
}),});
const stableDiffusionData = await stableDiffusionResponse.json();

Image generated by Stable Diffusion from given prompt

• Upload to IPFS The image generated by the Stable Diffusion API is fetched and uploaded to IPFS along with the rest of the NFT metadata using the NFTStorage library. NFTStorage provides an interface to easily store NFT data on the IPFS network.

const imageBlob = (await axios.get(imageUrl, { responseType: "blob" })).data;
const nftStorage = new NFTStorage({ token: nftStorageApiKey });

const imageFile = new File([imageBlob], `gelato_nft_${tokenId}.png`, { type: "image/png" });
const metadata = await nftStorage.store({
  name: `GelatoNFT #${tokenId}`,
  description: nftProps.description,
  image: imageFile,
  attributes: nftProps.attributes,
  // Other properties for the metadata...
});

• Execution of the Contract's Function The Web3 Functions then creates necessary call data with contract address token ID and token URI for revealNft function to update the tokenURI.

const callDatas: Array<{ to: string; data: string }> = [];
  tokensData.forEach((token) => {
    callDatas.push({
      to: nft.address,
      data: nft.interface.encodeFunctionData("revealNft", [
        token.id,
        token.url, ]), });
  });
  return {
    canExec: true,
    callData: callDatas,
  };
});

• Logging and State Update If all NFTs have been revealed or if there are no new NFTs to process, the function logs this information and updates the "lastProcessedId" in the storage to keep track of the progress. It will also return a canExec property set to false and an appropriate message.

Deploying GelatoNFT SmartContract

Deploying on Mumbai Network Run this command in root of the project directory: npx hardhat run deploy/deploy-contract.ts --network mumbai

Copy the smart contract address above

Deploying Web3 Functions code

In userArgs.json file in the web3-functions/stable-diffusion-nft folder replace GelatoNFT address with yours deployed address.

Then run the command to deploy the Web3 Functions . They are stores on IPFS,to compile your Web3 Function and deploy it to IFPS, use

npx hardhat w3f-deploy W3FNAME

In our case: npx hardhat w3f-deploy stable-diffusion-nft

Once uploaded, Gelato Nodes will pin the file on your behalf on IPFS. If the upload was successful, you should get the IPFS CID of your Web3 Function returned.

✓ Web3Function deployed to ipfs. ✓ CID: <YOUR_IPFS_CID>

To create a task that runs your Web3 Function every minute, visit:

https://beta.app.gelato.network/new-task?cid=<YOUR_IPFS_CID>

Creating a Web3 Function Task via the UI

1. Go to the Link: Click on the link you got when you set up your Web3 Function, the IPFS CID will already be there. This lets you look at the code of your Web3 Function.
2. Enter user arguments: Type in the arguments you entered in userArgs.Json file: nftAddress.
3. Choose Your Network: Pick the network for your task to work on.
4. Add Your Keys: Put in your NFT storage and Stable Diffusion API keys.
5. Give Life to Your Task: Last but not least, you'll be asked to name your task. Hit the "Create Task" button, confirm the name and task creation with your wallet, et voila!

You can keep an eye on how your task is doing from your Web3 Functions Task dashboard. It's got everything you need:

• Executions: Check how many times your task has run on specific dates.
• Task Logs: Look at logs Web3 Function's code has sent out.
• Code, Storage, & Secrets: Review your code, storage details, and your secrets for additional insights.

With this dashboard, you'll have all the information about your task right at your fingertips.

In conclusion, the innovative blend of AI and blockchain is rapidly transforming the landscape of NFT creation and engagement. Through this tutorial, you've learned how to apply these groundbreaking technologies to mint new NFTs with the GelatoNft Solidity smart contract, create unique AI-generated imagery for your NFTs using the Stable Diffusion tool from StabilityAI, and automate the NFT creation process with Gelato’s Web3 Functions.

This process enables an exciting new realm of possibilities for creators and developers in the NFT space, offering an enriching and immersive user experience.

Web3 Functions provide an innovative solution for developers to create serverless, decentralized applications with ease.

Join our community and developer discussion on Discord.

Web3 Functions are available today in private beta. For more information, please check out the Web3 Functions documentation. To learn how to write, test, and deploy your own Web3 Functions, check out our in-depth tutorial.

Apply here to be one of the first to test Gelato's Web3 Functions!