ChatGPT-generated NFTs with Web3 Functions

Over the past few months, we’ve seen a small but incredible glimpse of what OpenAI is capable of, and if you’re a web3 developer, you might be asking yourself what a good example of integrating web3 and OpenAI could be.

Within the web3 ecosystem, there are two primary use cases: DeFi and NFTs. Ultimately, DeFi is all about price movements, trades, options, derivatives, etc., and in this scenario the ability of OpenAI to predict price movements is as yet unproven.

Therefore, we’ll concentrate on the second use case: NFTs. NFTs are the registry of the digital property of an asset. NFT collections use randomness to create the traits and attributes of the NFTs, deterministically creating the image associated with the NFT.

Image generation and randomness are at the core of OpenAI, so how can we combine these two worlds? Bearing in mind that NFTs are minted on-chain and OpenAI images are generated off-chain, the only possible way to link both worlds is by using Gelato’s Web3 Functions or a similar service that provides communication between on-chain and off-chain resources.

The goal of Web3 Functions is to provide developers with a web3-native alternative to running server-side logic on centralized platforms like AWS or Google Cloud. Using Web3 Functions, developers can augment their smart contracts with a robust, scalable, and trust-minimized infrastructure that can power all their off-chain server components- without sacrificing decentralization!

Web3 Functions runs every minute, can check conditions on-chain and call any off-chain API. If certain conditions are met, the Gelato Web3 Function will send a transaction to a predefined contract.

In our use case, we not only want to mint NFTs, we want every NFT image to be generated with OpenAI and for the images to be generated on the fly. Without the power of Gelato Web3 Functions, this task seems like mission impossible. Thankfully, with Web3 Functions it’s actually quite simple:

A user mints an NFT, and the Gelato Web3 Function checks every minute where the new NFTs are minted. If there are new NFTs being minted, the Gelato Web3 Function calls the OpenAI API, downloads the image, stores it on IPFS, and updates the NFT metadata on the contract.

We will use a standard NFT ERC-721 contract where we allow anyone to mint one NFT, and we will create an additional method triggered by the Gelato Web3 Function when the OpenAI is generated and the metadata is available.

function mint(bool _isNight) external whenNotPaused {
       require(!hasMinted[msg.sender], "Already minted!");
       tokenIds.increment();
       uint256 newItemId = tokenIds.current();
       _mint(msg.sender, newItemId);
       _setTokenURI(newItemId, notRevealedUri);
       hasMinted[msg.sender] = true;
       tokenIdByUser[msg.sender] = newItemId;
       nightTimeByToken[newItemId] = _isNight;
       emit MintEvent(newItemId);


   }

   function revealNft(uint256 tokenId, string memory tokenURI) external onlyGelatoMsgSender {
       _setTokenURI(tokenId, tokenURI);
       emit MetadataUpdate(tokenId);
   }

We can see in the mint method that it is expecting a boolean _isNight. This is because we’re allowing users to choose whether they want the NFT’s background to be night or day!

It’s worth noting that the revealNft has the modifier only on GelatoMsgSender. This modifier ensures that only the proxy created by Gelato can execute this method.

The schematic code of a Gelato Web3 Function follows this schema:

Web3Function.onRun(async (context: Web3FunctionContext) => {
 const { userArgs, storage, secrets, provider } = context;

/// If conditions aren't met
return { canExec: false
          Message:’no execution due…’}


// if conditions are met
return { canExec: true
          callData: payload to be executed}


})

Let’s have a look at each code block of the implementation.

////// User Arguments
 const nftAddress = userArgs.nftAddress;
 if (!nftAddress) throw new Error("Missing userArgs.nftAddress please provide");


 ////// User Secrets
 const nftStorageApiKey = await secrets.get("NFT_STORAGE_API_KEY");
 if (!nftStorageApiKey) throw new Error("Missing secrets.NFT_STORAGE_API_KEY");


 const openAiApiKey = await secrets.get("OPEN_AI_API_KEY");
 if (!openAiApiKey) throw new Error("Missing secrets.OPEN_AI_API_KEY");


////// User Storage
 const lastProcessedId = parseInt((await storage.get("lastProcessedId")) ?? "0");

First we’ll need to get the nftContract address from the user arguments.

Second, we’ll get the nftStorageApiKey from our secrets to upload the OpenAI-generated images to IPFS, and we’ll also need the openAIKey to call the OpenAI API. Finally, we’ll get the last tokenID processed from our storage, so we can ensure only new NFTs are getting the new image.

const nft = new Contract(nftAddress as string, NFT_ABI, provider);


 const currentTokenId = await nft.tokenIds();
 if (currentTokenId.eq(BigNumber.from(lastProcessedId))) {
   return { canExec: false, message: "No New Tokens" };
 }
 const tokenId = lastProcessedId + 1;

After instantiating the NFT contract with ethes.js, we should check the current number of NFTs that have been minted. If this number equals the last processed NFT, we can return to our task without executing any transactions.

However, if there are new NFTs, then we will go on and generate the next NFT image.

This part is straightforward: we instantiate a client OpenAI, we create the prompt, and we get the image URL.

//// Generating Attribures
function generateNftProperties(isNight: boolean) {
 const timeSelected = isNight ? "at night" : "at sunset";


 const description = `A cute robot eating an icecream with Dubai background ${timeSelected} in a cyberpunk art, 3D, video game, and pastel salmon colors`;
 return {
   description,
   attributes: [
     { trait_type: "Time", value: timeSelected },
     { trait_type: "Place", value: "Eth Dubai" },
     { trait_type: "Eating", value: "Gelato" },
     { trait_type: "Powered", value: "Web 3 Functions" },
   ],
 };
}
const openai = new OpenAIApi(new Configuration({ apiKey: openAiApiKey }));
   let imageUrl: string;
   try {
     const response = await openai.createImage({
       prompt: nftProps.description,
       size: "512x512",
     });
     imageUrl = response.data.data[0].url as string;
     console.log(`Open AI generated image: ${imageUrl}`);

Once we have the image URL, we then download the image as a Blob, instantiate the nftStorage client, upload the image to IPFS, and get the metadata in return.

// Publish NFT metadata on IPFS
   const imageBlob = (await axios.get(imageUrl, { responseType: "blob" })).data;
   const client = new NFTStorage({ token: nftStorageApiKey });
   const imageFile = new File([imageBlob], `gelato_bot_${tokenId}.png`, { type: "image/png" });


   const metadata = await client.store({
     name: `Eth Dubai GelatoBot #${tokenId}`,
     description: nftProps.description,
     image: imageFile,
     attributes: nftProps.attributes,
     collection: { name: "EthDubai-GelatoBots", family: "ethdubai-gelatobots" },
   });
   console.log("IPFS Metadata:", metadata.url);

   await storage.set("lastProcessedId", tokenId.toString());

Finally, we will encode our transaction to reveal the NFT and send it on-chain

return {
     canExec: true,
     callData: nft.interface.encodeFunctionData("revealNft", [tokenId, metadata.url]),};

You can see the full code here

Gelato is a Web3 Cloud Platform empowering developers to create automated, gasless, and off-chain-aware Layer 2 chains and smart contracts. Over 400 web3 projects rely on Gelato for years to facilitate millions of transactions in DeFi, NFTs, and gaming.

• Gelato RaaS: Deploy your own tailor-made ZK or OP L2 chains in a single click with native Account Abstraction and all Gelato middleware baked in.

• Web3 Functions: Connect your smart contracts to off-chain data & computation by running decentralized cloud functions.

• Automate: Automate your smart contracts by executing transactions automatically in a reliable, developer-friendly & decentralized manner.

• Relay: Give your users access to reliable, robust, and scalable gasless transactions via a simple-to-use API.

• Account Abstraction SDK: Gelato has partnered with Safe, to build a fully-fledged Account Abstraction SDK, combining Gelato's industry's best gasless transaction capabilities, with the industry's most secure smart contract wallet.

Subscribe to our newsletter and turn on your Twitter notifications to get the most recent updates about the Gelato ecosystem! If you are interested in being part of the Gelato team and building the future of the Internet browse the open positions and apply here.