Application Workflow

Use the Crynux Network as the inference API

The easiest method to connect an application to the Crynux Network is to deploy a Crynux Bridge, and connect the application to the bridge using API. The Crynux Bridge will take care of the application wallet, and all the interactions with the blockchain and Relay. The tutorial can be found in the following document:

Crynux Bridge

Another convenient method is to use the SDK to embed the workflow directly into the application. The details of the SDKs are explained in the following document:

Crynux SDK

The application can utilize the Crynux Network as an API service. It sends inference tasks to the network and receives images or texts in return. Two types of inference tasks are supported: Stable Diffusion image generation and GPT text generation.

The application interacts with two network components: the blockchain node and the Relay. To send tasks successfully, it must have a wallet with sufficient Test CNX tokens for payment. Test CNX tokens can be acquired for free on the Discord Server of Crynux.

Reference applications are provided for both image generation and text generation tasks. The source code can be found on the GitHub.

As the first step, we will provide a high-level overview of the complete workflow, outlining the main steps involved in the process.

Overview

The application workflow is illustrated in the graph below:

The application initiates the workflow by calling the CreateTask method of the smart contract. This method receives task parameters related to the task criteria, such as the task type and VRAM requirements, which the network uses to select suitable nodes.

The application transfers the task fee to the contract address by specifying it in the transaction's value field. Upon task completion, tokens are sent to the nodes. If the task fails, the fee is refunded to the application's wallet.

After the transaction is confirmed on-chain, the application should then send the task parameters to the Relay.

Selected nodes will retrieve task parameters from the Relay and then execute the tasks locally.

When images or texts are generated, nodes will create proofs and send them to the blockchain. The blockchain will verify the correctness of these proofs and transfer tokens to the nodes upon successful verification.

The nodes will upload the result images/texts to the Relay, which will compare the results with the on-chain proofs to verify their accuracy.

After sending the task parameters to the Relay, the application should wait for the TaskSuccess event from the blockchain. Once the event is received, the application can retrieve the images or texts from the Relay, marking the completion of the task workflow.

The results have already been verified by the Relay, so no further verification by the application is necessary.

For a detailed workflow involving all network participants, please refer to the task lifecycle document:

Task Lifecycle

The Reference Applications

The workflow has been fully implemented in the showcase applications: the Image Generator and the AI Chatbot. Which can be accessed at:

The Image Generator: https://ig.crynux.ai

The AI Chatbot: https://chat.crynux.ai

Both applications utilize the Crynux Bridge as the backend. The Crynux Bridge includes a built-in wallet to cover task fees, eliminating the need for applications to manage their own wallets. Additionally, it isolates the blockchain and Relay from the applications. This allows applications to simply submit task parameters via API and await the result without further action.

The Crynux Bridge can be used by all the applications. The source code of the Crynux Bridge can be found at:

GitHub - crynux-ai/crynux-bridge: Crynux Bridge connects traditional applications to the Crynux Network. The bridge provides simple APIs to create task and receive results. All the wallet and blockchain related stuff are transparently taken care of at the bridge.GitHub

The source code of the web UI of the Image Generator:

GitHub - crynux-ai/ig-web: Stable Diffusion image generator web ui using Crynux NetworkGitHub

The source code of the web UI of the AI Chatbot

GitHub - crynux-ai/chat-web: Web application for crynux gpt taskGitHub

Application Workflow Step by Step

1. Prepare the application wallet

Crynux Network is currently deployed on the L2 blockchain of Dymension as a testnet Rollapp. Test CNX tokens are used. You could join the Discord Server of Crynux to get the test tokens for free.

An Ethereum compatible wallet must be generated. Which will be used by the application to invoke the smart contracts on-chain.

Ensure the wallet has sufficient Test CNX tokens to cover both task and transaction fees. The application should continuously monitor the wallet balance and alert admins to replenish tokens before it drops below the required amount for upcoming tasks.

In the DApp, an application wallet is not required. The DApp will create the transaction and send it to Metamask for the user to sign directly in the browser.

2. Create the Task on the Blockchain

Construct the task parameters

The task parameters are organized as a JSON string. An example of the parameters of an image generation task is given below:

{
    "version": "2.0.0",
    "base_model": {
        "name": "stabilityai/sdxl-turbo"
    },
    "prompt": "best quality, ultra high res, photorealistic++++, 1girl, desert, full shot, dark stillsuit, "
              "stillsuit mask up, gloves, solo, highly detailed eyes,"
              "hyper-detailed, high quality visuals, dim Lighting, ultra-realistic, sharply focused, octane render,"
              "8k UHD",
    "negative_prompt": "no moon++, buried in sand, bare hands, figerless gloves, "
                       "blue stillsuit, barefoot, weapon, vegetation, clouds, glowing eyes++, helmet, "
                       "bare handed, no gloves, double mask, simplified, abstract, unrealistic, impressionistic, "
                       "low resolution,",
    "task_config": {
        "num_images": 9,
        "steps": 1,
        "cfg": 0
    },
    "lora": {
        "model": "https://civitai.com/api/download/models/178048"
    },
    "controlnet": {
        "model": "diffusers/controlnet-canny-sdxl-1.0",
        "image_dataurl": "data:image/png;base64,12FE1373...",
        "preprocess": {
            "method": "canny"
        },
        "weight": 70
    },
    "scheduler": {
        "method": "EulerAncestralDiscreteScheduler",
        "args": {
            "timestep_spacing": "trailing"
        }
    }
}

The task definition above follows the schema given in the Stable Diffusion Task Framework. A wide range of common configurations are supported. The framework also provides a JSON schema to validate task parameters. More information about the framework can be found in the document below:

Text-to-Image Task

A similar framework for the GPT text generation task is also provided in the following document:

Text-to-Text Task

The application must validate task parameters against the schema before sending them to the network, especially when the task parameters are generated by the user on the frontend.

Send the create task transaction to the blockchain

Once the JSON string for the task parameters is ready, the application must create and send the CreateTask transaction to the blockchain.

CreateTask method of the Task Contract has five arguments:

function createTask(
    uint taskType,
    bytes32 taskHash,
    bytes32 dataHash,
    uint vramLimit,
    uint cap
)

• taskType is an integer that identifies the task type: 0 for SD task and 1 for GPT task.

• taskHash is the keccak256 hash of the JSON string of the task arguments.

• dataHash is reserved for the future features and is not used right now. The application could just pass 32 zero bytes to it.

• vramLimit indicates the minimum VRAM required to execute the task. The Crynux Network will select the capable nodes based on this value.

• cap indicates the task size. It is used to estimate the task execution time by the Crynux Network. It should be set to the number of images in the SD task, and 0 in a GPT task.

In addition to the arguments listed above, the task fee should be set in the value field of the transaction. The application is free to choose any task fee value, a higher task fee will result in a faster task execution, while lower task fee will result in longer waiting time.

The source code that implements the invocation of the CreateTask method in the Crynux Bridge can be found here.

Wait for the transaction confirmation

After sending the transaction, the application should wait for confirmation before proceeding. The transaction might be reverted by the blockchain for various reasons. All possible reasons for a transaction being reverted can be found in the source code.

If the transaction is reverted, no event will be emitted. Therefore, the creation result can only be queried using the transaction hash or the receipt provided by the blockchain when sending the CreateTask transaction.

3. Upload the Task Parameters to the Relay

Once the transaction is confirmed, the next step is to upload the task parameters JSON string to the Relay. Use the following API endpoint:

Create an inference task

POST/v1/inference_tasks

Body

application/json

signature*string

Signature

task_args*string

Task arguments

task_id*integer (int64)

Task id

timestamp*integer (int64)

Signature timestamp

Response

200400500

OK

Body

application/json

dataInferenceTask (object)

messagestring

The response message. Will be 'success' or a detailed error type

Request

JavaScriptCurlPython

const response = await fetch('/v1/inference_tasks', {
    method: 'POST',
    headers: {
      "Content-Type": "application/json"
    },
    body: JSON.stringify({
      "signature": "text",
      "task_args": "text"
    }),
});
const data = await response.json();

Response

200400500

{
  "data": {
    "CreatedAt": "2024-11-21T06:57:50.985Z",
    "DeletedAt": {
      "Time": "2024-11-21T06:57:50.985Z",
      "Valid": false
    },
    "SelectedNodes": [
      {
        "CreatedAt": "2024-11-21T06:57:50.985Z",
        "DeletedAt": {
          "Time": "2024-11-21T06:57:50.985Z",
          "Valid": false
        },
        "IsResultSelected": false,
        "NodeAddress": "text",
        "Result": "text",
        "UpdatedAt": "2024-11-21T06:57:50.985Z"
      }
    ],
    "UpdatedAt": "2024-11-21T06:57:50.985Z",
    "abort_reason": "text",
    "creator": "text",
    "data_hash": "text",
    "task_args": "text",
    "task_fee": 0,
    "task_hash": "text"
  },
  "message": "text"
}

The complete API documentation can be found in the OpenAPI Specifications of the Relay server.

To upload, simply invoke the API to the Relay server. Ensure the request is signed by the application wallet before sending.

The Relay tracks the blockchain for task creations, recording the task ID and the creator's address (application wallet) upon creation. To upload task arguments, the request must originate from the same task creator's wallet with a verified signature.

The signature is generated using ECDSA with the same curve as Ethereum, on the Keccak256 hash of a string. This string is created by including all query and body parameters (except timestamp and signature) from the request in a JSON string with keys sorted alphabetically and concatenated with the current Unix timestamp.

The reference implementation of the signing method in Crynux Bridge can be found here. The code to upload the task parameters to the Relay can also be found in the source code.

4. Wait for the Task to Finish

When the task finishes, either the TaskSuccess or TaskAborted event will be emitted. If the TaskSuccess event is emitted, the application can retrieve the result from the Relay. If the TaskAborted event is emitted, indicating a failure, the application can retry by creating a new task.

Several reasons can cause task execution failure. Task arguments might not pass node schema validation, some nodes might not run the consensus protocol correctly, or a task might take too long on a single node. The exact reason is included as an argument in the emitted event.

If a task is aborted, CNX tokens may either be returned to the application wallet or still paid to the nodes. This depends on who is at fault for the task's failure.

There are two ways the application could monitor the blockchain for relevant events.

Tracking new blocks and filtering the target events

The first method involves continuously tracking new blocks and filtering them for these two types of events.

To ensure reliable block tracking, the application must handle potential crashes caused by unhandled bugs. Additionally, extended downtime can result in delays when catching up with new blocks.

Query for the task status periodically

Another approach is to extract the task ID from the creation transaction, store it, and periodically check the blockchain for the latest task status. This method eliminates the need to track the block, but it is less efficient due to a high volume of unnecessary queries.

The Crynux Bridge uses the first method, the source code of the block synchronization can be found here.

5. Fetch the result from the Relay

The final step is to retrieve the actual images or texts from the Relay. This can be accomplished by calling the Relay's API as follows:

Get images

The URL could be treated like an image downloading link as it returns the binary stream of the image content directly. The signature and timestamp is still required.

Get the result of the inference task by node address

GET/v1/inference_tasks/{task_id}/results/{image_num}

Path parameters

image_num*string

Image number

task_id*integer (int64)

Task id

Query parameters

Response

200400

OK

Request

JavaScriptCurlPython

const response = await fetch('/v1/inference_tasks/{task_id}/results/{image_num}?signature=text', {
    method: 'GET',
    headers: {},
});
const data = await response.json();

Response

400

{
  "data": {
    "field_name": "text",
    "message": "text"
  },
  "message": "text"
}

Get texts

The API endpoint to get text results from the Relay is the same as the endpoint above, except that the image_num should be set to zero.

When the application accesses the above URL after the TaskSuccess event is received, it could keep getting 404 not found for a short while before it gets the correct results. The reason is that the node will start to upload images/texts to the Relay only after the TaskSuccess event is received. So before the uploading is done, the application can not find the results on the Relay. Several times of retrying is required at this place.

When the application accesses the URL after receiving the TaskSuccess event, it might encounter 404 not found errors temporarily. This occurs because the node initiates the upload of images/texts to the Relay only after the TaskSuccess event is triggered. Therefore, the results won't be available on the Relay until the upload is complete. Retrying the request several times may be necessary.

The source code where the Crynux Bridge downloads the images is located here.