Develop your first Projection

Pre-requisites

Before you start developing your first projection, you’ll need to install the following tools:

Docker, a containerization platform.
Node.js
Yarn, a package manager for Node.js. We used yarn but feel free to use any.
Code editor of your choice. We recommend Visual Studio Code.
Experience with The Graph Protocol is going to make your life very easy and most of the guide will be very familiar to you. In fact we encourage you to spend some time learning about it.
A URL of an Ethereum Mainnet Archive node.

Environment setup

Open VSCode
Install Dev Containers VSCode Extension

Read more about dev containers.
Create Datai App by running the following command in your terminal
terminal
```
npx create-datai-app
```
Open the project inside of a container

Using the “Command Palette” find a Reopen in Container option. F1 or Ctrl+Shift+P -> Reopen in Container

Project Structure

Let’s take a closer look at the project structure. For the purpose of this tutorial, and for the majority of use cases, you will only need to write code inside of assembly folder. Highlighted are the folders and files you’ll be working with during this tutorial.

Directory.devcontainer (VSCode Dev Container configuration)
- …
DirectoryAPI (protocol buffer definitions)
- …
Directorygraph-node (subgraph configuration)
- …
Directoryprojections
- Directoryassembly
  - Directorylibs (your common libraries of reusable code)
    …
  - Directoryprotocols (protocol configurations used by templates)
    Directory[protocol-name]
    Directory[protocol-name].[protocol-module]
    Directory[network]
    [manifest_name].yaml
  - Directorytemplates (actual implementation)
    Directory[manifest-name]
    Directoryabis (smart contract interfaces)
    …
    Directory.generated (output for generated code, do not edit)
    …
    Directorysubgraph (subgraph implementation)
    …
    Directorywatcher (watcher implementation)
    index.ts (watcher entry point)
    schema.graphql (subgraph schema)
    subgrah.yaml (yaml manifest template)
- Directorytypescript (utility scripts)
  - …

Your first projection

For the purpsoe of this tutorial we will implement Uniswap V2 Liquidity module projection on the Ethereum Mainnet network.

Projection configuration

Inside of the assembly/protocols create uniswap2.liquidity_pool directory, then a mainnet directory for Ethereum Mainnet network, finally a uniswap2_liquidity_pool.yaml file.

…
Directoryprojections
- Directoryassembly
  - Directoryprotocols
    Directoryuniswap2.liquidity_pool
    Directorymainnet
    uniswap2_liquidity_pool.yaml

Create data source configuration

Using the following configuration, you will define the data source for the projection. This configuration will be used by the Datai engine to fetch the data from the Ethereum blockchain.

dataSources:
  - <<: *refFactorySource
    name: Factory
    network: mainnet
    source:
      address: '0x5C69bEe701ef814a2B6a3EDD4B1652CB9cc5aA6f'
      abi: Factory
      startBlock: 10000834
templates:
  - <<: *refPairSource
    name: Pair
    network: mainnet

Template configuration

Next, we will take care of the implementation. Inside of the templates folder, create a new folder with the name of the projection and inside of it create a new file named subgraph.yaml.

…
Directoryprojections
- Directoryassembly
  - …
  - Directorytemplates
    Directoryuniswap2_liquidity
    subgraph.yaml

Template manifest

Now we will use the projection configuration from protocols/uniswap2.liquidity_pool/mainnet/uniswap2_liquidity_pool.yaml to create a subgraph manifest. For this we need to be familair with YAML anchors and YAML references. If you don’t know them yet, don’t worry. Just follow along.

specVersion: 1.2.0
schema:
  file: ./schema.graphql
refs:
  - &refFactorySource
    kind: ethereum/contract
    source:
      abi: Factory
    mapping:
      kind: ethereum/events
      apiVersion: 0.0.7
      language: wasm/assemblyscript
      file: ./subgraph/mappings/factory.ts
      entities: []
      abis:
        - name: Factory
          file: ./abis/UniswapV2Factory.json
      eventHandlers:
        - event: PairCreated(indexed address,indexed address,address,uint256)
          handler: handleNewPair

  - &refPairSource
    kind: ethereum/contract
    source:
      abi: Pair
    mapping:
      kind: ethereum/events
      apiVersion: 0.0.7
      language: wasm/assemblyscript
      file: ./subgraph/mappings/pool.ts
      entities: []
      abis:
        - name: Pair
          file: ./abis/UniswapV2Pair.json
        - name: Factory
          file: ./abis/UniswapV2Factory.json
      eventHandlers:
        - event: Transfer(indexed address,indexed address,uint256)
          handler: handleTransfer

Factory and Pair sources

As you may have noticed &refFactorySource and &refPairSource they’re the same sources we’ve defined previously in uniswap2_liquidity_pool.yaml.

Projections are split into configuration and projection templates for a simple reason. There are a number of pouplar forked protocols that share the same code and events. All that’s left for us developers to do, is to use different contract addresses, networks nad start blocks as sources.

Difference between data source and template

Templates also exist in the subgraph manifests. For more information see The Graph Protocol.

Data source defines a contract that is deployed once on the network.
Template is a smart contract that is deployed by another contract. A good example of this is a Factory contract that creates Pair contracts.

Event handlers

In both DataSource &refFactorySource and Template refPairSource we can see eventHandlers. They are a list of event signatures that are mapped to handlers which are exported functions from a file inside of our mapping. In our case file: ./subgraph/mappings/factory.ts.

This means that

eventHandlers:
  - event: PairCreated(indexed address,indexed address,address,uint256)
    handler: handleNewPair

Will be mapped directly to

...

export function handleNewPair(event: PairCreated): void {
// Do something with the event
}

...

Smart Contract ABIs

We’ve created our manifests but we still need to define the smart contract interfaces for our app to understand and give us full type-safety. Inside of the templates/uniswap2_liquidity folder create a new folder named abis.

…
Directoryprojections
- Directoryassembly
  - …
  - Directorytemplates
    Directoryuniswap2_liquidity
    …
    Directoryabis
    …

Knowing the smart contract addresses, we are able to find their ABIs in one of the popular scans. In our case let’s use Etherscan and look for the Factory contract that has an address of 0x5C69bEe701ef814a2B6a3EDD4B1652CB9cc5aA6f

On the etherscan page, go to the Contract tab and find the Code section.

Then towards the bottom of the page you will find the Contract ABI section. Click on the copy button to copy the ABI.

Now inside of templates/uniswap2_liquidity/abis create a new file named Factory.json and paste the copied ABI. Repeat the same process for the Pair contract with the address 0x0d4a11d5eeaac28ec3f61d100da8b32d6c6b6c1b.

Subgraph schema

Last configuration step before actual implementation is to define the schema for the subgraph. Inside of the templates/uniswap2_liquidity folder create a new file named schema.graphql. Use the following contents. We will go over it in the next section.

# common entities

type User @entity(immutable: true) {
  id: ID!
  positions: [UserPosition!]! @derivedFrom(field: "user")
}

enum UpdateType {
  SCHEDULE
  EVENT
}

type UpdateTrigger @entity {
  id: ID!
  updateType: UpdateType!
  nonce: Bytes!
}

type PositionUpdateTrigger @entity {
  id: ID!
  user: User!
  updateTrigger: UpdateTrigger!
}

interface UserPosition {
  id: ID!
  user: User!
}

# protocol-specific entities

type Uniswap2UserPosition implements UserPosition @entity(immutable: true) {
  id: ID!
  user: User!
}

type Uniswap2Pool @entity(immutable: true) {
  id: ID!
  token0: Bytes!
  token1: Bytes!
}

Common

These are required common types that are used by Datai engine to process your projections.

Protocol Specific Entities

Here we often need to keep track of specific events and data that. In our case we will define Uniswap2UserPosition and Uniswap2Pool entities.

Note that we’re using @entity(immutable: true) directive. This means that once the entity is created it cannot be updated. This is important for performance reasons.

`Subgraph` and `Watcher`

Projection relies on two main components. Subgraph is responsible for tracking whether a user has had a position in a given protocol at any time in the past. It is responsible for saving a trigger for a watcher to fetch contract state and compute final position of a user at the time of request.

It will all make sense when you see the implementation.

Subgraph Implementation

Now we get to the implementation part. We will put our events to work and implement their handlers. Inside of .../templates/uniswap2_liquidity/subgraph create a new file named factory.ts and paste the following contents.

import { PairCreated } from '../../generated/Factory/Factory'
import { Pair as PairTemplate } from '../../generated/templates'
import { Uniswap2Pool } from '../../generated/schema'

export function handleNewPair(event: PairCreated): void {
  // Store some entities
  let pool = Uniswap2Pool.load(event.params.pair.toHexString())
  if (pool == null) {
    pool = new Uniswap2Pool(event.params.pair.toHexString())
    pool.token0 = event.params.token0
    pool.token1 = event.params.token1
    pool.save()
  }

  // create the tracked contract based on the template
  PairTemplate.create(event.params.pair)
}

You’ll notice that your typescript language server will complain about missing types. This is because we haven’t generated the types yet. We will take care of that in just a bit.

Take a look at the above code, and notice the template creation.

...
PairTemplate.create(event.params.pair);
...

This tells the subgraph that from now on, it needs to listen to events on the address of a pair as provided by event.params.pair. Without it, it would have no knowledge of all the Uniswap V2 Liquidity Pools that have been deployed to the network by the Factory.sol contract.

Aside from creating a template, we also have a convenient access to the tokens that make up the pair. We first load the pool, if not already created, and then save token0 and token1 addresses to our subgraph storage. This makes it much faster than querying the contract state each time we need it in the future.

...
export function handleNewPair(event: PairCreated): void {
// Store some entities
let pool = Uniswap2Pool.load(event.params.pair.toHexString())
if (pool == null) {
  pool = new Uniswap2Pool(event.params.pair.toHexString())
  pool.token0 = event.params.token0
  pool.token1 = event.params.token1
  pool.save()
}

// create the tracked contract based on the template
PairTemplate.create(event.params.pair)
}
...

Next, create a new file named pool.ts and paste the following contents.

import { Transfer } from '../../generated/templates/Pair/Pair'
import { ensureUpdateTrigger, registerUser, ADDRESS_ZERO } from 'datai-sdk'
import { Uniswap2UserPosition } from '../../generated/schema'

export function handleTransfer(event: Transfer): void {
  if (
    event.params.to.toHexString() == ADDRESS_ZERO ||
    event.params.to == event.address
  ) {
    return
  }

  // Ids
  const userId = event.params.to.toHexString()
  const poolId = event.address.toHexString()
  const positionId = userId + '-' + poolId

  // If trigger exists do nothing
  if (ensureUpdateTrigger(positionId, userId)) {
    return
  }

  // Protocol logic
  let uniswap2UserPosition = Uniswap2UserPosition.load(positionId)
  if (uniswap2UserPosition == null) {
    uniswap2UserPosition = new Uniswap2UserPosition(positionId)
    uniswap2UserPosition.user = userId
    uniswap2UserPosition.save()
  }

  // Register User
  registerUser(userId)
}

The above code does a few things. Firstly, it handles the Transfer event of an ERC20 Liquidity Pool token.

It creates a trigger for our Watcher and if one already exists, it returns and does nothing.

In case the trigger is not yet created it creates a track record in Uniswap2UserPosition entity.

Finally, it registers the user’s position in the protocol.

Compile subgraph

As you may have noticed, our typescript transpiler is complaining about missing types. We need to generate them and compile our subgraph to make sure everything is correct. Run the following command in your terminal.

yarn build uniswap2_liquidity uniswap2 liquidity_pool mainnet graph

This will generate the necessary types and templates for your projection. Let’s look at the command itself and each of the arguments.

Here’s a quick explanation of the above command. As you can see, there are 5 arguments passed to the build script.

yarn build <TEMPLATE_NAME> <PROTOCOL_NAME> <MODULE_NAME> <NETWORK> codegen

`<TEMPLATE_NAME>`

This is the name of the projection template you’re building. In our case, it’s uniswap2_liquidity. Our template must be located in ../templates/uniswap2_liquidity

`<PROTOCOL_NAME>`

It’s the name of the protocol configuration you’re building. In our case it’s uniswap2. Our protocol configuration must be located in ../protocols/uniswap2.liquidity_pool

`<MODULE_NAME>`

It’s the name of the module you’re building. In our case it’s liquidity_pool. Our module must be located in ../protocols/uniswap2.liquidity_pool. Please note the convention of <PROTOCOL_NAME>.<MODULE_NAME> convention for naming of the folders.

`<NETWORK>`

Name of the network that the projection’s smart contracts are deployed to. See supported blockchains for available options. In our case it’s mainnet. Please note ../protocols/uniswap2.liquidity_pool/mainnet folder structure.

`codegen`

This command will build, transpile and generate classes and types from your schema.graphql, subgraph.yaml, protocol-name.yaml, ABIS inside of /abis folder as well as Datai SDK API utility classes so you can reference them in your implementation code.

Watcher Implementation

Having implemented the subgraph part, the last missing piece is the watcher. As mentioned in previous sections, it is responsible for fetching contract state and computing the user’s position for each pool that a subgraph has found.

Inside of .../templates/uniswap2_liquidity/watcher create a new file named index.ts and paste the following contents.

import { Address, BigInt } from '@graphprotocol/graph-ts'
import { Pair } from '../generated/templates/Pair/Pair'
import {
  activePositions,
  ActivePositionsResult,
  BI_0,
  bytesToAddress
} from 'datai-sdk'
import { Uniswap2Pool, Uniswap2UserPosition } from '../generated/schema'

export function GetActivePositions(): void {
  const uniswap2Position = activePositions.inputPosition<Uniswap2UserPosition>()
  const output = new ActivePositionsResult()

  const uniswap2UserPosition = Uniswap2UserPosition.load(uniswap2Position.id)
  if (uniswap2UserPosition == null) {
    console.error('Position is not registered!')
    throw new Error('Position' + uniswap2Position.id + 'is not registered!')
  }
  const userAddress = Address.fromString(uniswap2UserPosition.id.split('-')[0])
  const poolAddress = Address.fromString(uniswap2UserPosition.id.split('-')[1])
  const pool = Uniswap2Pool.load(poolAddress.toHexString())
  if (pool == null) {
    console.error('Pool is not registered!')
    throw new Error('Pool' + poolAddress.toHexString() + 'is not registered!')
  }

  const positionBalance = getPositionBalance(userAddress, poolAddress)
  const underlyingBalances = getUnderlyingAmounts(poolAddress, positionBalance)

  // Set results
  output.setSupplyBalance(bytesToAddress(pool.token0), underlyingBalances[0])
  output.setSupplyBalance(bytesToAddress(pool.token1), underlyingBalances[1])
  output.setPositionTokenBalance(poolAddress, positionBalance)
  output.poolAddress = poolAddress

  activePositions.output(output)
}

function getPositionBalance(user: Address, poolAddress: Address): BigInt {
  const poolContract = Pair.bind(poolAddress)
  const userBalance = poolContract.try_balanceOf(user)

  if (userBalance.reverted) {
    return BI_0
  }

  return userBalance.value
}

function getUnderlyingAmounts(
  poolAddress: Address,
  userBalance: BigInt
): Array<BigInt> {
  const poolContract = Pair.bind(poolAddress)
  const totalSupply = poolContract.try_totalSupply()
  const reserves = poolContract.try_getReserves()

  if (
    totalSupply.reverted ||
    reserves.reverted ||
    userBalance == BI_0 ||
    totalSupply.value == BI_0
  ) {
    return [BI_0, BI_0]
  }

  const balance0 = reserves.value
    .get_reserve0()
    .times(userBalance)
    .div(totalSupply.value)

  const balance1 = reserves.value
    .get_reserve1()
    .times(userBalance)
    .div(totalSupply.value)

  return [balance0, balance1]
}

This is by far the most complex piece of our projection. Let’s break it down.

`GetActivePositions()`

A watcher must export a void function named GetActivePositions. It will be called by the Datai Engine to fetch the user’s positions.

`Active Positions - inputPosition`

Datai Engine will inject subgraph’s triggers into the watcher. We can access it by calling activePositions.inputPosition<Uniswap2UserPosition>(). This will give us a list of all the positions that need to be computed. If a user has many positions, a watcher will be executed once for each trigger. Meaning that the execution context of a watcher is always one position.

const uniswap2Position = activePositions.inputPosition<Uniswap2UserPosition>()

Then we need to load the position details from the subgraph storage. We do this by getting the id from activePositions.inputPosition()

const uniswap2UserPosition = Uniswap2UserPosition.load(uniswap2Position.id)

After making sure the position exists in the subgraph storage, and splitting the id into userAddress and poolAddress, we need to fetch the balances and compute our amounts.

const positionBalance = getPositionBalance(userAddress, poolAddress)
const underlyingBalances = getUnderlyingAmounts(poolAddress, positionBalance)

getPositionBalance function is responsible for fetching the user’s balance of the pool token. getUnderlyingAmounts function is responsible for fetching the underlying token amounts.

Following up on our watcher implementation. After we’re able to fetch and compute user’s balances, we need to create our output which is an ActivePositionsResult().

...
// Set results
output.setSupplyBalance(bytesToAddress(pool.token0), underlyingBalances[0])
output.setSupplyBalance(bytesToAddress(pool.token1), underlyingBalances[1])
output.setPositionTokenBalance(poolAddress, positionBalance)
output.poolAddress = poolAddress

activePositions.output(output)
...

This is the final step of our watcher implementation. We set the results and output them to the Datai Engine.

Compile watcher

Following the same command structure of arguments for yarn build command we can now compile our subgraph and watcher.

Run the following command in your terminal to compile your watcher.

yarn build uniswap2_liquidity uniswap2 liquidity_pool mainnet watcher

Next Steps

If you followed along you should now have a fully functional Uniswap V2 Liquidity Pool projection and the compiler should give you no errors. However, if you encounter any errors, please reach out to us on our communication channels as listed in Community Support.

We also recommend you check the Datai SDK API Reference for more information on how to interact with the Datai Engine.

In the next section, we will deploy our projection to the Datai Network or to our local machine for testing.