MINA zkApp staking pool

Tue, 10 Jan 2023 00:00:00 +0000

This article is published on HackMD.

Currently, when delegating to a block producer (aka validator) on MINA, the distribution of rewards is conducted off-chain, so block producers are responsible for calculating and distributing payouts according to their terms and conditions. Typical terms and conditions would include the fee percentage, the payout frequency, and perhaps the payout algorithm used. While the delegator’s funds are not at risk (you merely delegate the right to produce a block), a pool operator can steal any rewards in the worst-case scenario. The only recourse for a delegator would be to switch, which due to the latency involved in delegation transactions, would result in at least two epochs of rewards lost.

An Introduction to MinaExplorer's BigQuery Public Dataset

Fri, 15 Oct 2021 00:00:00 +0000

This article is published on Medium.

Mina is a succinct blockchain, which means while we can verify the chain’s current state using a zero-knowledge proof, the prior history is not retained. So, if we want to analyze the chain’s history (for example, to see individual transactions), we need to obtain it from an archive node. The official implementation of an archive node stores its data to a Postgres database, which typically requires the use of recursive queries to determine the canonical chain information.

Evaluating a Mina staking pool’s performance

Wed, 21 Jul 2021 00:00:00 +0000

This article is published on Medium.

Mina uses a proof of stake consensus mechanism with a block producer selection similar to Ouroboros, known as Ouroboros Samisika. The opportunity to produce a block for a slot is determined by a verifiable random function (VRF). This selection can be thought of like a lottery with each block producer independently running this function for each slot. If they get a VRF output greater than a threshold proportional to the producer’s stake, they get the chance to produce a block at the designated slot.

Running a Mina Archive Node

Tue, 27 Apr 2021 00:00:00 +0000

This article is published on Medium.

Mina is a succinct blockchain, and as a result, consensus nodes only store the recent history of the chain before discarding it (the last k blocks, currently 290). While prior transaction history is not required to prove the current state is valid (this is handled via a recursive zero-knowledge proof), many applications would like access to this prior transaction history. Examples include block explorers and wallets. To solve this problem, users may optionally run an archive node that stores a summary of each block seen in a Postgres database.

Introduction to the Coda Javascript SDK

Thu, 23 Jul 2020 00:00:00 +0000

This article is published on Medium.

This post provides an introduction to the current implementation of the Coda Javascript SDK. Using the SDK, it is possible with Javascript to generate keys, sign and verify messages, and sign transactions that can be broadcast to the network. From the package’s Readme:

This is a NodeJS client SDK that allows you to sign transactions and strings using Coda’s keypairs. The project contains Typescript and ReasonML typings but can be used from plain NodeJS as well.

Consensus on the Coda Blockchain

Tue, 14 Jul 2020 00:00:00 +0000

This article is published on Medium.

Coda is a succinct blockchain where full nodes only require downloading and verification of a tiny proof to ensure that the state of the blockchain is valid. It achieves this through incrementally computed SNARKs where the latest block contains a proof that validates the new block in addition to the previous SNARK.

Coda utilizes a proof of stake (POS) consensus mechanism as opposed to the proof of work (POW) consensus mechanism popularized by Bitcoin. POS consensus algorithms have unique challenges, most notably the nothing at stake issue. Whereas in POW, miners are forced to complete work that expends energy and has a cost, in a POS system it is essentially free to participate, and an adversary can try different things to see what is favorable at no cost.

Running Coda with Docker

Wed, 08 Jul 2020 12:00:00 +0000

Docker provides one of the simplest ways of getting a Coda node up and running while providing the benefits of container isolation. It also resolves issues around dependencies on unsupported systems and allows specifying restart policies should the daemon crash.

It is important to use the correct image for each testnet release. The tags for all releases are available on DockerHub. For testnet 3.2b this tag is 0.0.12-beta-feature-bump-genesis-timestamp-3e9b174. This image is updated for each testnet, so the image used in the below commands should be updated accordingly.

At its simplest, we can start a Coda node with the following command. We’ll build upon this basic command for other Coda specific functions such as block production and SNARK workers.

$ docker run -d --name coda \
-p 8302:8302 \
-p 8303:8303 \
--restart always \
codaprotocol/coda-daemon:0.0.12-beta-feature-bump-genesis-timestamp-3e9b174 daemon \
-peer <PEER_1> \
-peer <PEER_2>

In the above command, you should replace <PEER_1> and <PEER_2> with the correct peer addresses that are made available.

The command will create a container named coda and output a container id, with the container running in the background (due to the -d or --detached option). The --restart always option means that if the container exits, it will automatically restart. There are two published ports (8302, 8303) to allow communication on the peer-to-peer network. To view the running container’s status, run docker ps that should display output similar to the below. Checking the status column, it will indicate any potential issues, such as the container crash-looping.

CONTAINER ID        IMAGE                                                                         COMMAND                  CREATED             STATUS              PORTS                              NAMES
ae9f09595a4f        codaprotocol/coda-daemon:0.0.12-beta-feature-bump-genesis-timestamp-3e9b174   "/usr/bin/dumb-init …"   25 hours ago        Up 25 hours         0.0.0.0:8302-8303->8302-8303/tcp   coda

To view the logs of the container, you can tail the log output using the command:

$ docker logs --follow coda

To interact with the Coda daemon, you can exec into the container to run the available commands. For example, running docker exec -it coda bash will open a terminal inside the container from where you can run any available coda commands, such as coda client status.

Alternatively, you can run from the command line non-interactively, e.g. docker exec coda coda client status. The format of this command is docker exec container_name <COMMAND TO RUN>.

Coda daemon status
-----------------------------------

Global number of accounts:       10089
Block height:                    4969
Max observed block length:       4969
Local uptime:                    6d1h18m32s
...

Running a block producer

As a next step, let’s run a block producer. We’ll remove the current instance via docker rm -f coda to remove the running container and avoid container name and published port conflicts. To run a block producer, you will need the private key and corresponding password for the key that you wish to stake with.

While this private key may be located anywhere on your host system in the following example, the private key (generated via the Coda keygen tool) is in a subdirectory keys and named my-wallet. This key will be bind mounted into the container at /root/keys. We also pass an environment variable of CODA_PRIVKEY_PASS to set the password for the key non-interactively.

Ensure you are running the following command from the directory that contains the keys subdirectory or adjust the bind mount accordingly below, which needs to be an absolute path.

$ docker run -d --name coda \
-e "CODA_PRIVKEY_PASS=<PASSWORD>" \
--mount type=bind,source="$(pwd)"/keys,target=/root/keys \
-p 8302:8302 \
-p 8303:8303 \
--restart always \
codaprotocol/coda-daemon:0.0.12-beta-feature-bump-genesis-timestamp-3e9b174 daemon \
-block-producer-key /root/keys/my-wallet \
-peer <PEER_1> \
-peer <PEER_2>

Once again, you will be returned a container id, and you can follow the logs to identify any potential issues. The following command should indicate a block producer is running with your public key:

$ docker exec coda coda client status | grep "Block producers"
Block producers running:         1 (4vsRCVGPNFDmHvuCctawApbyUg8J6iWzdHvJ2bNP8AL1GtMeaLwC6P8ZNTavKmAD4mMGbDq8H6DYramtiQP5fcf5xhdqAfYibt7Am9syUUU91oSf7HhLKrA26RPutsoyQSmZiMjPZoe1vDXW)

Running a SNARK worker

Another advantage of running the Coda daemon in Docker is limiting the available resources to the container. These resource constraints are of particular use to SNARK workers that will, by default, use all available resources. By default, a container has no resource constraints. However, you can set limits to, for example, the memory available and the number of cpus to use via the --memory and --cpus options.

It is not generally recommended to run a SNARK worker and a block producer on the same machine as it can affect block production. While workarounds exist to disable the SNARK worker at the time of block production, you could alternatively run multiple containers on a machine (adjusting published ports as necessary) with applicable resource constraints.

The following example limits the container running a SNARK worker to 4 CPUs and 8GB of RAM.

$ docker run -d --name coda \
-p 8305:8305 \
--memory 8g \
--cpus 4 \
--restart always \
codaprotocol/coda-daemon:0.0.12-beta-feature-bump-genesis-timestamp-3e9b174 daemon \
-run-snark-worker <PUBLIC_KEY> \
-snark-worker-fee 0.25 \
-work-selection seq \
-peer <PEER_1> \
-peer <PEER_2> \
-external-port 8305

Accessing the GraphQL API

The GraphQL API is available to interact with the Coda daemon. You can read more about the API in this article.

By default, the GraphQL API is bound to localhost. To access it outside of the container, you will need to pass the -insecure-rest-server option allowing the API to listen on all interfaces and then publish this port outside of the container. Crucially, if you do this, you must block the port (3085 by default) from external users via a firewall on the host, as the API is unauthenticated, and anyone with access could gain access to your wallet.

$ docker run -d --name coda \
-e "CODA_PRIVKEY_PASS=xxxxxx" \
--mount type=bind,source="$(pwd)"/keys,target=/root/keys \
-p 8302:8302 \
-p 8303:8303 \
-p 3085:3085 \
--restart always \
codaprotocol/coda-daemon:0.0.12-beta-feature-bump-genesis-timestamp-3e9b174 daemon \
-insecure-rest-server \
-peer <PEER_1> \
-peer <PEER_2>

Once this is running, on the host where Docker is running, visit localhost:3085/graphql. If you are using a VM, you can use SSH port forwarding to forward port 3085 on your development machine to the remote machine, e.g., ssh -L3085:127.0.0.1:3085 user@remotehost.

Additional options

There are many other options available to customize the running daemon, and you should consult the documentation. You can optionally add these options to the docker command, such as -coinbase-receiver that will direct any coinbase payments to an address other than the block producer key, such as to a cold storage wallet.

Prototyping a Coda Blockchain Explorer

Sun, 08 Sep 2019 00:00:00 +0000

This article is published on Medium.

A block explorer is a tool typically used to visualise the current and historical state of a blockchain including all transaction information. Coda, a succinct blockchain, presents some interesting challenges in this regard as this prior state information is not required to be stored by end-users or even block producers, but rather the protocol relies on cryptographic proofs to prove the current chain state (and thus everything preceding it).

Coda currently provides an option for users to run an archive node that persists block data. This data is then exportable via the GraphQL API and it is this data that has been used to produce a prototype block explorer to help users become better acquainted with the testnet and Coda’s functionality.

First steps with the Coda GraphQL API

Mon, 26 Aug 2019 12:00:00 +0000

Coda is a cryptocurrency that uses a succinct blockchain that is enabled through the recursive composition of zk-SNARKs. For clients, this means they can verify the chain with only a few kB of data which opens up exciting opportunities for scalable blockchain applications.

Coda recently launched their public testnet and encourages users who want to help develop the protocol to join the testnet and complete various weekly challenges. While Coda has a traditional command-line interface (CLI) that will be familiar to anyone running a full node implementation, e.g. Bitcoin Core, it also exposes a GraphQL API that we will explore in this article.

GraphQL, which is an open-source query language for APIs, is relatively new being publicly released in 2015 and was developed by Facebook to resolve some common issues with more traditional REST APIs. Now hosted under the umbrella of the GraphQL Foundation (part of the Linux Foundation) many organisations, notably including Github, Gitlab, Shopify, Facebook, and many more all offer a GraphQL API.

While REST APIs still dominate, with GraphQL, clients can obtain all of the data they need in one request and reduces the data returned to only what is required. Let’s illustrate these concepts through an example of retrieving block and transaction data from the Bitcoin blockchain. We’ll use the BitcoinChain API, though the exact implementation is largely irrelevant in exploring the concepts.

We can retrieve the latest block via https://api-r.bitcoinchain.com/v1/blocks/1/withTx which returns the following response containing the full block data as well as all the transactions contained in the block (only three shown in the example below for brevity).

{
  "hash": "0000000000000000001592abc23acc699f54145b5794f56cf60f63f7e412ba2c",
  "height": 591791,
  "prev_block": "0000000000000000000d8dd10509cbfbd778d4ddb359bee4f4700178aff7f015",
  "next_block": "",
  "mrkl_root": "b1700f7e82d199fb8102e88988dae3e5570c7d3731b3864c3131963104613f8b",
  "tx": [
    "e4ea88cf747e9cd3a848a9b933f8b231cdd6b2fbaf950f2b6a202aaefd7c5ea8",
    "09c33e084f7d6fcf2fdc94cb0a80681e82378a0afeeca8563a350c0a5ee2b9f9",
    "c7a3c795142bec4b7b3049a1e3af95adcf20c4490fd2ae631ba63ce7831a0d19",
  ],
  "tx_count": 2748,
  "reward": "12.50000000",
  "fee": "0.22152797",
  "vout_sum": "7230.03209165",
  "is_main": true,
  "version": "1073676288",
  "difficulty": "10183488432890.000000",
  "size": 1150937,
  "bits": "387687377",
  "nonce": "1158314858",
  "time": 1566791311
}

Let’s say for our sample application, we are merely interested in the hash and height of the block. In this case, we are discarding ~99% of the data received, which is not applicable to our use-case. This is an example of overfetching, where we are receiving more data then we require. In GraphQL, we construct the query to only contain the data we want to be returned. An overly simplified example (in reality we would likely pass in arguments such as first to specify how many results we want to return) of the same request would be:

query {
  blocks {
    hash
    height
  }
}

which would return the following JSON:

{
  "data": {
    "blocks": {
      "hash": "0000000000000000001592abc23acc699f54145b5794f56cf60f63f7e412ba2c",
      "height": 591791
    }
  }
}

Conversely, let’s say we wanted all the transaction information for the latest block from the REST API. In this case, we’d need to construct multiple further queries for each transaction using the transaction id to get the required data. As we don’t have all the data we require in our API response, this is known as underfetching. GraphQL solves this problem with the ability to nest our queries, and when it is executed, it can traverse related objects so we can make one request and get two types of data (in this case blocks and transaction data). Using GraphQL, such a query may look like the below:

query {
  blocks {
    hash
    height
    transactions {
      transaction_id
      block_time
      inputs {
        value
        sender
      }
      outputs {
        value
        receiver
      }
    }
  }
}

Whenever a query is executed against a GraphQL server, it is validated against a type system defined by a schema. Coda’s GraphQL schema is published here which defines the objects and types permitted by the API, a number of which we will explore in the remainder of this article. Note that the Coda GraphQL API is in active development and liable to change.

Initial setup of Coda GraphQL API

We’ll first need to install Coda for the current active testnet. Once installed the GraphQL API is enabled via passing the -rest-port flag to the coda daemon startup command, specifying the port to expose, e.g. the following exposes the GraphQL API on port 8000.

coda daemon -rest-port 8000

Note that the GraphQL API is bound to localhost for security. The API is unauthenticated, and as we will demonstrate later with the sendPayment mutation, anyone with access can send payments from a wallet known to the daemon. You can disable this security feature with the -insecure-rest-server option which will listen on all interfaces and not just localhost. If you are running Coda on a remote host the best way is to use SSH port forwarding to forward a local port on your machine to the GraphQL port on the remote machine e.g.

ssh -L 8000:127.0.0.1:8000 user@remotehost

Finally, if you want to work with an existing wallet, you’ll need to import it with a temporary workaround that copies it and strips it of its password. You can also alternatively, as we will demonstrate, create a new wallet via the GraphQL API.

coda daemon -propose-key /keys/my-private-key -unsafe-track-propose-key -rest-port 8000

Now you should be able to point your browser to localhost:8000/graphql and can explore the GraphQL API via the GraphiQL tool, an in-browser IDE for explorer GraphQL APIs.

On the right-hand side, you will see the Documentation explorer that lists the different types available to use. At the root level, GraphQL supports query, mutation and subscription types. Let’s start with some GraphQL queries.

Querying the Coda GraphQL API

Clicking on the query type in the interactive documentation displays a list of fields, the first being syncStatus which returns the “Network sync status”. Enter the following into the left panel and execute the query:

query {
  syncStatus
}

The return value SYNCED is defined as one of the following values. You can use the GraphiQL IDE to explore and test the various queries available. In this example, there was a single return value, but now we’ll use a more advanced query to retrieve block information that requires specifying the exact data we require via nesting.

Blocks data

In Coda, a node by default will only store information about blocks it has seen (unless it is running as an archive node), so before running these queries leave your node running for a while before executing these queries to get some meaningful results.

According to the documentation, the blocks object accepts the before, last, after, first and filter arguments to specify which blocks to return. All of these are optional (nullable) as indicated by the lack of a trailing !. It returns a BlockConnection object (required) which itself is comprised of various fields that we can explore by clicking through on the explorer documentation.

blocks(
  before: String
  last: Int
  after: String
  first: Int
  filter: BlockFilterInput): BlockConnection!

First, let’s get the totalCount of blocks known to this node:

query {
  blocks {
    totalCount
  }
}

Which returns:

{
  "data": {
    "blocks": {
    "totalCount": 236
    }
  }
}

We can also filter the blocks to only contain blocks that contain a public key whose transaction is in a block or produced this block. Applying this to my staking node and getting the first result:

query {
  blocks(first: 1, filter: {relatedTo: "tNciEHZuC9SdpdKjG3cLw8dpdKUdXpbrqkhctcE422NyspG8G3ddArHJupGvV591mRKd6hEMBoPNPNRZMSjyN1S8Jp4jD8zMyHSzvsFhzKhqa7Q3F2a3uNmNUHp2rtVsj3HuLt9Zw3whHM"}) {
    totalCount
    nodes {
      creator
      stateHash
      transactions {
        coinbase
        userCommands {
          id
          nonce
          from
          to
          amount
          fee
        }
      }
    }
  }
}

This returns the following response. In this instance, there were no transactions in the block, but the public key produced the block. Note, I have specified the fields I am interested in returning, but there are many more available as demonstrated in the schema.

{
  "data": {
  "blocks": {
    "totalCount": 17,
    "nodes": [
      {
        "creator": "tNciEHZuC9SdpdKjG3cLw8dpdKUdXpbrqkhctcE422NyspG8G3ddArHJupGvV591mRKd6hEMBoPNPNRZMSjyN1S8Jp4jD8zMyHSzvsFhzKhqa7Q3F2a3uNmNUHp2rtVsj3HuLt9Zw3whHM",
        "stateHash": "4ApWKSeBhPyZDgYVFCYg6WzN1oGHpQxmdTfPbVs4LEd2JCxvwR3hzcVsjto45Kt58cpbciFxvt7ZoCq6HsM3iqZfpNh5p9DWXR8RHZFnVRczvPij7xeMYvBXcJ3kVzqkf9aufNS344VoR",
        "transactions": {
          "coinbase": "0",
          "userCommands": []
          }
      }
    ]
    }
  }
}

Mutations, modifying data with the Coda GraphQL API

So far, we have explored some basic queries using the GraphQL API. GraphQL supports two additional root types of mutations and subscriptions. We won’t touch on subscriptions in this article, but they allow GraphQL clients to subscribe to published events. Coda’s GraphQL API provides two events newSyncUpdate that triggers when the network sync status changes and newBlock that triggers when a new block is created.

Mutations, as the name suggests, modify data in some way. Coda’s GraphQL API includes the following mutations:

addWallet
deleteWallet
sendPayment
sendDelegation
addPaymentReceipt
setStaking

Let’s list any existing wallets known to the daemon:

query {
  ownedWallets{
    publicKey,
    balance {
      total
      unknown
    }
    nonce
  }
}

If you didn’t import an existing wallet, likely this will be empty. Create a new one:

mutation {
  addWallet{
    publicKey
  }
}

This will return the public key of your new wallet. Use this to request some Coda from the testnet faucet on Discord) or any other helpful Codans. Now retrying the ownedWallets query should return the wallet and balance associated.

{
  "data": {
  "ownedWallets": [
    {
    "publicKey": "tNciDhKWNmKpoQ4jHvP4oqt7gnuQYaUYz8pMSE1wewiXjaozksHVL5YpkC4bSx3hqaZHH6cgxwLicrXAdddGyHx8eh3WJBZN6XzpAZdfYM861t53khP9v74Kj3CBxHCbfVPisD8ikNaBGE",
    "balance": {
      "total": "470",
      "unknown": "470"
    },
    "nonce": "0"
    }
  ]
  }
}

Next, let’s send a payment specifying the from address as your wallet with funds in it and a to address (e.g. to the echo service) and the fee associated with the transaction, and we’ll return the full details of the transaction.

mutation {
  sendPayment(input: {fee: 5, amount: 10, to: "tNciEHZuC9SdpdKjG3cLw8dpdKUdXpbrqkhctcE422NyspG8G3ddArHJupGvV591mRKd6hEMBoPNPNRZMSjyN1S8Jp4jD8zMyHSzvsFhzKhqa7Q3F2a3uNmNUHp2rtVsj3HuLt9Zw3whHM", from: "tNciDhKWNmKpoQ4jHvP4oqt7gnuQYaUYz8pMSE1wewiXjaozksHVL5YpkC4bSx3hqaZHH6cgxwLicrXAdddGyHx8eh3WJBZN6XzpAZdfYM861t53khP9v74Kj3CBxHCbfVPisD8ikNaBGE"}) {
    payment {
      id
      isDelegation
      nonce
      from
      to
      amount
      fee
      memo
    }
  }
}

This produced the following response:

{
  "data": {
    "sendPayment": {
      "payment": {
        "id": "3XCgvAHM3twoJ2T8dcTvxpuyrnBJ12b7Y1HXYjAgVXVLHNPcr95VSaQ9eoGZTmq3LXh2rKvePPes58Zi4EJXjiFo77uo1meAMh3qty88RXmfrPjP1xQ58wm9oop499ZUgT5Fspbok7Yxg4eCFR8zRvd8bob7RdojY38DLDiHD9hrEmEXbZhAThZWS6sdxuHBpsimL4ZZJkvbQcs24MonG2Fd1MvbjNhfSE5sR7PGGjqAX4hbEb6hskjF8czVDnCqD3yLwGRFrL2tchCrZUHc6njci3294SsFMmbgAqwog6nBVQojE42sYgcXdPK7oPEFV2Com53QCSPPSfdiCeRNF5QhG5tEC1viwGtBLbKmF9qFR7PVg8UEJhBwsyu5LpLNwxzVEA4rdEy3syDZFrC4GfYmKdohQvBWv2DgVgUKwe7xRH5fh5UPx2f3GyV87pcxvdoq5BnZYDB6MkktsQxpeTk8zykrvXacHtYkZhMfLVLhgxW9EwEb6dfJTUMHaSp2F1uXFzwa6cJe8RdUt4EWQiVYgDUN8Hwai5AGLaXfn86Yjck3D6E5MiBDwhrcQBiKF6xFSzEFJGcnjTrFd5aMKE9KKqyyUuUjoj8YKz4woUXUBHbyaTBrJP2dVs8vaePyieueq5pFxtD9Lhqwx6f9ejUQnbz53HPzfsNzkDNA9n5NJimrgWDrZvkUmq8F2B3zj35Mzf4M9PFbjeWkUwAjRveRpBEy2oBWswX",
        "isDelegation": false,
        "nonce": 0,
        "from": "tNciDhKWNmKpoQ4jHvP4oqt7gnuQYaUYz8pMSE1wewiXjaozksHVL5YpkC4bSx3hqaZHH6cgxwLicrXAdddGyHx8eh3WJBZN6XzpAZdfYM861t53khP9v74Kj3CBxHCbfVPisD8ikNaBGE",
        "to": "tNciEHZuC9SdpdKjG3cLw8dpdKUdXpbrqkhctcE422NyspG8G3ddArHJupGvV591mRKd6hEMBoPNPNRZMSjyN1S8Jp4jD8zMyHSzvsFhzKhqa7Q3F2a3uNmNUHp2rtVsj3HuLt9Zw3whHM",
        "amount": "10",
        "fee": "5",
        "memo": "2pmu64f2x97tNiDXMycnLwBSECDKbX77MTXVWVsG8hcRFsedhXDWWq"
      }
    }
  }
}

The nonce value is used to prevent replaying transactions on the network. As this was our first transaction, the nonce was 0. If we repeat this transaction we’ll see the nonce has increased to 1.

{
  "data": {
    "sendPayment": {
      "payment": {
        "id": "3XCgvAHM3txTHrcrWcodUm1nkUVEQjUa8Q4WxkrwVueTvAvCM6NVehmxiVZuzMY3gV5cRanzk2A41vQp59sdGfAothQuoydrcW9s12NSgM9wqqgGtJidUq29b6qDmUh7N6RVkaJtynNuEtYgbeubWaYQfNvN3nruehRHnY6Xuk4uyuvqob4zGerMKS4Qpzq1nirb6GbWYCYouwz5fhZPmRtPuX3FDR5LtgSY1JqV8VrcHY7BRgb6qQJco83n1HcDZS1uQsxRTJEDzM4EvHCeg6qx7Wc6GeuDEDTMmkvZyP3jgvRXS3oLRhnXLDzR9EhoHu2pwLZGmwEBBYrSeoEY7sFeMTByRTbsaHZi69rJsrxBAHmpY8CY7WZypf9mxiz2aRfRffy5AGCEY74EsGpKbAYszBww6k5p6PUvnuk2PTnFdagdMeJ6Udm7qGDcRwA6FL9VixEazyMYtZSMNaFweyTo1zic5viusJfZiTAQZ6JnJ7R88uAbRiwZ7v5oweqLrkHi9gLcqF8ULwWTQwh84zW9ei8GGNneFHn4SY2c2AZK7NLPMtQhLXbtALG7CMN3V1u8NoFzCUCYx3omAP8qHiDCAjxPvjkPViYj9m3YeBEkukMzjW7BMQK52BxBKSbQC7xXbxm4x4dyCKW9BSkrvkz3a27ePbwouSESoHURDmSqGYUjd9uSaboVmkvdqpFdw43sd3dPqbEaXXDxvdgkGrUBXd24DzuriaF",
        "isDelegation": false,
        "nonce": 1,
        "from": "tNciDhKWNmKpoQ4jHvP4oqt7gnuQYaUYz8pMSE1wewiXjaozksHVL5YpkC4bSx3hqaZHH6cgxwLicrXAdddGyHx8eh3WJBZN6XzpAZdfYM861t53khP9v74Kj3CBxHCbfVPisD8ikNaBGE",
        "to": "tNciEHZuC9SdpdKjG3cLw8dpdKUdXpbrqkhctcE422NyspG8G3ddArHJupGvV591mRKd6hEMBoPNPNRZMSjyN1S8Jp4jD8zMyHSzvsFhzKhqa7Q3F2a3uNmNUHp2rtVsj3HuLt9Zw3whHM",
        "amount": "10",
        "fee": "5",
        "memo": "2pmu64f2x97tNiDXMycnLwBSECDKbX77MTXVWVsG8hcRFsedhXDWWq"
      }
    }
  }
}

Now finally, we’ll check the status of our transaction on the Coda network using the returned id of the payment.

query {
  transactionStatus(payment: "3XCgvAHM3txTHrcrWcodUm1nkUVEQjUa8Q4WxkrwVueTvAvCM6NVehmxiVZuzMY3gV5cRanzk2A41vQp59sdGfAothQuoydrcW9s12NSgM9wqqgGtJidUq29b6qDmUh7N6RVkaJtynNuEtYgbeubWaYQfNvN3nruehRHnY6Xuk4uyuvqob4zGerMKS4Qpzq1nirb6GbWYCYouwz5fhZPmRtPuX3FDR5LtgSY1JqV8VrcHY7BRgb6qQJco83n1HcDZS1uQsxRTJEDzM4EvHCeg6qx7Wc6GeuDEDTMmkvZyP3jgvRXS3oLRhnXLDzR9EhoHu2pwLZGmwEBBYrSeoEY7sFeMTByRTbsaHZi69rJsrxBAHmpY8CY7WZypf9mxiz2aRfRffy5AGCEY74EsGpKbAYszBww6k5p6PUvnuk2PTnFdagdMeJ6Udm7qGDcRwA6FL9VixEazyMYtZSMNaFweyTo1zic5viusJfZiTAQZ6JnJ7R88uAbRiwZ7v5oweqLrkHi9gLcqF8ULwWTQwh84zW9ei8GGNneFHn4SY2c2AZK7NLPMtQhLXbtALG7CMN3V1u8NoFzCUCYx3omAP8qHiDCAjxPvjkPViYj9m3YeBEkukMzjW7BMQK52BxBKSbQC7xXbxm4x4dyCKW9BSkrvkz3a27ePbwouSESoHURDmSqGYUjd9uSaboVmkvdqpFdw43sd3dPqbEaXXDxvdgkGrUBXd24DzuriaF")
}

This returns the transactionStatus which in our case is PENDING i.e. is waiting to be included in a block.

{
  "data": {
  "transactionStatus": "PENDING"
  }
}

Next steps

We’ve only touched on a few queries available via the Coda GraphQL API, and there are many more to explore. While the API is in active development, it’s a great learning experience to interact with the Coda protocol this way. It offers some improvements over the existing CLI interface but more importantly allows the development of future tools such as block explorers, wallets etc…

You can also interact with the GraphQL API using multiple client libraries such as Apollo, building a status page for your own node, for example. Additionally, the excellent Postman client now supports GraphQL, which can be used to further experiment with and develop for the Coda GraphQL API.

Zcash Insight Blockchain Explorer

Sun, 11 Aug 2019 12:00:00 +0000

With the launch of 2.0.7, zcashd now includes full support for the Insight blockchain explorer. The Insight explorer, which will be familiar to those who have used one of the many popular block explorers it powers, was developed by Bitpay as part of its bitcore suite of tools. The tool, released under a permissive open-source license (MIT), contains many development tools to “build bitcoin and blockchain applications” but we will concentrate on installing the explorer portion for this article which has been ported to support Zcash.

While there are existing implementations of the Insight explorer on Zcash it relies on running a patched version of zcashd. This caused issues with maintaining multiple versions which required regular updating as versions by default reach end of support halt and shut down every 16 weeks. In addition, the Blossom network upgrade will activate on testnet at height 584000 and so a compatible version of zcashd is required to follow the upgraded chain.

These updates to zcashd specifically include new fields to the getrawtransaction RPC call as well as a set of new RPC (CLI) methods that allow, for example, to get the balance of (transparent) addresses. While you can run zcashd in isolation to extract this information, the Insight explorer user interface presents the information in a visual and easily searchable way.

For the remainder of the article, we’ll walk through installing Insight explorer on an Ubuntu 18.04 machine, such as a local machine or VM (cloud-hosted or otherwise). I’ve tested these instructions successfully on a local machine as well as on a Digital Ocean cloud instance. As this is running zcashd in addition to other components, make sure you meet the minimum specs and allow some headroom in disk space to accommodate future growth of the chain (running zcashd in this manner will also increase the disk space used by the data directory).

The following installation is heavily based on the Dockerfile in the zcash-hackworks repository. Rather than assume Docker knowledge and detail additional requirements such as persistent data volumes (you’ll want to mount the zcash data directory and parameters), I’ll outline the process on Ubuntu 18.04 which can be easily adapted to a Dockerfile if necessary or deployed as is to a cloud VM.

Installation

For this tutorial, we will be running everything as the zcashd user. As some of the paths below are absolute, you will need to either create a new user of that name or simply update references to the zcashd user to your (non-root) username.

Installation consists of building zcashd (though this isn’t strictly required, as you could substitute the pre-built binaries), installing Node.js and pulling in the relevant Zcash ported bitcore packages and editing some configuration files.

To begin, we’ll start by pulling in the required dependencies for our zcashd build:

sudo apt update
sudo apt install -y gnupg2 wget libzmq3-dev git \
    build-essential pkg-config libc6-dev m4 g++-multilib \
    autoconf libtool ncurses-dev unzip git python \
    python-zmq zlib1g-dev curl bsdmainutils automake

Next, we’ll install Node.js. We’ll use version 8 as this has been confirmed to work and we’ll use the installation tool from nodesource.com. This script involves piping to bash with the sudo command so if you don’t want to proceed with this method simply follow the instructions for a manual installation.

curl -sL https://deb.nodesource.com/setup_8.x | sudo bash -
sudo apt-get install -y nodejs

To get around some node permissions issues on this version of Node.js we’ll use the following configuration:

mkdir -p /home/zcashd/.npm-global
export NPM_CONFIG_PREFIX=/home/zcashd/.npm-global

Finally, add the installation directory to our path so we can simplify references to the Node.js libraries later in the installation and apply these changes to the current shell (note these values won’t persist so if you don’t complete the installation in a single terminal you’ll need to add these environment values again or persist them e.g. using bashrc).

export PATH=/home/zcashd/.npm-global/bin:$PATH
source ~/.profile

Next, we’ll clone the zcash repo and checkout version 2.0.7. If version 2.0.7 isn’t available at the time of reading then install the release candidate v2.0.7-rc1. Update this version tag if a later version is available or remove this line entirely to build the current master.

git clone https://github.com/zcash/zcash.git
cd zcash
git checkout v2.0.7 # ignore the detached head message

Start the build and wait for it to complete:

./zcutil/build.sh -j$(nproc)

Next, we’ll download the Zcash proving and verifying keys which will be placed in ~/.zcash-params. These keys are just under 1.7GB.

./zcutil/fetch-params.sh

Now we’ll create a zcash conf file to complete the zcashd setup. Create a file using your favourite editor named zcash.conf in the ~/zcash directory (the following instructions use the nano editor).

mkdir -p ~/.zcash
cd ~/.zcash
nano zcash.conf

Use the following values to run on testnet (see this for mainnet configuration).

testnet=1
addnode=testnet.z.cash
server=1
whitelist=127.0.0.1
zmqpubrawtx=tcp://127.0.0.1:28332
zmqpubhashblock=tcp://127.0.0.1:28332
txindex=1
experimentalfeatures=1
insightexplorer=1
addressindex=1
timestampindex=1
spentindex=1
rpcport=18232
rpcallowip=127.0.0.1
rpcuser=<username>
rpcpassword=<secretpassword>
showmetrics=0
uacomment=bitcore

You can confirm that everything is working with the command ./src/zcashd —version which should output the version information you installed.

./src/zcashd --version
Zcash Daemon version v2.0.7

Now install npm and the bitcore-node-zcash library.

cd ~
sudo npm install -g npm@latest
npm install -g zcash-hackworks/bitcore-node-zcash
bitcore-node create zc

You should see the output like the following: Successfully created node in directory: zc. This final command creates a configuration file named bitcore-node.json in the zc directory that we will need to edit to match our installation.

cd zc
nano bitcore-node.json

Update this configuration to match the below, making a note of the port and also update the paths of the datadir and zcashd executable exec if you installed as a different user.

{
  "network": "testnet",
  "port": 3001,
  "services": [
    "bitcoind",
    "web"
  ],
  "servicesConfig": {
    "bitcoind": {
      "spawn": {
        "datadir": "/home/zcashd/.zcash",
        "exec": "/home/zcashd/zcash/src/zcashd"
      }
    },
    "insight-api-zcash": {
      "routePrefix": "api"
    },
    "insight-ui-zcash": {
      "apiPrefix": "api",
      "routePrefix": ""
    }
  }
}

Then install the Insight component from this zc directory.

bitcore-node install zcash-hackworks/insight-api-zcash \
	zcash-hackworks/insight-ui-zcash

If all works, you should see Successfully added services(s): zcash-hackworks/insight-api-zcash, zcash-hackworks/insight-ui-zcash.

Finally we can start with the following command:

/home/zcashd/zc/node_modules/bitcore-node-zcash/bin/bitcore-node start

If all works after a few seconds you’ll see output similar to the below indicating that the node is syncing and you can browse to your IP and port listed above e.g. localhost:3001 and, while the explorer will not be functional until the sync completes, it will display the progress.

/home/zcashd/zc/node_modules/bitcore-node-zcash/bin/bitcore-node start
[2019-08-11T01:27:58.376Z] info: Using config: /home/zcashd/zc/bitcore-node.json
[2019-08-11T01:27:58.377Z] info: Using network: testnet
[2019-08-11T01:28:03.956Z] info: ZMQ connected to: tcp://127.0.0.1:28332
[2019-08-11T01:28:18.496Z] info: Zcash Height: 133 Percentage: 0.01
[2019-08-11T01:28:33.500Z] info: Zcash Height: 241 Percentage: 0.02

As we started this in the foreground, closing your terminal without detaching it will kill the process. Therefore it is recommended you set it up to run as a service or at a minimum detach the process from the terminal with an application such as screen or nohup.

nohup /home/zcashd/zc/node_modules/bitcore-node-zcash/bin/bitcore-node start > explorer.log 2>&1 &

The above example redirects the output to a file named explorer.log so to follow the output logs use tail -f explorer.log.

Running on mainnet

If you wish to run on mainnet (or livenet in bitcore parlance), you’ll need to update zcash.conf to the below and optionally updating the RPC port.

testnet=0 
addnode=mainnet.z.cash

Finally update var testnet = true; in /home/zcashd/zc/node_modules/insight-ui-zcash/public/src/js/app.js and stop and restart the process.

Making the explorer publically available

If you are running a local instance, you can access your explorer via localhost:3001, and wait for your node to fully sync for the explorer to function currectly. If on a public IP you can also get access via <your_ip_address:3001> if your firewall permits access to that port.

Let’s quickly add to our build to use NGINX to map port 443 to 3001 as a reverse proxy and add a free Let’s Encrypt certificate. You’ll need a domain name that has its DNS pointed to the instance IP such as explorer.mydomainname.com.

sudo apt install nginx
sudo systemctl start nginx
sudo systemctl enable nginx # optionally enable NGINX as a service

At which point you should be able to browse to <your_ip_address> and see the default NGINX screen. Next, follow the instructions to add the Certbot PPA and install Certbot:

sudo apt-get -y install certbot python-certbot-nginx
sudo certbot --nginx

Follow the command prompts, enter your domain name for the certificate request, e.g. explorer.mydomainname.com and choose the option (2) to redirect all traffic to HTTPS. Then edit the default site to pass all traffic through to port 3001.

cd /etc/nginx/sites-available
sudo nano default

Update the location of the server block for your domain name to match the below:

location / {
                proxy_pass http://localhost:3001/;
}

Finally, reload NGINX, and your site should be accessible on your domain name and served over HTTPS. While this will work for a low traffic hosted explorer, it would not be recommended for a production instance and never use the instance as a wallet to store actual funds.

sudo nginx -s reload

Examining Transactions

Zcash is best known for its shielded transactions in which transaction details are not visible on the blockchain. Let’s explore this by examining a fully shielded transaction such as this transaction via the Insight explorer.

As we can see, the only detail available for this fully shielded Sapling transaction is the number of shielded inputs (similar to UTXOs in Bitcoin but known as notes in Zcash) and the number of shielded outputs.

All of this data is also available on the zcashd instance and we can examine the raw transaction data using the following commands:

~/zcash/src/zcash-cli getrawtransaction 5328354a0b14c95b1f8ac26ec99c7b365638cd877a04aa3fb27d19deedc90fb7

This will output the transaction in hex format. You can decode this by passing the output into the following command:

~/zcash/src/zcash-cli decoderawtransaction <output of previous command>

This will output the full transaction detail that you can compare to the explorer output. There are also additional RPC methods such as to get the address balance, which were not previously available and allow for additional functionality.

./zcash/src/zcash-cli getaddressbalance '"tmYXBYJj1K7vhejSec5osXK2QsGa5MTisUQ"'