This is a monthly report on the progress of 🐲 Hydra and 🛡 Mithril projects since October 2024. This document serves as a preparation for and a written summary of the monthly stakeholder review meeting, which is announced on our Discord channels and held on Google Meet. This month, the meeting was held on 2024-12-09 using these slides and you can see the recording here.

Mithril​

Issues and pull requests closed in November

Roadmap​

Here’s the latest on our roadmap:

• Cardano stake distribution certification #955: the feature is pending activation on mainnet
• Protocol usage metrics/statistics #2028: the feature is completed and has been released in November
• Explore signer registration solutions #2029: we are exploring the signer registration options and preparing a document summarizing them
• Cardano database incremental snapshots PoC #2047: we have started working on a proof of concept for incremental snapshots of the Cardano database.

Distributions​

We released Mithril distribution 2445.0, which includes:

• 🔥 Breaking changes in the Mithril client library, CLI, and WASM with the removal of deprecated beacon in Mithril certificates
• Stable support for Cardano node 10.1.2 in the signer and the aggregator
• Stable support for Cardano stake distribution client library, CLI, and WASM
• Stable support for the Prometheus metrics endpoint in the aggregator
• Bug fixes and performance improvements.

Future distributions​

We plan to release new distributions in December:

• 2450:
  • Availability of NodeJS and bundler targets in the Mithril client WASM npm package
  • Stable support for Cardano node 10.1.3 in the signer and the aggregator
  • Execution rights of pre-built binaries in GitHub releases.

Dev blog​

We have published the following posts:

• One line installer for Mithril binaries
• New Protocol Insights Dashboard released
• Mithril aggregator Prometheus endpoint is available.

Protocol Insights Dashboard v2​

We have created a new version of the Mithril Protocol Insights Dashboard, which is now available here. It provides a comprehensive view of the Mithril network and its performance metrics.

The Protocol Insights Dashboard is a valuable tool for monitoring the network and understanding its behavior:

• Participation metrics
• Usage metrics
• Health metrics
• Artifacts metrics
• Software metrics.

One line installer for Mithril binaries​

To facilitate the installation and updating of Mithril binaries, we have created a one line installer that downloads and installs the Mithril binaries and supports Linux and macOS. It is available for the Mithril signer, Mithril aggregator, and Mithril CLI.

Here is an example command to download the latest Mithril signer binary in the current directory:

curl --proto '=https' --tlsv1.2 -sSf https://raw.githubusercontent.com/input-output-hk/mithril/refs/heads/main/mithril-install.sh | sh -s -- -c mithril-signer -d latest -p $(pwd)

This will download the latest signer binary for the correct platform and architecture (if available) in the current directory:

Fetching release information from https://api.github.com/repos/input-output-hk/mithril/releases/latest...
Downloading mithril-signer to latest from https://github.com/input-output-hk/mithril/releases/download/2445.0/mithril-2445.0-linux-x64.tar.gz...
Congrats! mithril-signer has been upgraded to 0.2.209+67dc6e4 from distribution latest and installed at ./home/user!

Client NPM package compatible with NodeJS​

We have created a new version of the npm package for the Mithril client WASM, which is now compatible with NodeJS and bundler targets. This feature was requested by the community and will be released in the next 2550 distribution.

New explorer status​

We have enhanced the Mithril Explorer with a new status section that provides some key metrics and insights about the Mithril network. This new section is currently available on testing networks and will be generally available with the release of the next 2550 distribution.

Nightly and backward compatibility workflows​

We have implemented nightly and backward compatibility workflows to ensure more reliable and consistent releases. The nightly workflow builds Mithril Docker images every night and runs the backward compatibility workflow, which checks the compatibility of the upcoming Mithril distribution with previous versions, summarizing the changes.

Hydra​

We continued working on incremental commits, began supporting multiple versions in the Hydra Explorer, and started moving the hydra-explorer out of the monorepo.

Issues and pull requests closed in November

Notable updates on our roadmap this month include:

• Progress on incremental commits #199
• Hydra + Midnight support #1727
• Continued support of Hydra Doom, especially around the tournament!

Hydra Explorer​

The hydra-explorer has been moved to its own repository with independent deployment. As a result, supporting multiple versions of the hydra-node in the explorer is now necessary, and we are currently working on it.

Hydra Doom​

Following the success of Hydra Doom at Rare Evo and as an activation at various events organized by Input | Output (IO) and others(!), the Hydra Doom project has taken a significant step forward with the launch of a tournament offering 100,000 USDM. The prize is intended to be distributed automatically to the winners based on the outcome of the game as measured in the Hydra heads used for the finale.

In launching the tournament, we reached several milestones and pushed Hydra to its very limits in many ways. This included testing the performance of individual Hydra heads, which supported a multi-player game where every frame of each participant’s game session submitted smart contract transactions. At its peak (on December 3), the system processed 1.04 million transactions per second. Over a 24-hour period, it handled more than 15.5 billion transactions in total. This was achieved with a mix of real player traffic from around the world and artificial load generated by bots running the game, contributing transactions to more than 14,000 Hydra Heads.

While more work remains, this marks an impressive milestone, and the overall reception has been positive.

https://github.com/user-attachments/assets/c9546d61-212b-490f-88b8-a3b15229aa15

Incremental commits​

The incremental commits feature is nearly ready for release. As a final step, we invite community members to test-drive this feature. The goal is to ensure the best user experience and identify potential bugs. The remaining tasks include improving documentation and testing and updating the protocol specification to reflect these changes.

Hydra + Midnight​

We’ve been collaborating with the Midnight team to explore how Hydra can align with their goals. As part of this effort, we’re developing a version of Hydra with a custom ledger to support arbitrary built-in operations that validators may need. This work is ongoing.

Links​

The monthly review meeting for November 2024 was held on 2024-12-09 via Google Meet, presenting these slides and this recording.

This is a monthly report on the progress of 🐲 Hydra and 🛡 Mithril projects since September 2024. This document serves as a preparation for and a written summary of the monthly stakeholder review meeting, which is announced on our Discord channels and held on Google Meet. This month, the meeting was held on 2024-10-23 using these slides and you can see the recording here.

Mithril​

Issues and pull requests closed in October

Roadmap​

Here’s the latest on our roadmap:

• Cardano transactions signature/proving MVP #1457: the feature has been activated on mainnet
• Cardano stake distribution certification #955: the feature has been activated on preview, pre-production and is pending activation on mainnet
• Decentralization of signature orchestration #1777: the feature has been activated on preview, pre-production and mainnet
• CIP for Mithril signature diffusion through the Cardano network #1775: the draft CIP has received the number CIP-0137 and is in final review stage
• Protocol usage metrics/statistics #2028: the feature is being implemented and will be released in November.

Distributions​

We released Mithril distribution 2442.0, which includes:

• Stable support for decentralized signature orchestration
• Stable support for Cardano transaction client library, CLI, and WASM
• Stable support for new Pythagoras Mithril era
• Bug fixes and performance improvements.

Future distributions​

We plan to release new distributions in November:

• 2444:
  • Stable clients for Cardano stake distribution certification
  • Stable support for Prometheus metrics endpoint in the aggregator.

Dev blog​

We have published the following posts:

• Certification of Cardano stake distribution
• Certification of Cardano transactions (updated).

Protocol status​

Decentralized signature orchestration​

To enable multiple aggregators on the same Mithril network, the signature orchestration must be decentralized to run independently across all signers and aggregators.

We have implemented and deployed this feature onmainnet with the signer released in 2442.0 distribution:

• The signer and aggregator nodes can independently compute the beacon, which determines the messages to sign and certify
• The aggregator no longer advertises the pending certificates; the pending certificate is deprecated and temporarily kept alive for legacy signer nodes until sufficient adoption of the new version is reached
• The aggregator buffers the individual signatures received from signers until it has computed the associated beacon and will try to aggregate them thereafter.

Aggregator usage metrics and Grafana dashboard​

We have been working on a new Prometheus endpoint for the aggregator, which provides detailed insights about the production of artifacts and certificates, the events received from the signers, and the artifacts and proofs served to the clients. The feature can be easily activated with some configuration parameters.

Additionally, we created a Grafana template to easily set up a dashboard for this Prometheus endpoint.

Hydra​

Issues and pull requests closed in October

Notable updates on our roadmap this month include:

• Added the raw CBOR datum in transaction outputs #1543
• Used Aiken for commit validator #1680
• Updated to cardano-api 9.4 #1706
• Implemented the off-chain user journey for incremental commits #1522 and made good progress on the on-chain validators
• Added Blockfrost mode to the hydra-chain-observer #1631.

Argentina​

IOG attended a local satellite Cardano Summit event, as well as Tech Expo – a multichain Web3-focused event – in Argentina on October 18 and 19. During both events, the Hydra Doom demo was showcased on-site, using locally built cabinets and a new setup with a computer as an on-site server, connecting laptops to aggregate stats from multiple sessions. Many attendees had the chance to play and enjoy the demo, with our very own Tamara Haasen displaying true grit and remarkable skill! Notable attendees included ICP, Polkadot, OKX, ByBit, and the founder of Render Network. Overall, it was a fantastic event and a great opportunity to connect with Argentinian builders. Now it’s time to prepare for the next leg of the Hydra Doom adventure and gear up for a return visit soon!

Aiken for commit validator​

We began porting our validators to Aiken, starting with the Commit validator. This resulted in a saving of over 1Kb to the validator script and increased the maximum number of head participants from 6 to 9.

SnapshotConfirmed has the full Tx​

The SnapshotConfirmed event now has the full transaction information, which makes it much easier to build an app that watches for confirmed transactions and responds appropriately. For example, here is the change required in Hydraw:

diff --git a/hydraw/static/bundle.js b/hydraw/static/bundle.js
index 1bea9e96fe7..326c69c39bc 100644
--- a/hydraw/static/bundle.js
+++ b/hydraw/static/bundle.js
@@ -13,19 +13,20 @@ let n = 0
 client.addEventListener("message", e => {
   const msg = JSON.parse(e.data);
   switch (msg.tag) {
-    case "TxValid":
-      // TODO: Should only draw pixels on SnapshotConfirmed
-      const cborHex = msg.transaction.cborHex;
-      console.log("New transaction cborHex seen", cborHex);
-      const transaction = cbor.decodeFirstSync(cborHex);
-      const auxiliaryData = transaction[3]
-      if (auxiliaryData !== undefined && auxiliaryData !== null) {
-        console.log("Transaction has auxiliary data", auxiliaryData);
-        const aux = auxiliaryData.value;
-        const [x, y, r, g, b] = (aux.get(0) || aux.get(1)).get(metadataLabel);
-        n += delay;
-        setTimeout(() => drawPixel(x, y, [r, g, b]), n);
-      }
+    case "SnapshotConfirmed":
+      msg.snapshot.confirmed.forEach( (tx, _) => {
+        const cborHex = tx.cborHex;
+        console.log("New confirmed transaction cborHex seen", cborHex);
+        const transaction = cbor.decodeFirstSync(cborHex);
+        const auxiliaryData = transaction[3]
+        if (auxiliaryData !== undefined && auxiliaryData !== null) {
+          console.log("Transaction has auxiliary data", auxiliaryData);
+          const aux = auxiliaryData.value;
+          const [x, y, r, g, b] = (aux.get(0) || aux.get(1)).get(metadataLabel);
+          n += delay;
+          setTimeout(() => drawPixel(x, y, [r, g, b]), n);
+        }
+      })
     default:
       console.log("Irrelevant message", msg);
   }

Links​

The monthly review meeting for October 2024 was held on 2024-10-23 via Google Meet, presenting these slides and this recording.

This is a monthly report on the progress of 🐲 Hydra and 🛡 Mithril projects since August 2024. This document serves as a preparation for and a written summary of the monthly stakeholder review meeting, which is announced on our Discord channels and held on Google Meet. This month, the meeting was held on 2024-09-25 using these slides and you can see the recording here.

Mithril​

Issues and pull requests closed in September

Roadmap​

Here’s the latest on our roadmap:

• Cardano transactions signature/proving MVP #1457: the feature is pending activation on mainnet
• Cardano stake distribution certification #955: the feature is pending activation on preview, preprod and mainnet
• Decentralization of signature orchestration #1777: feature implementation is being finalized
• CIP for Mithril signature diffusion through the Cardano network #1775: the draft CIP is in review.

Distributions​

We released Mithril distribution 2437.1, which includes:

• Breaking changes in the Mithril client WASM:
  • Seamless transition from unstable to stable features
  • A new unstable option in the client allows using unstable features
  • The previous client.unstable implementation is not supported anymore and must be replaced with client
• Stable support for Cardano transaction certification in signer and aggregator
• Stable support for Cardano stake distribution certification in signer and aggregator
• Bug fixes and performance improvements.

Future distributions​

We plan to release new distributions in October:

• 2440:
  • Stable clients for Cardano transaction certification
  • Activation of the certification of Cardano stake distribution in pre-release-preview, release-preprod, and release-mainnet
  • Stable support for new Pythagoras Mithril era
• 2443:
  • Stable clients for Cardano stake distribution certification.

Dev blog​

We have published the following posts:

• Mithril client WASM breaking change
• Certification of Cardano transactions (Updated).

Protocol status​

Decentralized message queue for Cardano​

We continued working on the decentralized message queue (DMQ) for Cardano, as proposed in this CIP draft.

The DMQ is designed to:

• Leverage the Cardano network layer
• Initially be used by Mithril for the diffusion of signatures from signers to aggregators
• Be adaptable for future Cardano protocols, each operating on a different DMQ topic.

The DMQ will be operated by a 'side' node, known as the DMQ node, which will run a separate peer-to-peer network powered by the Ouroboros network stack. Here are some key technical details:

• It will run as an external process that exposes a Unix socket to support node-to-client mini-protocols
• It will implement node-to-client and node-to-node mini-protocols to allow for message local submission, local notification, and peer-to-peer submission
• It will retrieve the Cardano stake distribution from its local Cardano node to authenticate incoming messages
• It will retrieve other peers' information about the peer-to-peer network from its local Cardano node (this topic is being investigated: either with a ledger hard fork, redirecting connections to the DMQ node, or leveraging the SRV record in the SPOs' DNS)
• Each topic of the message queue will run a different DMQ node instantiated with a specific configuration.

Producers and consumers exist for the DMQ topic running on some DMQ nodes. In the case of Mithril:

• The Mithril signers will be the producers of the message (Mithril individual signatures)
• The Mithril aggregators will be the consumers of the message (Mithril individual signatures).

DMQ producers and consumers run on various DMQ nodes:

Hydra​

Issues and pull requests closed in September

Notable updates on our roadmap this month include:

• Released version 0.19.0 introducing Conway ledger support with compatibility for Babbage transactions #1608
• Completed the Hydra 'head-in-head' spike #1590
• Investigated Raft consensus for networking #1591
• Added 'HeadId' into the 'Greetings' message #1557
• Implemented the initial suite of network-resilience tests #1532
• Changed network semantics to broadcast to everyone #1624.

Spike: head-in-head​

When a Hydra head is opened, part of the underlying ledger state gets locked and made available off-chain to a small group of participants. While this already creates a 'small world' to process transactions (which makes it fast and cheap), we encountered use cases where it would make sense to open additional heads with different or even smaller groups of participants on parts of the layer 2 state, forming layer 3 heads.

One example of this use case is the Hydra Doom demonstration we presented at Rare Evo last month. For example, individual multiplayer game sessions require low network latency from regionally close machines. If those machines run a game session head that, in turn, locks funds and game state from a slower global tournament head instance, we are discussing Hydra heads in heads.

This construction raises many open questions about liveness: what happens to the layer 3 head if the layer 2 head stops processing transactions? Such pessimistic scenarios can occur in any optimistic protocol and must be addressed, though not necessarily optimized for. Despite these and other uncertainties, we spent time this month exploring whether it’s even possible as part of the spike issue #1590.

Technically, this means that a hydra-node communicates with another hydra-node as its 'chain backend', interpreting snapshots of the layer 2 head as blocks, observing transactions that open or close the head, and submitting its state transition transactions to the underlying layer 2 ledger. The spike issue provides more details and instructions on reproducing these findings. The following issues demonstrate this prototype --inception mode:

Hydra 🤝 Blockfrost: chain backend concept​

As outlined in #1305, we are considering an option to run the hydra-node in a more lightweight mode, without requiring the full cardano-node. This feature is particularly relevant when considering the use of Hydra with the pay-per-use Blockfrost API.

To complete this component, the Blockfrost chain layer must be capable of:

• Following the chain
• Submitting Hydra transactions
• Handling relevant internal wallet queries for the hydra-node.

As an initial step, we have developed:

• A Hydra chain observer that operates in Blockfrost mode (although it has not yet been integrated into the hydra-node)
• A variant of the hydra-explorer tailored to the Blockfrost-enabled Hydra chain observer.

To achieve this, we have used a straightforward roll-forward approach via the Blockfrost HTTP API, relying on the following key API calls:

• GET /blocks/{hash}
  • confirmations: number of block confirmations
  • next_block_hash: (nullable) hash of the next block
• GET /blocks/{hash}/txs
• GET /txs/{hash}/cbor

Basically, we continuously fetch specific blocks by their hash, using the number of confirmations to indicate the block's safety (minimizing the likelihood of a rollback). From there, we roll forward using the reference to the next block hash.

While this mechanism might appear simplistic, it is highly effective for most use cases except scenarios involving exchanges.

To further optimize the performance of this new chain component, we’ve raised two new issues on the Blockfrost side to support our work:

• #209: add an endpoint to fetch all transaction CBORs in a block, reducing the number of API calls needed to retrieve Hydra head observations from block transactions
• #67: implement concurrent fetching to fully leverage parallel requests and enhance performance.

Moving forward, our next objective is to enable the hydra-node to publish Hydra scripts via Blockfrost, as outlined in #1668.

Spike: Raft-based networking​

Last month, we developed a new test suite #1532 to assess the resilience of our network stack. Now that this is in place, we can start exploring different approaches to achieve our goal of a crash-tolerant network layer.

Recently, we stumbled upon this fairly old research paper that explored various consensus protocols used in blockchain space. It reminded us of the correspondence between consensus and broadcasts:

The form of consensus relevant to blockchain is technically known as atomic broadcast

Additionally, it mentioned at least one early permissioned blockchain that achieved crash tolerance of t < n/2 by using etcd with its off-the-shelf Raft consensus algorithm. This prompted us to explore the possibility of replacing our custom network stack with this (arguably overkill) alternative in an experiment #1591 to evaluate its performance.

The GitHub issue contains all the details, but it turns out that this idea might not be as exotic as we initially thought! Implementing broadcast functionality as put requests to the etcd cluster works well when combined with an outbound, persisted buffer that handles failed writes while disconnected (or only connected to a minority). Additionally, the revision mechanism of etcd and storing the last known revision on disk allow us to create a hydra-node that is fully resilient to crashes and network failures, as demonstrated in these fault injection test runs.

Even the performance of this early prototype matches or exceeds our current implementation, especially when multi-threading performance is available. For example, using a low-powered container hosted on GitHub, we observed average confirmation times on the three-node benchmark decrease from 20ms to 100ms, while on a desktop machine with 8+ cores, the same benchmark improved from 100ms to 50ms. Since the starting numbers of the released version also vary widely between machines, these results are only indicative.

Incremental commits​

The incremental commits feature is being developed, and we demonstrated the envisioned user workflow during the monthly meeting.

Notably, users will first need to lock their UTXO in the Hydra deposit script. Once we have a signed snapshot on layer 2, the increment transaction will consume this output. Eventually, this will make the specified UTXO part of the head UTXO state on layer 2. In case of any issues, any Hydra node will be able to post a recovery transaction to unlock the UTXO.

The hydra-node API includes endpoints for depositing and recovering UTXOs, providing a convenient way to build and post these transactions. We aimed to create a user-friendly experience for these actions.

Next steps involve implementing comprehensive on-chain security, along with documentation and tutorials, to ensure our users understand how to use this new feature. This is essential since many builders on Hydra have requested this feature.

Links​

The monthly review meeting for September 2024 was held on 2024-09-25 via Google Meet, presenting these slides and this recording.

This is a monthly report on the progress of the 🐲 Hydra and 🛡 Mithril projects since July 2024. This document serves as both preparation for and a written summary of the monthly stakeholder review meeting, announced on our Discord channels and held on Google Meet. This month, the meeting took place on 2024-08-28, using these slides and you can see the recording here.

Mithril​

Issues and pull requests closed in August

Roadmap​

Here is an update on our current roadmap:

• Cardano transactions signature/proving MVP #1457: the feature is finalized and the last step to completion is to release on the mainnet.
• Cardano Stake Distribution certification #955: the feature is finalized and the last step is to release on preview, preprod and mainnet.
• Mithril signature diffusion with Cardano network layer PoC #1837: the proof-of-concept is completed.
• Decentralization of signature orchestration #1777: we have started implementing the feature with some preliminary works.
• CIP for Mithril signature diffusion through Cardano network #1775: the draft CIP has been published on the Cardano foundation CIPs repository and is under review.

Distributions​

We have released the new Mithril distribution 2430.0. This distribution includes several critical updates and enhancements:

• Support for Cardano node 9.1.0
• Support for Cardano transactions certification in release-preprod and pre-release-preview
• Bug fixes and performance improvements.

Future distributions​

We plan to release new distributions in September following the Chang hard fork:

• 2436:
  • Activation of the certification of Cardano transactions in the release-mainnet network
  • Activation of the certification of Cardano stake distribution in pre-release-preview, release-preprod, and release-mainnet.

Protocol status​

Cardano stake distribution certification​

We have completed the implementation of the Cardano stake distribution certification with Mithril and are currently preparing for the rollout of this feature to production:

• We have implemented a new type of data certified for the latest snapshot of the Cardano stake distribution done by the Cardano node at the end of each epoch.
• Under the hood, we use a Merkle tree representation of the stake distribution and sign its Merkle root.
• The Mithril clients (library, CLI and WASM) have the capability to download and verify a snapshot of the Cardano stake distribution.
• We are preparing to activate the feature in the release-preview, release-preprod, and release-mainnet networks with the release of the next Mithril distribution.
• We have created documentation and some example code implementation for developers.

Hydra​

Issues and pull requests closed in August

Notable updates on our roadmap this month include:

• Incremental decommits, Conway support fixes, and more were released in 0.18.0
• New landing page, SSL support, and bug fixes around transaction pruning were released in 0.18.1
• Incremental commits (coming in a 0.19 release) have pivoted to a deposit-based scheme based on open discussions (thanks @GeorgeFlerovsky!)
• Ready to shift ledger to Conway when hardfork is complete #1338
• Settled on Partial Fanout as a good solution to a few known problems
• Initial work on a suite of network-resiliance tests #1552
• The Hydra Doom demo yielded several ideas that need to be refined into features

Hydra Doom​

At the Rare Evo event in Las Vegas, we showcased the Hydra Doom demo, a technology demonstration of Hydra where users could play the 1993 id software game Doom and for each frame of the game, a smart contract transaction was submitted to a Hydra head which recorded game state, user inputs, and validated game state transitions (as a proof of concept, using only some minor logic pertaining to the movement of the player and dying in-game).

People could connect remotely via https://doom.hydra.family or in person on two custom-made arcade cabinets we had constructed for the occasion. Each game session contributed 35 transactions per second. In aggregate, we processed 106 million transactions with Hydra during the 78 hours of operation, which is more than the entire history of Cardano mainnet. The peak was around 7,200 tps.

The demo was very positively received on social media and seems to have rekindled the flames around the Hydra narrative both internally and externally, which hopefully bodes well for both future adoption interest among builders, as well as the prospects of securing funding for the continued development of Hydra from Intersect.

Network tests with fault injection​

We have set up a simple yet effective workflow capable of simulating a Hydra cluster under high transaction loads and packet loss network failures.

With this, we aim to consistently reproduce and resolve issues that are causing the head to become stuck, as outlined in #1436.

The workflow specifies a set of high-load scenarios, each of which spins up the same Hydra cluster on devnet used during the getting started tutorial and injects network faults using Pumba.

The idea is to allow and encourage everyone to experiment with this workflow and collect summary results, which contain details about the performance within the head. Because these tests are performed with the hydra-node being a 'black box,' it also allows us to explore alternative network stacks as a whole.

New landing page​

For the Rare Evo event with the public Hydra doom demo (see above), we also figured it would be a good idea to improve the first impression new visitors get from the https://hydra.family website.

The marketing team at IOG collaborated on creating new content and design for the landing page. In addition to highlighting the key features and explaining the importance of the Hydra Head protocol, the updated landing page also includes an analogy of how Hydra works, using a cargo plane as an example.

The original design included an alternative color scheme using Teal as the primary accent color and Plum as the secondary. An example is included below. Do you think a new color scheme would be appealing or do you prefer the Purple main color as-is? Shoot us a message on Twitter (X) or #ask-hydra on Discord with your thoughts.

Bug fix: layer 2 transactions during decommits​

After releasing the decommit feature that enables users to transfer funds from layer 2 to layer 1 while the head is operational, we quickly identified a bug. When a decommit is pending ( funds have not yet been withdrawn from the head) and a user attempts to create a new layer 2 transaction, the hydra-node attempts to re-apply the pending decommit to the local ledger state, resulting in an error.

We promptly realized that any pending decommit needs to match with the decommit in the new snapshot and should be preserved in the next snapshot(s) until it is observed. Without this fix, our decommit feature would be extremely fragile, so we made sure to quickly draft a failing test and do a proper fix.

This is a pull request that fixes this bug and we also demonstrated this fix in one of our monthly meetings.

Conclusion​

The monthly review meeting for August 2024 was held on 2024-08-28 via Google Meet, presenting these slides and this recording.

This month we saw demonstrations from both projects about new features and improved testing capabilities. Optimization results and test results underline the teams' commitment to evidence-based engineering. The Hydra Doom demo was a great success and we received good feedback from users and use cases.

While roadmaps have not moved much over the last couple of months, but more re-orientation is likely as plans for 2025 are coming together in the working groups and other Intersect committees.

This is a monthly report on the progress of 🐲 Hydra and 🛡 Mithril projects since June 2024. This document serves as a preparation for and a written summary of the monthly stakeholder review meeting, which is announced on our Discord channels and held on Google Meet. This month, the meeting was held on 2024-07-22 using these slides and you can see the recording here.

Mithril​

Issues and pull requests closed in July

Distributions​

2423.0​

We have released the new Mithril distribution 2428.0. This distribution includes several critical updates and enhancements:

• Support for Cardano node 9.0.0.
• Database cleanup and optimization once per epoch.
• Bug fixes and performance improvements.

Future distributions​

We have planned to release two new distributions in August:

• 2430 (before the Chang hardfork):
  • Activation of the certification of Cardano transactions in the release-preprod network.
  • Support for Cardano node 9.1.0 on all networks.
• 2434 (after the Chang hardfork):
  • Activation of the certification of Cardano transactions in the release-mainnet network.

Protocol status​

Here is the status of the Mithril protocol on the Cardano mainnet:

Transaction verification in Nami​

As Blockfrost now supports the Mithril endpoints within their API, they have presented an implementation example.

• The demo shows transaction verification within the Nami light wallet (using Blockfrost as its backend).
• This allows the light wallet user to verify that their transaction is on-chain with the high certainty provided by Mithril security.
• While it is just an experimental implementation, it might be accepted by the Nami upstream in the future.

Transaction certification​

We have continued working on the implementation of the Cardano transaction certification with Mithril and are currently preparing for the rollout of this feature to production:

• We are finalizing the minimization of the signer footprint on the SPO infrastructure.
• We are preparing for activating the feature in the release-preprod network with the following release.
• We are preparing thorough documentation about the mechanism used to certify the transactions
• We have selected final parameters for signing the transactions:
  • 100 blocks from the tip of the chain (~30 minutes).
  • 30 blocks between snapshots certifying the transactions (~10 minutes).
  • With these parameters, we can consider the transactions final with very high probability.
  • These parameters are hot parameters that can be adjusted without requiring a new distribution

Decentralized Message Queue CIP​

We have been collaborating with the Cardano networking team to design a mechanism for implementing a decentralized message queue to decentralize the signature diffusion from Mithril signers to Mithril aggregators based on the Cardano network layer. The idea is to create a separate process for the decentralized message queue implementation:

• A new dedicated process (aka "Mithril network node") based on the Ouroboros network stack
• More efficient handling of resource consumption (network and CPU) and reduced impact on the Cardano node performance and availability
• Enhanced security with no impact on the Cardano node
• Different release pace for both nodes
• Both nodes will be bundled in the Cardano node distribution release
• A hard fork for SPO registration of the port of their Mithril network node is likely needed.

Hydra​

Issues and pull requests closed in July

Notable updates on our roadmap this month are:

• Completed incremental decommits, to be released separately as 0.18.0 after some remaining cleanup tasks and documentation updates
• Release 0.19.0 will be incremental decommits and, depending on the hard-fork date of Cardano mainnet, switching to Conway on the Hydra L2 ledger (see March 2024 for more details)
• Refined scope of "what is Hydra V1" through the Hydra working group
  • Added SDK for wallet integration with Hydra #1509 to provide a better developer experience
  • Decided to implement Directly open heads #1329 and Partial fanout to overcome our known issue & limitations and supersede several other 💭 ideas.
  • Prioritized Optimistic head closure #198 higher based on user feedback.
  • Declared React to protocol parameter changes #195 as not essential for a V1.

Incremental decommits completed​

This month we finally finished the implementation of the Incremental decommit #1057 feature.

As evident from the GitHub feature description, the number of comments and linked pull requests, this was a complex change.

Last month, we discussed why versioning the open state on-chain is necessary and the subsequent changes to the specification and implementation. After addressing these in hydra#1473 and in hydra#1474, a thorough review and requiring tests for all changes (compared to master) in hydra#1483 concluded in a last cleanup PR, which made the feature consistent in specification, implementation and documentation.

In the monthly review meeting we demonstrated the feature and the video below also shows how it works using a small extension to the hydra-tui:

Repository moved​

This month, the Hydra repository was moved to the cardano-scaling GitHub organization: https://github.com/cardano-scaling/hydra.

This allows for a more flexible use and sharing of GitHub runners between the main repository and for example hydra-explorer.

Most links should be automatically redirected, but if you encounter broken links, just reach out on Discord or create an issue 🙏 https://github.com/cardano-scaling/hydra/issues/new/choose

Working group updates​

There were two Hydra working group meetings this month and multiple people showed interest in joining. Current member list can be found in the charter and meeting notes of all meetings are kept in the repository too.

The working group started work on the first item "What is Hydra Head V1" and gathered valuable feedback in discussions between fellow layer 2 architects and Hydra users.

Notable inputs were the need for an SDK, wallet integration and the ability to operate without a full cardano-node, which got captured in hydra#1509 and hydra#1305 respectively. Even brand new ideas emerged from these discussions like "resumable Hydra heads".

The immediate goal is to achieve a common understanding of what must be included for a Hydra Head V1 and consequently when/how this could be audited?

Discussions also revolved around discoverability of Hydra heads, specifically, but also general interoperability with off-chain ledgers (any layer 2). In that regards, the group is considering to work on a CPS/CIP in the mid term.

Conclusion​

The monthly review meeting for July 2024 was held on 2024-07-22 via Google Meet, presenting these slides and this recording.

Despite the likely impact of summer breaks on attendance, it was great to have Blockfrost demonstrate their experiments using Mithril certificates delivered through Blockfrost in a forked Nami version. This demo clearly illustrates how light clients can benefit from Mithril verifiable transactions.

On the Hydra side, we are very pleased to have successfully merged and prepared the incremental decommits feature for release. Although this process took some time, it marks the first substantial feature not originally covered by the Hydra head paper, demonstrating the protocol's capacity to grow and meet user needs. We appreciate all the valuable input from the Hydra working group regarding the features we should develop next and are excited to see the growing community interest in Hydra's future.

This is a monthly report on the progress of 🐲 Hydra and 🛡 Mithril projects since May 2024. This document serves as a preparation for and a written summary of the monthly stakeholder review meeting, which is announced on our Discord channels and held on Google Meet. This month the meeting was held on 2024-06-24 using these slides and you can see the recording here.

Mithril​

Issues and pull requests closed in June

We have released the new Mithril distribution 2423.0. This distribution includes several critical updates and enhancements:

• BREAKING changes in the Mithril client CLI:
  • The deprecated snapshot command has been removed
  • It has been superseded by the cardano-db command
• Bug fixes and optimizations.

We have also created a developer blog post about the removal of the Mithril client CLI 'snapshot' command.

Additionally, we have released a draft version of the Mithril threat model analysis. We expect to receive external feedback and contributions before considering it final.

Transaction certification​

We kept working on the implementation of the Cardano transaction certification with Mithril and made good progress with the development:

• Transactions are now retrieved with the chain sync mini-protocol using the Pallas library
• The performance of the prover route of the aggregator has significantly improved.

Low latency certification​

The transactions are imported using the native chain sync mini-protocol of the Cardano node. This has been made possible with the Pallas library, which fully supports this mini-protocol. This allowed us to retrieve transactions much closer to the tip of the chain and at more frequent intervals. To achieve this, we have enhanced the transaction importer to handle chain rollbacks, where some blocks previously recorded may be discarded at a later time, with an increased probability as we get closer to the tip of the chain.

We are currently calibrating the system parameters (depth from the tip and certification pace) before activating it on the mainnet.

Increased prover performance​

We have identified some bottlenecks in the computations of the merkelized proof for transaction membership of the transaction set as well as in the database access. These bottlenecks have been fixed, allowing us to achieve a new order of magnitude in the performance of the prover route. It is now able to consistently deliver 10,000 transaction hashes certification per second.

Mithril/Cardano integration​

We have been working with the Cardano networking team to design a closer integration of Mithril into Cardano. In particular, we have focused on decentralizing the signature diffusion from Mithril signers to Mithril aggregators based on the Cardano network layer.

Here is the current roadmap:

• Assessing Cardano network bandwidth capacity to support Mithril signature diffusion
• Designing a node-to-node mini-protocol for signature diffusion
• Designing two node-to-client mini-protocols for signature submission and notification
• Preparing the submission of a CIP draft to the community.

Hydra​

Issues and pull requests closed in June

Here is a high-level overview of the team's work over the past month:

• Incremental decommits. We continued our work on incremental decommits, focusing on investigating adversarial attacks on the head script contract.
• Specification update. We updated our specification with the latest changes following our exploration of adversarial attacks. These updates aim to enhance the security of the protocol.
• Agda specification for the Hydra protocol. Work has begun on writing an Agda specification for the Hydra protocol. You can follow the progress here.
• Incremental commit. We discussed various options for the incremental commit work, including potential designs for a hydra-ledger to improve system efficiency and security.
• Node compatibility. The node has been successfully updated and is now compatible with cardano-node version 8.11-pre.
• Website. We updated our website tutorials and documentation to enhance expressivity and ease of use.
• Committing internal wallet UTXOs. We made breaking changes to the hydra-node API /commit endpoint #1463 to remove the check that prevented committing UTXOs from an internal hydra-node wallet.

We have not planned the next release, so our roadmap remains largely unchanged. Our current focus is on incremental decommits and commits and specifying our protocol in Agda.

Incremental decommit​

We were about to release the incremental decommit feature when we discovered a bug during a demo. This bug related to handling the decommit snapshot in close/fanout. When one decided to decommit funds from the head and everyone signed the decommit snapshot, the same snapshot was used to close/fanout. If this snapshot contained some UTXOs to decommit but one already decommitted funds from the head, the fanout would distribute the decommit funds again.

This issue was significant, and although we have many tests around the decommit feature, we had not checked user balances upon head closure.

We have implemented a fix that involves adding versioned snapshots. This allows us to distinguish between two cases:

• The decommit snapshot was signed, but the decommit was never observed
• The decommit snapshot was signed, and the decommit was also observed.

When we observe the decommit transaction locally, we increment the version in the local state. This allows us to compare the snapshot version against the expected local version on-chain.

We also needed to keep extra data in the Close redeemer to successfully check for a valid snapshot signature while altering the close datum to remove the UTXOs to decommit if the versions do not match. This ensures that the fanout behaves correctly, as it is only concerned with the close datum and the actual transaction output hashes.

The remaining work involves updating the specification with the changes in the implementation. We expect to release this feature in the coming days.

Community​

Subbit.xyz​

Algy Wallis presented 'Subbit', a layer 2 solution focused on subscription-based services, aimed at addressing the complexities of building blockchain products. Subbit is designed to be easier to integrate and manage compared to layer 1 solutions, reducing latency and transaction fees. The system allows providers to verify consumer IOU notes, facilitating periodic payments and withdrawals with minimal on-chain interactions. This model mirrors Bitcoin's Lightning Network and aims to enhance scalability and interoperability for subscription services on Cardano.

Hydrozoa​

George Flerovsky introduced the Hydrazoa proposal, currently in Catalyst Fund12. Hydrazoa aims to enhance the existing Hydra framework by adding dynamic capabilities for incremental commits and decommits, along with the ability to add or remove participants in state channels while they are still open. The goal is to simplify layer 1 scripts and lower costs, with fallback mechanisms similar to Hydra's in case of layer 2 consensus failure. George highlighted the differences and potential integrations with Hydra, emphasizing a more flexible and simplified approach to state channel management.

Technical working groups​

Reza Baram provided an update on the Hydra working group, which recently had its first meeting to discuss its vision and collaborative processes. The primary goal is to create an environment for sharing knowledge and working towards scaling Cardano beyond just Hydra. The group is currently focused on defining and delivering Hydra version one, prioritizing incremental commits and robustness improvements. Feedback from the community and working group members will be sought to refine and enhance the roadmap, ensuring effective scaling solutions for Cardano applications. The Hydra working group is open to new members and contributions from the community. Read more here.

Conclusion​

The monthly review meeting for June 2024 was held on 2024-06-24 via Google Meet, presenting these slides and this recording.

This is a monthly report on the progress of 🐲 Hydra and 🛡 Mithril projects since April 2024. This document serves as a preparation for and a written summary of the monthly stakeholder review meeting, which is announced on our Discord channels and held on Google Meet. This month the meeting was held on 2024-05-28 using these slides and you can see the recording here.

Mithril​

Issues and pull requests closed in May

We have released the new Mithril distribution 2418.1. This distribution includes several critical updates and enhancements:

• A breaking change introduced in the Mithril client / Mithril client CLI:
  • The certificate chain structure has been modified to remove coupling with the immutable file number
  • The client must be updated to verify the certificate chain
• We have switched the memory allocator to jemallocator on the signer and the aggregator to avoid memory fragmentation
• We have enabled the BLST portable feature by default to benefit from runtime check of intel ADX instruction set.

Also, we have started the process of removing the deprecated snapshot command from the client which is superseded by the cardano-db command. This change will be effective with the release of the next Mithril distribution.

Transaction certification​

We have kept working on the implementation of the Cardano transactions certification with Mithril and we have achieved a new milestone with running the certification in a test network (aka testing-mainnet) which operates on the Cardano mainnet.

This allowed us to refine our roadmap with clear objectives before the MVP can be released:

• Keep working on the low latency implementation of the certification
• Optimize the aggregator prover route to reach good throughput
• Optimize the signer footprint to limit the impact on the SPO infrastructure
• Optimize the signer and aggregator warmup to avoid network instability/disruption once the MVP is released.

A first optimization of the prover route has been implemented, drastically improving performance. By implementing resource pooling on the Merkle tree that signs transactions by block range, we have achieved a 100x increase in throughput.

We have also implemented a pruning mechanism on the signer that retains only the transactions needed to compute upcoming signatures. This has reduced the storage requirements from 32GB to 100MB.

In/Out SPOs dashboard​

We have added a new page to the Mithril explorer that lists the newly registered and recently de-registered signers in the Mithril protocol.

The protocol insights dashboard​

A new Mithril protocol insights dashbord has been released. It provides metrics such as the number of SPOs and total stake involved, the daily number of Cardano database restorations, and the breakdown of running signer versions.

Feel free to request some new metrics on the dashboard!

Hydra​

Issues and pull requests closed in May

This month, Hydra version 0.17.0 was released which includes

• breaking changes of the /commit endpoint, yielding full flexibility when committing from scripts (see this how-to for details)

• new GET /snapshot/utxo HTTP endpoint

• detecting network protocol mismatches

• Full release notes and a list of delivered features

Besides corresponding features being released through 0.17.0, the roadmap did not change much this month. Current focus is on incremental decommits and commits, while we also wait for the Conway hard-fork to happen before we can switch the L2 ledger also to Conway (see March report for more details).

Transaction trace testing​

In the process of implementing the incremental decommits feature, we constantly considered how to ensure our test suite covered all potential scenarios. For example, what happens if we request a decommit, have a signed snapshot, but never post a decommit transaction? Or how do we handle close and fanout with decommit UTXO in the snapshot? There are various scenarios we are interested in testing, and we quickly realized that we would need to test at different levels to achieve the coverage we expect, especially for our on-chain code.

Our mutation test suite was also very helpful in testing whether our on-chain code produces the expected errors when manipulating the transaction to make it invalid. However, this framework tests individual transactions, and we needed to test if multiple transactions in sequence produce valid results.

To address this, we decided to utilize quickcheck-dynamic once again to model simple snapshots, which are used to construct Hydra transactions. Our hope was that if the model is simple enough, we would be able to reason about it without delving into many details and decide if a particular UTXO/snapshot should be valid in the current transaction context or not. In the quickcheck-dynamic world, our 'Action' contains snapshots, which are then used to construct different transactions, evaluate, and observe them. Our model keeps the extra information we need to evaluate these transactions, and all snapshots are signed by everybody to keep things simple.

This test suite has already exposed some bugs we had and enabled us to reason about our code using a simpler model without burdening us with real-world details, which are more involved and harder to reason about.

Hydra-related Catalyst proposals​

Catalyst Fund12 is in full swing, with over 30 proposals this round mentioning 'Hydra' in one way or another. While some directly draw inspiration from Hydra and tackle similar challenges, others aim to enhance transaction processing scalability but take different approaches (such as rollups and sidechains). Additionally, some proposals focus solely on scaling payment transactions, offering refreshingly simple solutions.

After a quick screening, we also engaged with some of the proposal authors this month to address questions about Hydra, discuss similarities, and explore opportunities for mutual learning and collaboration:

Use cases​

• Wine Supply Chain Tracking and Reporting System. Great use case that requires the seamless integration of recording data as transactions off-chain, with resulting states and tokens accessible from the Cardano layer 1. The team is already developing a proof of concept using Hydra, and this proposal aims to advance it into a minimum viable product (MVP).

• Ikigai + MLABS - Grabbit / Hydra auctions: Running auctions within Hydra heads is a popular use case, allowing for fast and cost-effective bids on layer 2, while the final purchase of auction items seamlessly occurs on the Cardano layer 1. This proposal, in particular, will utilize incremental commits and decommits to facilitate fully-backed auctions on layer 2.

Using / extending hydra​

• Hydra-Enabled Micropayments System. The same team behind the wine supply chain tracking system wants to also provide a decentralized payment system that offers great point-of-sales experience using Hydra at its core. While they aim to use Hydra Head as a core component, this would also be a great use case for the more payment-focused variants in the long run (see below).
• Open-source Hydra Head L2 Web-based Explorer and API. Improving the visibility of what's going on in a Hydra Head is a great idea. Besides exploring what heads are available on the network, this would make transactions inside a head more visible and is very much related to this idea on our roadmap.
• Cardano Layer 2: Hydrozoa protocol for lightweight and flexible Hydra Heads for Cardano. An exciting re-interpretation of the core Hydra Head protocol and ambitious simplifications of roadmap features like directly open heads, incremental de-/commits and optimistic head closure. This was also discussed in the past with core contributors, and we would love to explore how those protocol variants can find their way into the upstream Hydra Head protocol.

Alternative scaling approaches​

• Subbit.xyz : Cardano's featherweight L2. Refreshingly simple take on how to process payments off-chain. Very much inspired by lightning payment channels and maybe even Chaumean eCash systems. Having payment channels that work in such a simple and flexible way is exciting and could be a great basis for any payment channel networks, ie Cardano Lightning.
• µgraph: Instant, Untraceable Payments in Cardano. Ambitious, but promising, take on what the Hydra tail could be – an asymmetric construction that benefits individual users by using zero-knowledge proofs to ensure correct operation of the node.
• Anastasia Labs Cardano Layer2 - Midgard. A layer 2 construction, similar to Hydra Head, that seeks to utilize the Cardano ledger isomorphically, but operates with a limited validator set and employs a distinct consensus mechanism to offer an experience akin to sidechains. We always wondered what you would get if you ran a Hydra Head with a non-total consensus and checkpointing state on layer 1!

Technical working groups​

As the age of Voltaire unfolds, it fosters the formation of community alliances and member-based organizations. At the same time, there's a popular belief that governance over core Cardano projects should become more open, with decisions regarding the inclusion or exclusion of elements on the roadmap of core infrastructure being transparent.

The current strategy devised by Intersect and its members to address this involves establishing technical working groups that operate openly and transparently on roadmaps and high-level backlog items. This approach mirrors what we're already doing here (see also last months starmap update section), but with a more concrete democratic approach to ratifying a roadmap. Presently, roadmaps for both Hydra and Mithril projects are expected to be governed in this manner.

Specifically for the Hydra working group, initial steps were taken this month by drafting a barebones charter and brainstorming first agenda items to engage potential members and get first sessions going as we gather more interested stakeholders. The charter and any future outputs of the working group will be hosted on cardano-scaling/wg-hydra and we also registered the working group at Intersect.

Conclusion​

The monthly review meeting for May 2024 was held on 2024-05-28 via Google Meet, presenting these slides and this recording.

May was a short month because of the public holidays and some people being off, but we still made good progress on both projects. The new dashboards for Mithril will visualize existing participation and surely garner more interest in the protocol. We are also excited about the progress on the Cardano transactions certification with Mithril, especially as multiple projects building bridges or layer 2s are keeping a close eye on this feature.

While we are busy on the Hydra side wrapping our head around incremental decommits and ensuring they are correctly implemented through transaction trace testing, we are also excited about the upcoming Catalyst Fund12 proposals and the discussions coming along with it as others are also looking into building similar, yet different scaling solutions and tackle the same challenges.

We are also making progress on making Hydra and Mithril less IOG-driven, but more community-owned through working groups and Intersect. We are excited to see how this will turn out.

This is a monthly report of progress on 🐲 Hydra and 🛡 Mithril projects since March 2024. This document serves as a preparation for and a written summary of the monthly stakeholder review meeting, which is announced on our Discord channels and held on Google Meet. This month, the meeting was held on 2024-04-30, using these slides and you can see the recording here.

Logo design work​

During the months of March and April, we focused on refreshing the logos of Hydra and Mithril. We also created a separate logo for the shared workstream aimed at scaling Cardano. It's crucial to have a logo for this workstream as it will help bring various projects together to enhance the scalability of Cardano.

Above, you can see the latest logo drafts created by our designer, Alexander Wende. Let us know your thoughts through any of our communication channels like Discord or Twitter.

Mithril​

Issues and pull requests closed in April

As a part of our strategy to increase the participation in the Mithril protocol we have been reaching out to SPO's who don't yet run a Mithril signer, providing them with the support they need to set up and run their signer. We appreciate the SPO's such as Cardano Foundation who took the step in joining the protocol. As of epoch 481, the number of stake pools registered in the Mithril protocol as signers has reached 258. This represents approximately 5 billion ADA, which is approximately 22% of Cardano's active stake.

We have prepared the pre-release of the new Mithril distribution 2418.1-pre. This release includes several critical updates and enhancements:

• A breaking change is introduced in Mithril client / Mithril client CLI:
  • The certificate chain structure has been modified to remove coupling with immutable file number.
  • The client must be updated to verify certificate chain.
• We have switched the memory allocator to jemallocator on the signer and the aggregator to avoid memory fragmentation.
• We have enabled the BLST portable feature by default in order to benefit from runtime check of intel ADX instruction set.

Signer registration decentralization PoC​

We have kept working on the decentralization experiments for Mithril networks. The signature diffusion decentralization has already been implemented in a previous proof-of-concept, and we have created a new proof-of-concept for decentralizing the signer registration process.

In order to do so, we have created a new peer-to-peer pubsub topic using libp2p and adapted the relay so that:

• When a signer relay receives a signer registration from a signer, it broadcasts it to the new P2P topic.
• When an aggregator relay receives a new signer registration from the P2P topic, it calls back the aggregator with it.
• A repeater mechanism has been implemented so that a signer relay sends the signer registration multiple times during an epoch (at regular pace) to prevent missed messages.

We don't need strong guarantees or consensus for the diffusion of signatures: different aggregators can create valid multi-signatures from different subsets of individual signatures as long as the quorum is reached. We could also implement the repeater mechanism so that a signature is broadcast a few times in order to prevent missed messages and reach more aggregators.

However, we need stronger guarantees for the signer registration: in order to create a valid multi-signature a minimum number of signers and aggregators must use exactly the same set of signer registrations (enough signers and aggregators so that the individual signatures created allow to reach the quorum). In case of a partition, the network could get in a situation where the quorum is never reached during an epoch: this would lead to a gap in the certification and the security of the protocol could not be guaranteed. Adversarial behaviors from some signers could for example be responsible for such partitions. Moreover, with the current design, as the number of aggregators is much lower than the number of signers, the risk of partition could be even higher.

This will lead us to:

• Reach for a consensus or quasi-consensus for the signer registrations.
• Adapt the topology of the network so that there are approximately the same number of signers and aggregators (i.e. all signers are also aggregators) or modify the aggregation process so that aggregators don't verify the message that is certified.

The next steps for the decentralization of Mithril networks are:

• Draft a CIP for leveraging the Cardano network layer and implement mini-protocols that will operate the diffusion of messages in a pubsub topic.
• Draft a CPS for stating the problem about signer registration.
• Get some feedback from the community, feel free to contribute!

Hydra​

Issues and pull requests closed in April

This month, the roadmap received updates to reflect an upcoming 0.17.0 release, which will include several API updates and two notable features:

• Revise POST /commit endpoint interface #1350 and
• Use versioned handshake when connecting hydra-nodes #1010

The latter was distilled out of the much broader and long-standing Hydra node software, scripts & protocol updates #191 as it was the last actionable piece of the included scenarios on how to handle change. Having this will make the hydra-node detect incompatible network protocols and improve robustness of the node implementation.

Besides feature ideas which are to be discussed and refined in the upcoming weeks, our focus lies on the two major features of incremental commits and decommits are currently being worked on by the team.

Why are incremental commits hard?​

Both, Incremental decommit #1057 and Incremental commit #199 turned out to be non-trivial and are in progres still.

While the plan is clear by now (see issue comments for a history), changes to the protocol are substantial and can be considered literally Hydra Head V2 (but any change to the scripts is a major change). Currently, the off-chain workflow is mostly implemented end-to-end, but the on-chain part is still in the works and requires a lot more testing.

As these features will allow to update the Open state on-chain, rollbacks and interleavings of so-called increment, decrement and the close transactions need to be considered. This is a complex problem that requires a lot of testing and validation to ensure that the protocol is secure and correct.

In particular, we need to be testing (ideally proving) that:

• funds already commited / not yet decommitted are always returned (completeness)
• funds are never returned twice (soundness)
• the head can process transactions throughout (liveness)

Our approach to testing is similar to when we implemented the basic protocol: end-to-end tests ensure overall integration, while property-based mutation tests ensure individual validators work correctly. This time, however, we are also looking at stateful property-based testing of many sequences of transactions (see this task and this module on the feature/incremental-decommit branch).

Hydra /commit endpoint enhancements​

We are about to release a refactor on the /commit endpoint which provides greater control for drafting commit txs, allowing for adjustments to the transaction context, which is crucial for real-world scenarios involving transaction validity checks, required signers, etc.

This refactor changes the endpoint to now accept a "blueprint" tx, CBOR-encoded, along with the UTxO, JSON-encoded. The UTxO is used to resolve inputs spent in the transaction, and the "blueprint tx" serves as the foundation for creating the resulting drafted commit tx, retaining anything that complies with the requirements of a valid head commit transaction.

This enhancement was initiated following a discussion led by the MLabs team, as documented here. The need arose to unblock a sub-validator utilized in the hydra-auction for checking spending conditions to move a standing bid to L2. The discussion was promptly converted into an issue and subsequently added to our roadmap after internal grooming by the Hydra team.

While this refactor represents a breaking change, the hydra-node can still be utilized as before if the provided UTxO is at a public key address. However, to spend from a script UTxO and unlock more complex use-cases, users must provide an additional unsigned transaction that accurately specifies required data, such as redeemers and validity ranges.

Cardano Buidler Fest​

This month the who-is-who of Cardano builders got lured to Toulouse, France for a two-day tech-only conference. It was a great occasion to meet fellow contributors and build bridges (literally) to new projects and their creators.

Two of many notable presentations and how they relate to the Hydra and Mithril projects are detailed as contributed by Pi (Sundae Labs) and Sebastian (Cardano Foundation).

IBC bridge​

Cardano Foundation has started to work on an implementation of the Inter Blockchain Communication protocol (IBC) to allow builders to connect other blockchains to the Cardano network that also implement IBC like e.g. Cosmos SDK or Substrate based blockchains. Whereas IBC has many similarities with other communication protocols like e.g. TCP and is based on the pretty straight forward delivery of data packets from one blockchain to the other by so called Relayers the general security model is based on the assumption that the consensus of both blockchains to be connected can be trusted. The latter resolves in the challenge that so called Light Clients need to be implemented that allow one network to check the validity of the consensus of the other, to verify if a specific operation like e.g. the locking of certain assets has been done as communicated via IBC. This typically implies that a blockchain network is able to generate so called state proofs that can be presented to the Light Client which then is able to perform the validation.

Unfortunately, the Cardano blockchain does not generate such state proofs as the underlying architecture and Ouroboros consensus do not rely on such a primitive. This leads to the the challenge that implementing a Light Client purely based on available data from the consensus (e.g. block data) has several shortcomings with regards to security or if traded for it liveness of the client causing significant latency in the message delivery between both networks (as the Light Client would have to wait until a sequence of blocks is presented for validation), which is not a feasible option for a real world production deployment.

However, there are other networks out there with a similar setup in that specific regard that recently claimed to have implemented IBC like e.g. Solana. Their approach is based on overlay networks that take care of collecting additional signatures from network validators for state proof data (like blocks, transactions, etc.) increasing the trustworthiness of those proofs. Fortunately, there is a similar solution currently under development in the Cardano ecosystem, which is Mithril.

The Cardano Foundation Engineering team has recently concluded the first phase of the IBC implementation demonstrating the technical feasibility of connecting Cardano to a Cosmos SDK based blockchain taking the disadvantages mentioned above into account. The work has been made public in a repository that can be found here: https://github.com/cardano-foundation/cardano-ibc-incubator

In a second phase that is scheduled to run from April till roughly beginning of July the team is working on replacing the existing supposedly weak Light Client implementation of the Ouroboros consensus on the Cosmos based blockchain with a Light Client that leverages the functionalities of Mithril. As a first approach we are intending to use specifically a recently introduced method that allows to check if a specific transaction is part of the global set of transactions of the network.

Gummiworm​

Gummiworm is a low-divergence fork of the Hydra node being built by Sundae Labs, which teases apart execution and custody; By sacrificing some control over the opening and closing of heads, it dramatically strengthens the security model specifically in the case when non-head participants are transacting in the head.

At Buidler-fest, we demo'd a very early, duct-tape and bubble-gum demo of depositing into a smart contract, minting those funds in the hydra head, and achieving auditability of the transactions in the head.

We also spoke about our longer term plans, such as changes to the ledger to accomodate a better user experience, and making the custody model "plug and play", so you could provide different mechanisms for different use cases.

Starmap update​

Every now and then, roughly quarterly, we like to take a step back from individual project roadmaps and think on an even higher level of what should be the priorities for the next couple months.

This time, the themes we would like to focus on are open governance, use cases and adoption, as well as measure and visualize. Across Hydra and Mithril we believe the following items, which ideally are backed by technical features on individual roadmaps, are suitable next steps:

As one can see from past quarters, opening up the governance of scalability projects is a recurring theme and of highest priority. Ironically, the proposed work items above were mostly shaped and prioritised in private discussions between IOI and PaloIT. We will need to open discussions of what is valuable to build, to gather feedback and ensure individual projects tie into a compelling and fundable portfolio across the whole ecosystem.

A first glimpse of this already happened at the Buidler Fest where a spontaneous (open space) session on "the future of Hydra and Mithril" spawned great interest and new ideas like

• Shallow node would be interesting
• Once per epoch Mithril-signed ledger state
• BLS accumulators for Hydra partial fanout; also relevant for Mithril?
• The need for another audit of latest Hydra Head protocol

As also listed on the starmap, a logical next step is to create an open working group with interested experts from the Cardano community. A working group to discuss whether and how a Hydra Tail protocol should be built out? What it would entail (pun intended)? Built by whom? How would it relate to Gummiworm, Cosmex, Subbit, zkFold, ... which should maybe prioritised by the Cardano community instead?

Conclusion​

The monthly review meeting for April 2024 was held on 2024-04-30 via Google Meet, presenting these slides and this recording.

While we saw some demos on Mithril and Hydra projects' progress, the real highlight this month was the Cardano Buidler Fest with its many great talks and discussions - often involving scalability limitations and technology to overcome it. Exciting discussions around zero knowledge primitives, infrastructure projects, and open source likely inspired everyone attending to keep building and improving Cardano.

There is still some uncertainty in how exactly governance of Hydra and Mithril can be made more open, collaborating on consistent high-level plans across multiple projects in the realm of improving Cardano scalability. However, we might have found with cardano-scaling.org a welcoming home not only for Hydra and Mithril, but for contributors and projects from various backgrounds, while first steps of involving the community more actively in a Hydra working group at Intersect are being made.

Besides this, the monthly review meeting held some sneak peeks of a new paper on fresh scalability tech (Ouroboros Leios) and fun conversations on games and other great ways to potentially show off a definite Hydra V1 at Rare Evo this summer - "Can it run doom?" 😈.

This is a monthly report of progress on 🐲 Hydra and 🛡 Mithril projects since February 2024. These reports have recently moved to https://cardano-scaling.org/monthly (more details here).

This document serves as a preparation for and a written summary of the monthly stakeholder review meeting, which is announced on our Discord channels and held on Google Meet. This month, the meeting was held on 2024-03-27, using these slides and you can see the recording here.

Mithril​

Issues and pull requests closed in March

This month, there has been a noticeable increase in SPOs joining the Mithril protocol. In particular, many new multi-pool operators have set up the Mithril signer in their infrastructure. At the end of epoch #475, there were 230 pools registered with 4.6B₳ stake involved representing approximately 20% of the total active stake. We will continue to provide regular updates to the community to encourage their participation in the protocol.

We have released the new Mithril distribution 2412.0. This release includes several critical updates and enhancements, such as support for Prometheus metrics endpoint in signer, deprecation of the snapshot command in the client CLI, full Pallas based implementation of the chain observer, and support for Cardano node version 8.9.0.

Transaction signatures​

We have developed a signing structure that enables quick proof generation while maintaining a light footprint on the signer for the mainnet. This is crucial as we need to handle a vast amount of transactions (100 million). To achieve this, we created a block range Merkle tree, which stores transactions in a Merkle tree and uses its root to create another Merkle tree. According to the our benchmark tests, a batch proof for 100 transactions can now be created in less than 250 ms.

We have also completed the roadmap of the MVP that will be released on mainnet:

Transaction verification in frontend​

The Mithril client WASM library has been used to implement a Cardano transaction verifier inside the Mithril explorer. This feature is currently available only on the testing-sanchonet network and will be progressively rolled out to the other Mithril networks. Wallet and DApp developers could leverage the client WASM library to implement a security layer that verifies transactions provided by a third party.

Prometheus metrics​

We have released a new Prometheus endpoint with the latest stable version of the signer. The feature can be easily activated by the SPOs as explained in this developer blog post.

Additionally, we created a Grafana template to easily set up a dashboard for this Prometheus endpoint: https://grafana.com/grafana/dashboards/20776-mithril-signer/

Pallas updates​

The latest stable distribution now includes a fully implemented chain observer powered by the pallas crate. Additionally, we have been collaborating with TxPipe to implement the chainsync mini-protocol with Pallas. This will significantly reduce the latency of signatures for Cardano transactions.

Deprecation warning​

We have deprecated the snapshot command of the Mithril client CLI as explained in this developer blog post.

We recommend using the replacement cardano-db snapshot command for users of the client CLI.

Hydra​

Issues and pull requests closed in March

Hydra project roadmap did not change much this month:

Notable updates​

• Evolved the user-requested feature idea #1337 into a new feature on the roadmap to revise POST /commit endpoint interface #1350. This will allow committing into a head directly from advanced smart contracts. It remains unclear whether this will be part of the upcoming 0.16.0 release.

• Captured the new feature idea of Directly open heads #1329 as item on the roadmap. This requires further grooming and could supersede other ideas like Always abortable head #699.

• Mostly completed Drop support for JSON encoded transactions #1213 and Streaming plugins #1325, which will be released soon as 0.16.0.

• Made substantial progress on Incremental decommit #1057, however, this is unlikely to be released as part of 0.16.0.

• Fully prepared Switch L2 ledger to Conway #1178, such that it can be released directly after the hard fork on the Cardano layer 1.

• As we improve our understanding of pivotal features such as incremental commits and decommits, the feature set required to address [known issues and limitations] (https://hydra.family/head-protocol/docs/known-issues/) is being narrowed down, and the scope for a convincing version 1.0.0 is becoming clearer.

Conway support​

As mentioned in previous roadmap updates, we separated work to support the upcoming Conway era on Cardano in two:

• Conway support #1177 will make the hydra-node support Conway and allow users to keep heads open across the hard fork.
• Drop Babbage support #1178 will eventually drop support for the Babbage era and retain maintainability.

While the former was already merged and is currently released in version 0.15.0 onwards, the team prepared the latter this month.

Ideally switching the transaction format in the hydra-node is as simple as setting the type alias type Era = Conway. However, the code was Babbage-specific in several areas and work on this resulted in various pull requests making switching easier in several places: #1326, #1327, #1328, #1334, and #1342.

The result is a relatively simple switch in this draft pull request and available on branch lc/conway-support, which we intend to bump and merge once the public Cardano networks hard fork.

Having full Conway era support, including PlutusV3, on the Hydra layer 2 already prepared on a branch allows early adopting users from the community to explore and use this already. Modulo-P has been on the forefront here by developing zero-knowledge hybrid DApps and wrapping the Hydra branch up in this example repository.

Streaming plugins​

SundaeLabs is working on a Hydra-based layer two protocol (gummiworm) that requires the usage of hydra-node components in a different way than the default setup. As part of this development, SundaeLabs submitted a Fund10 Catalyst proposal to enhance the Hydra node by adding features that would facilitate not only their work but also the work of everyone building on Hydra.

They have been working on a plugin architecture to enable the infiltration and exfiltration of events, which enables hygienic forks of Hydra that feature additional connections to the outside world. They are interested in consuming the Hydra event stream via Amazon Kinesis and storing transaction archives on S3 for later reference.

After an initial design phase resulting in ADR29, they contributed the event source and sink abstractions, and refactored the Hydra node to use these new extension points together with core contributors. The EventSource and EventSink interfaces, along with the configurable list of eventSinks in the main HydraNode handle, allow to extend the hydra-node with any external system. It also paves the way for more scalable persistence methods on the mainline Hydra node, and might even allow refactoring the existing API server as an EventSink.

Community​

Mithril office hours​

As the Catalyst project of TxPipe on Mithril is completed, we want to build upon this initial maintenance work in a more open format. For this, we are transforming our alignment calls into a public weekly office hour where we invite the community to join and discuss concrete pull requests, issues and questions on Mithril.

👉 Mithril office hours happen every Tuesday at 16:00 UTC on the #mithril-live Discord channel.

Hydra governance walkthrough​

This month, the Hydra team has been asked to provide a walkthrough of how we use GitHub to run and govern the project. We have prepared a video that explains how we use Github issues, pull requests, discussions, and projects to manage the development of the Hydra project. However, it is important to note that this information represents only a current view and we are enthusiastic about continuously improving the processes to make them more transparent.

We are excited to announce that we are looking to establish a Hydra working group at Intersect. This group will aim to involve more community members in proposing feature backlogs, roadmaps, and RFPs. Initially, the focus of this working group will be on the concrete Hydra Head protocol (state channels). However, variations such as Hydrozoa, and combined scalability protocols like Gummiworm, Lightning-like payment channel networks, or the Hydra Tail protocol (~ rollups) will also require more community involvement. Stay tuned for more updates on this in the upcoming months.

Conclusion​

We held the monthly review meeting for March 2024 on 2024-03-27 via Google Meet, presenting these slides and this recording.

This month we have made significant progress on Mithril transaction signatures, while the Hydra team has been busy working on the incremental commits/decommits features. Both projects are fully prepared for the upcoming Conway hard fork on Cardano and continuously demonstrate user experience improvements.

Besides the Hydra explorer user interface, the teams also showcased the Mithril transaction verification feature in the Mithril explorer frontend. This demonstrates that Cardano transactions can be additionally verified using Mithril certificates from within a web browser. Taking this even further, we believe that such a "Verified with Mithril" checkmark can provide additional level of confirmation for users of light wallets and Cardano DApps!

Besides these, Modulo-P showcased their early adoption of Hydra heads using Conway and PlutusV3 for their quite involved zero-knowledge protocols. Briefly after the meeting they released a first look on their zkMastermind DApp. Also, SundaeLabs presented their streaming plugins for Hydra. Both are examples of how the Hydra protocol can be extended and used in different ways, and we are excited to see more community contributions in the future.

As a closing remark, we see the trend of decentralizing governance on both projects, Hydra and Mithril, while at large the overall 'age of Voltaire' is unfolding. Consequently, we anticipate moves, merges, or restructuring of repositories, projects, and working groups. What an exciting time to be a part of this big Cardano community!

This is the first joint report of the 🐲 Hydra and 🛡 Mithril projects, which summarizes the teams' progress since January 2024.

Previously, both teams had been releasing progress updates in different formats and intervals within their respective repositories:

• Hydra monthly reports
• Mithril dev blog

Both sites are created from the individual project repositories. However, it is not ideal to keep 'procedural' documents as part of the code base, since it requires special handling when building versioned, published documentation. For example, this workflow in the Hydra project which publishes to https://hydra.family required several attempts to work around how Docusaurus websites are built. Consequently, we introduced a dedicated repository that builds into a website. This not only helps keep the original product repositories clean of inherently dated information, but also serves as a new home for joint reports, announcements, or occasional articles.

This monthly report serves as preparation for the monthly stakeholder review meeting (see slides and recording). At the meeting, the teams update stakeholders on recent developments and gather feedback on proposed plans.

Mithril​

Issues and pull requests closed in February

Release and publication​

This month, the Mithril team released distribution 2408.0 and published two blog posts:

• Mithril SanchoNet network released.
• Mithril client CLI output breaking change.

Signing Cardano transactions proof of concept​

The team has been working on implementing a proof of concept of a new data type that will enable the certification of Cardano transactions within Mithril networks. Last month, the team extended the Mithril client library to support the certification of Cardano transactions. This month, the team have upgraded the client CLI and introduced a new cardano-transaction command that can list snapshots and certify a list of Cardano transactions. Additionally, they have implemented the same API surface on the WASM client that now supports transaction certification in the browser.

Enhanced support for Conway and SanchoNet​

The team has conducted thorough testing on the upcoming Conway era and are confident that Mithril networks will transition smoothly to this new era. Additionally, the team have released a new testing-sanchonet network that has been opened to SPOs to enable collaborative testing of the upcoming and experimental features (eg the aforementioned signature of the Cardano transactions).

Hydra​

Issues and pull requests closed in February

This month, the Hydra project roadmap underwent a few changes:

Notable updates​

• New: Drop support for JSON encoded transactions #1213 due to breaking changes when upgrading cardano-node and cardano-ledger versions.
• Smoke tests on SanchoNet #1257 took longer than expected, revealing issues, but seeing smoke tests run on SanchoNet makes the team confident the hydra-node is ready for the Conway hard fork
• Build and deploy a Hydra heads explorer #696 is around 50% complete. Data is available for sanchonet at http://explorer.hydra.family/heads but a user interface is still needed.
• Started working on Incremental commit #199 as the UX will be similar to Incremental decommit #1057. Implementing both features off-chain allows early adoption while improving on-chain security.
• Added the Streaming plugins #1325 feature to cover SundaeLab's work on their Catalyst proposal

Contest after fanout bug hunt​

While working on hydra-chess, the team encountered a bug in the Hydra node. In the two-party Hydra Head, nobody was able to perform a Fanout. This happened because one of the nodes was in the initial Snapshot state and missed the request for a new Snapshot, while the other node already had one confirmed Snapshot locally.

When the Close was posted, the Head was closed with the initial Snapshot and then contested by the node that saw a newer Snapshot. After the contest, the team observed a strange behavior where both nodes could not Fanout. We expected this failure in the node that only had the initial Snapshot since it did not have the correct UTxO locally to Fanout. However, the failure in the other node with the newer Snapshot should not have happened.

While reviewing the code, the team discovered that the Contest observation was not extending the deadline after the contestation was made. As a result, the Fanout transaction was invalid because the validator checks that the Fanout transaction's lower validity bound is higher than the deadline to contest. This check ensures that one can only fan out after the contestation period is over.

Before fixing this bug, the team tried reproducing it using the MBT (Model Based Tests) framework, one of the tools in the test suite. However, to do that, they needed to complete modeling the entire head lifecycle and the specific scenario they observed in real life and arbitrary actions in all head states did not reveal this bug.

After also improving the framework itself to correctly validate transactions and a fix for simulating rollbacks, the team was able to fix the bug easily. However, it proves that being able to reproduce a bug is not only improving the code, but also allowed the team to gain knowledge and improve their test suites.

Hydra explorer supporting multiple versions​

This month, the team deployed the hydra-explorer to serve data under http://explorer.hydra.family/heads. Alongside this, they upgraded the Hydra scripts and switched to the latest pre-release version of cardano-node for Conway, and the system is currently running on the sanchonet network.

The Hydra explorer currently has the capability to support only one version on a single network. The reason for this limitation is that the Hydra scripts are compiled into two binaries- hydra-node and hydra-chain-observer - and the observer is connected to a single cardano-node that operates as a client on one network.

To support multiple versions of the protocol #1282, the team needs to compile the scripts of each version into individual chain observer binaries. These binaries can then communicate with the hydra-explorer components through an API. While Plutus validators change with any modification (thus affecting their hashes and addresses), this API can serve as an integration point that the team can support through multiple versions of the protocol.

Community​

Both projects saw community contributions this month:

TxPipe on Mithril​

The TxPipe team has kept working on the implementation of the Pallas Chain Observer for Mithril nodes and has contributed to retrieving the stake distribution and Mithril era markers (stored in the UTxOs of the chain) directly from the Cardano node with mini-protocols.

Ikigai auctions on Hydra​

As it also was briefly shown in the review meeting, Ikigai continued work on Hydra for Auctions in their Catalyst-backed project. Their use case is about starting auctions on the Cardano mainnet, but then collecting bids off-chain and only committing the final result on-chain. This is a perfect use case for Hydra, as it allows for a large number of off-chain transactions to be processed in parallel and then committed to the blockchain in a single transaction.

Conclusion​

We held the monthly review meeting for February 2024 on 2024-02-26 via Google Meet, presenting these slides and this recording.

This month, both projects focused on preparing for Conway and the upcoming hard fork combinator event. Preparations for the hard fork made the cardano-node a moving target, but the teams believe that early integration is key. Unexpected issues popped up in both projects, but they were able to address them.

New feature development is progressing well, with Mithril's transaction signing now available to early adopters. This feature opens up the possibility of making hydra-nodes more lightweight, as they won't need to rely on a single third party for trusted chain data. Instead, they can verify on the client (the Hydra node) using Mithril certificates. Additionally, Hydra's incremental decommit feature is in progress. While there's still significant work to be done on the on-chain parts, the teams have already implemented the off-chain workflow for incremental commits. Making both features available for users to try out will help validate the API changes. However, this cannot be released to mainnet without completing the required on-chain security work. During the review meeting, the teams sought feedback on a potential simplification of the Hydra on-chain protocol to have 'directly open' heads. Any thoughts or feedback on this fresh feature idea #1329 would be appreciated.

In general, if you read this far we would like to hear from you, dear reader, on any of our communication channels on Discord or a shout out on twitter. Maybe along with some feedback on the new, common website and the joint monthly report format? 🙏

We are looking forward to the next month and when it happens, the Conway hard-fork on the first public testnets (no date yet) when 🛡 Mithril serving certificates and 🐲 Hydra heads remaining open throughout.