This is a monthly report on the progress of 🐲 Hydra and 🛡 Mithril projects since February 2025. It serves as preparation for, and a written summary of, the monthly stakeholder review meeting. The meeting is announced on our Discord channels and held on Google Meet. This month, the meeting took place on March 26, 2025, using these slides, and you can see the recording here.

Mithril​

Issues and pull requests closed in March

Roadmap​

Below are the latest updates on our roadmap:

• Cardano database incremental snapshots MVP #2047: we completed the implementation of the MVP for incremental snapshots of the Cardano database and plan to start releasing it on test networks next month.
• Multiple aggregators on a Mithril Network PoC #2238: we finalized the proof of concept (PoC) for multiple aggregators on a Mithril network. We are adapting the infrastructure to support this feature on selected test networks.
• Redesign the website and improve documentation #2051: the Home page redesign is in progress.

Distributions​

We released Mithril distribution 2513.0, which includes:

• ⚠️ Breaking changes in Mithril nodes:
  • The minimum required glibc version for pre-built Linux binaries has been upgraded from 2.31 to 2.35
  • Mithril signers running versions <=0.2.200 must be updated due to the removal of Thales era legacy code
  • The with_snapshot_uploader function in the Mithril client library has been renamed to with_file_uploader
• Added support for Cardano node 10.2.1 in the signer and aggregator
• Ended support for macOS x64 pre-built binaries for the client CLI
• Bug fixes and performance improvements.

In April, the following events are planned:

• Release of a new distribution
• Activation of the incremental Cardano database certification (Cardano DB v2) on the pre-release-preview and release-preprod network.

Dev blog​

We have published the following post:

• Distribution 2513 is now available

Signer registration with multiple aggregators​

We explored solutions for running multiple aggregators on the same Mithril network. This is a complex problem that requires consensus on signer registration among enough signers and aggregators so that at least the required quorum can be achieved. Currently, the aggregator stores signer registration payloads and distributes them back to signers a couple of epochs later. This is a centralized approach (a single point of failure) that may be subject to censorship, but there is no trust assumption on the aggregator because every party in the protocol independently verifies the cryptographic proofs included in the payloads.

We identified three possible solutions:

1. Extend the centralized approach to multiple aggregators: slave aggregators pull signer registration payloads from the master aggregator at every epoch change, rejecting any registrations sent directly to their API. We prototyped this solution, which unlocks decentralized signature diffusion while maintaining a central authority for registration. 2.Use the Cardano chain: aggregators create a transaction to store their Mithril keys on chain at the start of each epoch. This approach is fully decentralized but more complex operationally. It requires SPOs to maintain a hot wallet for transaction creation, adding cost and maintenance. This is likely the more natural long-term solution.
2. Use the DMQ: the DMQ (Decentralized Message Queue as described in the CIP-0137) is authenticated which means that we can detect adversary signers which would equivocate their signer registration (i.e. send different payloads to different peers on the network to create a split of the network and thus prevent reaching the quorum needed to create a Mithril multi-signature). Any peer on the network that would receive two different payloads from the same signer would create a proof of equivocation and broadcast it on the network. Upon reception of a valid equivocation proof, the other peers of the network would discard the registration from the adversary and reach an agreement on the signer registration. This solution would be fully decentralized and would not incur any cost for the SPOs. It is at a very early stage of exploration and we will keep working on it.
3. Use the DMQ: the DMQ (Decentralized Message Queue) described in CIP-0137 is authenticated, so the network can detect adversaries who equivocate signer registrations. If a signer attempts to submit conflicting payloads, other peers can broadcast a proof of equivocation, causing the network to discard the adversary’s registration. This approach is fully decentralized and does not impose additional costs on SPOs. Exploration is in progress. The proposed signer registrations

Protocol status​

The protocol has operated smoothly on the release-mainnet network with the following metrics: The protocol operated smoothly on the release-mainnet network with the following metrics:

• Registered stake: 4.6B₳ (21% of the Cardano network)
• Registered SPOs: 246 (9% of the Cardano network)
• Full Cardano database restorations: 500 restorations
• Signer software adoption: 98.6% of the SPOs are running a recent version (one of the last three releases).

You can find more information on the Mithril protocol insights dashboard.

Hydra​

Issues and pull requests closed in March

This month, notable roadmap updates include:

• Bounded memory #1618
• Side-load of fully-signed snapshot #1858
• Etcd control via environment variables #1883
• Fixed memory bug when loading large state file #1917
• Investigated mirror nodes #1910.

Etcd-based networking​

Feature: #1720

The etcd-based networking feature is now fully merged and has undergone several improvements. Highlights include bug fixes, improved observability, protocol version checking, and a breaking change in command line options (--listen and --advertise) see release notes for more details.

Extensive testing through our friends working on the Midnight glacier drop makes us confident to release this soon as a cornerstone of the 0.21.0 release.

Bounded memory​

After preliminary work on bounding memory by streaming events and implementing the API server as an event sink last month, we achieved fully bounded memory usage of hydra-node when processing millions of transactions and loading from disk (#1920).

A typical memory profile of a hydra-node using ~1GB of memory when loading millions of transactions from the state file before this work:

Which indicates a Haskell-classic space leak because of thunk build-up. Indeed we spotted that allTxs in the HeadState was not fully forced when loading events from disk and with an additional strictness annotation, loading the same state file now while only using 12MB of memory:

Mirror nodes​

We explored mirror nodes to improve fault tolerance for Hydra heads. A participant can run multiple instances of hydra-node using the same --hydra-signing-key and --cardano-signing-key, allowing snapshot signing to continue even if one node goes down.

Preliminary experiments showed this approach works without modifications to hydra-node, but:

• Each additional node increases the number of messages submitted through the Hydra network
• Too many mirrors of one party could imbalance the etcd quorum and make the network unavailable overall although there would be enough signers for the Hydra consensus.

Side-load snapshots​

Feature: #1858

Originally, we wanted to address so-called 'stuck' heads to address ledger state divergence among Hydra nodes, we introduced snapshot sideloading. This mechanism allows nodes to adopt a confirmed snapshot to regain consensus when misalignment occurs, preventing the Hydra head from getting stuck and ensuring a consistent state across all nodes.

Withdraw zero trick​

Feature: #1795

This feature was requested by multiple users in GitHub and on our Discord channel, and it was identified as beneficial for the Midnight glacier drop.

The so-called 'withdraw zero trick' is a common technique to achieve verification on a transaction level (once per tx). In fact, it's the only way to do this until CIP-112 gets implemented by the cardano-ledger (and plutus).

While the layer 2 ledger in Hydra is isomorphic to Cardano with respect to the EUTXO ledger model, this excludes Cardano's proof-of-stake features. Consequently, one could not register scripts as reward accounts and use withdrawing 0 lovelace as a trick to have those scripts validate transactions.

To implement this feature, @coll78 and @ch1bo found that the hydra-node can mock RewardAccounts for each 0 lovelace withdrawal observed in a transaction on-the-fly. This results in the script being evaluated, and even does not require the script to be registered via a stake delegation certificate beforehand.

See this new How-to for more details.

Links​

The monthly review meeting for March 2025 took place on March 26, 2025, via Google Meet. The presentation slides and the recording are available for review.

This is a monthly report on the progress of 🐲 Hydra and 🛡 Mithril projects since January 2025. This document serves as a preparation for and a written summary of the monthly stakeholder review meeting, which is announced on our Discord channels and held on Google Meet. This month, the meeting took place on February 26, 2025, using these slides, and you can see the recording here.

Mithril​

Issues and pull requests closed in February

Roadmap​

Below are the latest updates on our roadmap:

• Cardano database incremental snapshots MVP #2047: we have started the optimization and production readiness phase on the MVP for incremental snapshots of the Cardano database
• Redesign the website and improve documentation #2051: the Mithril certification section has been created. Home page redesign is in progress.

Distributions​

We released Mithril distribution 2506.0, which includes:

• ⚠️ Security:
  • This distribution embeds a fix for the Mithril certificate chain could be manipulated by an adversarial signer security advisory GHSA-724h-fpm5-4qvr
  • All users running a client library, client CLI or client WASM are strongly encouraged to update them to the latest version
• Stable support for Cardano node 10.1.4 in the signer and the aggregator
• Dropped support for Thales era in the signer and the aggregator
• Stable support for traffic compression in signer, aggregator and client
• Bug fixes and performance improvements.

In February, the following events have taken place:

• Switch to the Pythagoras era on the release-mainnet network (which occured at transition to epoch 539)
• Re-genesis of the certificate chain of the release-mainnet network (which occurred at the end of epoch 539).

In March, the following events are planned:

• Release of a new distribution

Dev blog​

We have published the following posts:

• Minimum required glibc version bump
• Distribution 2506 is now available
• Mithril certificate chain security advisory
• Decommissioning the testing‑sanchonet network
• Ending support for macOS x64 pre-built binaries.

Incremental Cardano DB certification​

We have worked on the implementation of the client library, CLI and WASM to support the incremental Cardano DB certification. This feature allows the Mithril client to restore a partial Cardano database rather than the full one, by providing a range of immutable files:

• Restoration up to a given immutable file
• Restoration from a given immutable file
• Restoration for a range of immutable files
• Full restoration of all the immutable files.

We will keep working on stabilizing and optimizing the feature to make it ready for production.

New 'Mithril certification' section on the website​

We have created a new section on the website to provide a better understanding of the Mithril certification process. You can find it on the Mithril certification page.

The section includes:

• Mithril certification: an overview of the Mithril certification process
• Cardano transactions: a detailed description of the Cardano transactions certification process
• Cardano stake distribution: a detailed description of the Cardano stake distribution certification process
• Cardano node database: a detailed description of the Cardano database certification process
• Cardano node database v2: a detailed description of the upcoming incremental Cardano database certification process.

Protocol status​

The protocol has operated smoothly on the release-mainnet network with the following metrics:

• Registered stake: 4.7B₳ (22% of the Cardano network)
• Registered SPOs: 248 (9% of the Cardano network)
• Full Cardano database restorations: 325 restorations
• Signer software adoption: 99.9% of the SPOs are running a recent version (one of the last three releases).

You can find more information on the Mithril protocol insights dashboard.

Hydra​

We have released incremental commits in version 0.20.0.

Issues and pull requests closed in January

This month, some notable roadmap updates include:

• Multiple-version support in Hydra Explorer
• etcd-based networking #1720
• Progress on bounded memory #1618.

Multiple versions in Hydra explorer​

Starting from version 0.19.1 (a backport), we now list all known versions of Hydra on the explorer.

Etcd-based networking​

A breaking change in command-line arguments and system requirements (see release notes) is currently unreleased but available on the master branch. This update significantly improves the networking architecture, leading to greater stability (ie, fewer stuck heads!). Please try it out!

Bounded memory for hydra-node​

In combination with the etcd-based networking, we have been working on bounding the memory of the hydra-node by switching to an event-streaming model, instead of loading and keeping all events in memory. This should allow for the long-term running of hydra nodes. This work remains in progress.

Links​

The monthly review meeting for February 2025 took place on February 26, 2025, via Google Meet. The presentation slides and the recording are available for review.

This is a monthly report on the progress of 🐲 Hydra and 🛡 Mithril projects since December 2024. This document serves as a preparation for and a written summary of the monthly stakeholder review meeting, which is announced on our Discord channels and held on Google Meet. This month, the meeting was held on 2025-01-29 using these slides and you can see the recording here.

Mithril​

Issues and pull requests closed in January

Roadmap​

Below are the latest updates on our roadmap:

• Cardano stake distribution certification #955: the feature has been activated on mainnet
• Cardano database incremental snapshots MVP #2047: we have kept working on the MVP for incremental snapshots of the Cardano database
• Redesign the website and improve documentation #2051: the User manual and About Mithril sections have been restructured. The redesign is in progress.

Distributions​

We released Mithril distribution 2450.0, which includes:

• 🔥 Breaking changes in the Mithril client library, CLI, and WASM with the removal of the deprecated network field from the internal CardanoDbBeacon in Mithril certificates
• Stable support for Cardano node 10.1.3 in the signer and the aggregator
• Stable support for one line shell installation script of the Mithril nodes' pre-built binaries
• Bug fixes and performance improvements.

In February, the following events are planned:

• Switch to the Pythagoras era on the release-mainnet network (at the transition to epoch 539)
• Re-genesis of the certificate chain of the release-mainnet network (at the end of epoch 539)
• Release of a new distribution at epoch 540.

Dev blog​

We have published the following posts:

• Era switch to Pythagoras
• Distribution 2450 is now available.

Incremental Cardano DB certification​

We have worked on the signature and artifact production processes of the incremental Cardano DB certification. The features are now deployed on the developer networks and are being tested.

The incremental certification will provide the following benefits:

• Faster certification of the Cardano DB on the aggregator
• Capability to restore a partial Cardano database instead of the full one (by providing a range of missing immutable files)
• Capability to support an evolutive range of artifact storages, particularly decentralized storages (eg, IPFS or BitTorrent).

Website new structure​

We have restructured the main sections of the website:

• the User manual section has been restructured to make the content more accessible and provide entries for each user profile
• the About Mithril section has been restructured to provide information for beginner and advanced user audiences.

SPO participation​

We have worked on a plan to reach the full participation of SPOs in the Mithril network, which is essential for the security of the protocol:

• Phase 0: Guild Operators’ scripts adjustment
  • Install Mithril by default (currently opt-in)
• Phase 1: node integration
  • Bundle Mithril with the Cardano node
  • Seamless integration
  • Opt-out option
• Phase 2: centralized incentive mechanism
  • Centralized reward distribution
  • Treasury funded via external funds
• Phase 3: on-chain incentive mechanism
  • On-chain reward distribution mechanism
  • Treasury funded through revenues.

Protocol status​

The protocol has operated smoothly on the release-mainnet network with the following metrics:

• Registered stake: 4.9B₳ (25% of the Cardano network)
• Registered SPOs: 250 (9% of the Cardano network)
• Full Cardano database restorations: 600 restorations
• Signer software adoption: 94% of the SPOs are running a recent version (one of the last three releases).

More information is available at the Mithril protocol insights dashboard.

Hydra​

We've released incremental commits in version 0.20.0.

Issues and pull requests closed in January

Notable updates on our roadmap this month include:

• Released incremental commits #199
• Verifyed custom-ledger support #1727
• Full transaction in SnapshotConfirmed #1685
• Raw CBOR datum in transaction outputs #1543

Hydra Doom finale​

On January 24, 2025, the Hydra Doom tournament concluded with its finale at the C-Play Summit event at the HyperX Arena in Las Vegas. This live e-sports event featured finalists from previous hydra-doom stages battling in a multiplayer deathmatch on stage, using our reimagining of the 1993 first-person shooter Doom. For a high-level summary, see this Hackernoon article.

Let’s look a bit under the hood at what was deployed for this finale. Previous installments of the hydra-doom application processed player inputs and game state updates as Cardano transactions on a fleet of remotely hosted hydra-node processes and Hydra heads. This time, our fork of the Doom doom-wasm port used Hydra to create a net_module_t that converts Doom multiplayer network packets to Cardano transactions and uses the Cardano ledger in a Hydra head to exchange them between multiple players. On top of this, each player’s kills were also recorded 'on-chain' from similar transactions as used in the past. This allowed a referee Hydra participant to use the datum on-chain to update a 'Series' smart contract and gather kill totals as a datum in a Cardano UTXO (on layer 2).

Given the results of three deathmatch rounds, a winner was determined, and all finalists received prize tokens for 1st to 4th place. These tokens were minted before the finale on layer 1, committed into the head, and distributed to known finalist addresses in the head. Closing and finalizing this head onto mainnet distributed the prize NFTs.

The tournament’s head ID was d3ad8ab3bbaf06ccadd7076ff3a84eaca0771bb9dcea8bcce239eca9, and you can view the associated transactions here. This is the fanout transaction that distributed the prize tokens.

Using these tokens, the winners could redeem their prize money from smart contracts where the USDM prize money for the finale was locked, as seen here for the 1st place trophy.

Incremental commits​

The incremental commits feature has been released in version 0.20.0. We invite community members to test-drive this feature. The goal is to ensure the best user experience and identify potential bugs.

Hydra + custom ledger experiment​

We have verified that we can run a Hydra head with a customized layer 2 ledger. We merged a test that can be used for future interested parties to test this with their own forks.

Links​

The monthly review meeting for January 2025 was held on 2025-01-29 via Google Meet, presenting these slides and this recording.

This is a monthly report on the progress of 🐲 Hydra and 🛡 Mithril projects since October 2024. This document serves as a preparation for and a written summary of the monthly stakeholder review meeting, which is announced on our Discord channels and held on Google Meet. This month, the meeting was held on 2024-12-09 using these slides and you can see the recording here.

Mithril​

Issues and pull requests closed in November

Roadmap​

Here’s the latest on our roadmap:

• Cardano stake distribution certification #955: the feature is pending activation on mainnet
• Protocol usage metrics/statistics #2028: the feature is completed and has been released in November
• Explore signer registration solutions #2029: we are exploring the signer registration options and preparing a document summarizing them
• Cardano database incremental snapshots PoC #2047: we have started working on a proof of concept for incremental snapshots of the Cardano database.

Distributions​

We released Mithril distribution 2445.0, which includes:

• 🔥 Breaking changes in the Mithril client library, CLI, and WASM with the removal of deprecated beacon in Mithril certificates
• Stable support for Cardano node 10.1.2 in the signer and the aggregator
• Stable support for Cardano stake distribution client library, CLI, and WASM
• Stable support for the Prometheus metrics endpoint in the aggregator
• Bug fixes and performance improvements.

Future distributions​

We plan to release new distributions in December:

• 2450:
  • Availability of NodeJS and bundler targets in the Mithril client WASM npm package
  • Stable support for Cardano node 10.1.3 in the signer and the aggregator
  • Execution rights of pre-built binaries in GitHub releases.

Dev blog​

We have published the following posts:

• One line installer for Mithril binaries
• New Protocol Insights Dashboard released
• Mithril aggregator Prometheus endpoint is available.

Protocol Insights Dashboard v2​

We have created a new version of the Mithril Protocol Insights Dashboard, which is now available here. It provides a comprehensive view of the Mithril network and its performance metrics.

The Protocol Insights Dashboard is a valuable tool for monitoring the network and understanding its behavior:

• Participation metrics
• Usage metrics
• Health metrics
• Artifacts metrics
• Software metrics.

One line installer for Mithril binaries​

To facilitate the installation and updating of Mithril binaries, we have created a one line installer that downloads and installs the Mithril binaries and supports Linux and macOS. It is available for the Mithril signer, Mithril aggregator, and Mithril CLI.

Here is an example command to download the latest Mithril signer binary in the current directory:

curl --proto '=https' --tlsv1.2 -sSf https://raw.githubusercontent.com/input-output-hk/mithril/refs/heads/main/mithril-install.sh | sh -s -- -c mithril-signer -d latest -p $(pwd)

This will download the latest signer binary for the correct platform and architecture (if available) in the current directory:

Fetching release information from https://api.github.com/repos/input-output-hk/mithril/releases/latest...
Downloading mithril-signer to latest from https://github.com/input-output-hk/mithril/releases/download/2445.0/mithril-2445.0-linux-x64.tar.gz...
Congrats! mithril-signer has been upgraded to 0.2.209+67dc6e4 from distribution latest and installed at ./home/user!

Client NPM package compatible with NodeJS​

We have created a new version of the npm package for the Mithril client WASM, which is now compatible with NodeJS and bundler targets. This feature was requested by the community and will be released in the next 2550 distribution.

New explorer status​

We have enhanced the Mithril Explorer with a new status section that provides some key metrics and insights about the Mithril network. This new section is currently available on testing networks and will be generally available with the release of the next 2550 distribution.

Nightly and backward compatibility workflows​

We have implemented nightly and backward compatibility workflows to ensure more reliable and consistent releases. The nightly workflow builds Mithril Docker images every night and runs the backward compatibility workflow, which checks the compatibility of the upcoming Mithril distribution with previous versions, summarizing the changes.

Hydra​

We continued working on incremental commits, began supporting multiple versions in the Hydra Explorer, and started moving the hydra-explorer out of the monorepo.

Issues and pull requests closed in November

Notable updates on our roadmap this month include:

• Progress on incremental commits #199
• Hydra + Midnight support #1727
• Continued support of Hydra Doom, especially around the tournament!

Hydra Explorer​

The hydra-explorer has been moved to its own repository with independent deployment. As a result, supporting multiple versions of the hydra-node in the explorer is now necessary, and we are currently working on it.

Hydra Doom​

Following the success of Hydra Doom at Rare Evo and as an activation at various events organized by Input | Output (IO) and others(!), the Hydra Doom project has taken a significant step forward with the launch of a tournament offering 100,000 USDM. The prize is intended to be distributed automatically to the winners based on the outcome of the game as measured in the Hydra heads used for the finale.

In launching the tournament, we reached several milestones and pushed Hydra to its very limits in many ways. This included testing the performance of individual Hydra heads, which supported a multi-player game where every frame of each participant’s game session submitted smart contract transactions. At its peak (on December 3), the system processed 1.04 million transactions per second. Over a 24-hour period, it handled more than 15.5 billion transactions in total. This was achieved with a mix of real player traffic from around the world and artificial load generated by bots running the game, contributing transactions to more than 14,000 Hydra Heads.

While more work remains, this marks an impressive milestone, and the overall reception has been positive.

https://github.com/user-attachments/assets/c9546d61-212b-490f-88b8-a3b15229aa15

Incremental commits​

The incremental commits feature is nearly ready for release. As a final step, we invite community members to test-drive this feature. The goal is to ensure the best user experience and identify potential bugs. The remaining tasks include improving documentation and testing and updating the protocol specification to reflect these changes.

Hydra + Midnight​

We’ve been collaborating with the Midnight team to explore how Hydra can align with their goals. As part of this effort, we’re developing a version of Hydra with a custom ledger to support arbitrary built-in operations that validators may need. This work is ongoing.

Links​

The monthly review meeting for November 2024 was held on 2024-12-09 via Google Meet, presenting these slides and this recording.

This is a monthly report on the progress of 🐲 Hydra and 🛡 Mithril projects since September 2024. This document serves as a preparation for and a written summary of the monthly stakeholder review meeting, which is announced on our Discord channels and held on Google Meet. This month, the meeting was held on 2024-10-23 using these slides and you can see the recording here.

Mithril​

Issues and pull requests closed in October

Roadmap​

Here’s the latest on our roadmap:

• Cardano transactions signature/proving MVP #1457: the feature has been activated on mainnet
• Cardano stake distribution certification #955: the feature has been activated on preview, pre-production and is pending activation on mainnet
• Decentralization of signature orchestration #1777: the feature has been activated on preview, pre-production and mainnet
• CIP for Mithril signature diffusion through the Cardano network #1775: the draft CIP has received the number CIP-0137 and is in final review stage
• Protocol usage metrics/statistics #2028: the feature is being implemented and will be released in November.

Distributions​

We released Mithril distribution 2442.0, which includes:

• Stable support for decentralized signature orchestration
• Stable support for Cardano transaction client library, CLI, and WASM
• Stable support for new Pythagoras Mithril era
• Bug fixes and performance improvements.

Future distributions​

We plan to release new distributions in November:

• 2444:
  • Stable clients for Cardano stake distribution certification
  • Stable support for Prometheus metrics endpoint in the aggregator.

Dev blog​

We have published the following posts:

• Certification of Cardano stake distribution
• Certification of Cardano transactions (updated).

Protocol status​

Decentralized signature orchestration​

To enable multiple aggregators on the same Mithril network, the signature orchestration must be decentralized to run independently across all signers and aggregators.

We have implemented and deployed this feature onmainnet with the signer released in 2442.0 distribution:

• The signer and aggregator nodes can independently compute the beacon, which determines the messages to sign and certify
• The aggregator no longer advertises the pending certificates; the pending certificate is deprecated and temporarily kept alive for legacy signer nodes until sufficient adoption of the new version is reached
• The aggregator buffers the individual signatures received from signers until it has computed the associated beacon and will try to aggregate them thereafter.

Aggregator usage metrics and Grafana dashboard​

We have been working on a new Prometheus endpoint for the aggregator, which provides detailed insights about the production of artifacts and certificates, the events received from the signers, and the artifacts and proofs served to the clients. The feature can be easily activated with some configuration parameters.

Additionally, we created a Grafana template to easily set up a dashboard for this Prometheus endpoint.

Hydra​

Issues and pull requests closed in October

Notable updates on our roadmap this month include:

• Added the raw CBOR datum in transaction outputs #1543
• Used Aiken for commit validator #1680
• Updated to cardano-api 9.4 #1706
• Implemented the off-chain user journey for incremental commits #1522 and made good progress on the on-chain validators
• Added Blockfrost mode to the hydra-chain-observer #1631.

Argentina​

IOG attended a local satellite Cardano Summit event, as well as Tech Expo – a multichain Web3-focused event – in Argentina on October 18 and 19. During both events, the Hydra Doom demo was showcased on-site, using locally built cabinets and a new setup with a computer as an on-site server, connecting laptops to aggregate stats from multiple sessions. Many attendees had the chance to play and enjoy the demo, with our very own Tamara Haasen displaying true grit and remarkable skill! Notable attendees included ICP, Polkadot, OKX, ByBit, and the founder of Render Network. Overall, it was a fantastic event and a great opportunity to connect with Argentinian builders. Now it’s time to prepare for the next leg of the Hydra Doom adventure and gear up for a return visit soon!

Aiken for commit validator​

We began porting our validators to Aiken, starting with the Commit validator. This resulted in a saving of over 1Kb to the validator script and increased the maximum number of head participants from 6 to 9.

SnapshotConfirmed has the full Tx​

The SnapshotConfirmed event now has the full transaction information, which makes it much easier to build an app that watches for confirmed transactions and responds appropriately. For example, here is the change required in Hydraw:

diff --git a/hydraw/static/bundle.js b/hydraw/static/bundle.js
index 1bea9e96fe7..326c69c39bc 100644
--- a/hydraw/static/bundle.js
+++ b/hydraw/static/bundle.js
@@ -13,19 +13,20 @@ let n = 0
 client.addEventListener("message", e => {
   const msg = JSON.parse(e.data);
   switch (msg.tag) {
-    case "TxValid":
-      // TODO: Should only draw pixels on SnapshotConfirmed
-      const cborHex = msg.transaction.cborHex;
-      console.log("New transaction cborHex seen", cborHex);
-      const transaction = cbor.decodeFirstSync(cborHex);
-      const auxiliaryData = transaction[3]
-      if (auxiliaryData !== undefined && auxiliaryData !== null) {
-        console.log("Transaction has auxiliary data", auxiliaryData);
-        const aux = auxiliaryData.value;
-        const [x, y, r, g, b] = (aux.get(0) || aux.get(1)).get(metadataLabel);
-        n += delay;
-        setTimeout(() => drawPixel(x, y, [r, g, b]), n);
-      }
+    case "SnapshotConfirmed":
+      msg.snapshot.confirmed.forEach( (tx, _) => {
+        const cborHex = tx.cborHex;
+        console.log("New confirmed transaction cborHex seen", cborHex);
+        const transaction = cbor.decodeFirstSync(cborHex);
+        const auxiliaryData = transaction[3]
+        if (auxiliaryData !== undefined && auxiliaryData !== null) {
+          console.log("Transaction has auxiliary data", auxiliaryData);
+          const aux = auxiliaryData.value;
+          const [x, y, r, g, b] = (aux.get(0) || aux.get(1)).get(metadataLabel);
+          n += delay;
+          setTimeout(() => drawPixel(x, y, [r, g, b]), n);
+        }
+      })
     default:
       console.log("Irrelevant message", msg);
   }

Links​

The monthly review meeting for October 2024 was held on 2024-10-23 via Google Meet, presenting these slides and this recording.

This is a monthly report on the progress of 🐲 Hydra and 🛡 Mithril projects since August 2024. This document serves as a preparation for and a written summary of the monthly stakeholder review meeting, which is announced on our Discord channels and held on Google Meet. This month, the meeting was held on 2024-09-25 using these slides and you can see the recording here.

Mithril​

Issues and pull requests closed in September

Roadmap​

Here’s the latest on our roadmap:

• Cardano transactions signature/proving MVP #1457: the feature is pending activation on mainnet
• Cardano stake distribution certification #955: the feature is pending activation on preview, preprod and mainnet
• Decentralization of signature orchestration #1777: feature implementation is being finalized
• CIP for Mithril signature diffusion through the Cardano network #1775: the draft CIP is in review.

Distributions​

We released Mithril distribution 2437.1, which includes:

• Breaking changes in the Mithril client WASM:
  • Seamless transition from unstable to stable features
  • A new unstable option in the client allows using unstable features
  • The previous client.unstable implementation is not supported anymore and must be replaced with client
• Stable support for Cardano transaction certification in signer and aggregator
• Stable support for Cardano stake distribution certification in signer and aggregator
• Bug fixes and performance improvements.

Future distributions​

We plan to release new distributions in October:

• 2440:
  • Stable clients for Cardano transaction certification
  • Activation of the certification of Cardano stake distribution in pre-release-preview, release-preprod, and release-mainnet
  • Stable support for new Pythagoras Mithril era
• 2443:
  • Stable clients for Cardano stake distribution certification.

Dev blog​

We have published the following posts:

• Mithril client WASM breaking change
• Certification of Cardano transactions (Updated).

Protocol status​

Decentralized message queue for Cardano​

We continued working on the decentralized message queue (DMQ) for Cardano, as proposed in this CIP draft.

The DMQ is designed to:

• Leverage the Cardano network layer
• Initially be used by Mithril for the diffusion of signatures from signers to aggregators
• Be adaptable for future Cardano protocols, each operating on a different DMQ topic.

The DMQ will be operated by a 'side' node, known as the DMQ node, which will run a separate peer-to-peer network powered by the Ouroboros network stack. Here are some key technical details:

• It will run as an external process that exposes a Unix socket to support node-to-client mini-protocols
• It will implement node-to-client and node-to-node mini-protocols to allow for message local submission, local notification, and peer-to-peer submission
• It will retrieve the Cardano stake distribution from its local Cardano node to authenticate incoming messages
• It will retrieve other peers' information about the peer-to-peer network from its local Cardano node (this topic is being investigated: either with a ledger hard fork, redirecting connections to the DMQ node, or leveraging the SRV record in the SPOs' DNS)
• Each topic of the message queue will run a different DMQ node instantiated with a specific configuration.

Producers and consumers exist for the DMQ topic running on some DMQ nodes. In the case of Mithril:

• The Mithril signers will be the producers of the message (Mithril individual signatures)
• The Mithril aggregators will be the consumers of the message (Mithril individual signatures).

DMQ producers and consumers run on various DMQ nodes:

Hydra​

Issues and pull requests closed in September

Notable updates on our roadmap this month include:

• Released version 0.19.0 introducing Conway ledger support with compatibility for Babbage transactions #1608
• Completed the Hydra 'head-in-head' spike #1590
• Investigated Raft consensus for networking #1591
• Added 'HeadId' into the 'Greetings' message #1557
• Implemented the initial suite of network-resilience tests #1532
• Changed network semantics to broadcast to everyone #1624.

Spike: head-in-head​

When a Hydra head is opened, part of the underlying ledger state gets locked and made available off-chain to a small group of participants. While this already creates a 'small world' to process transactions (which makes it fast and cheap), we encountered use cases where it would make sense to open additional heads with different or even smaller groups of participants on parts of the layer 2 state, forming layer 3 heads.

One example of this use case is the Hydra Doom demonstration we presented at Rare Evo last month. For example, individual multiplayer game sessions require low network latency from regionally close machines. If those machines run a game session head that, in turn, locks funds and game state from a slower global tournament head instance, we are discussing Hydra heads in heads.

This construction raises many open questions about liveness: what happens to the layer 3 head if the layer 2 head stops processing transactions? Such pessimistic scenarios can occur in any optimistic protocol and must be addressed, though not necessarily optimized for. Despite these and other uncertainties, we spent time this month exploring whether it’s even possible as part of the spike issue #1590.

Technically, this means that a hydra-node communicates with another hydra-node as its 'chain backend', interpreting snapshots of the layer 2 head as blocks, observing transactions that open or close the head, and submitting its state transition transactions to the underlying layer 2 ledger. The spike issue provides more details and instructions on reproducing these findings. The following issues demonstrate this prototype --inception mode:

Hydra 🤝 Blockfrost: chain backend concept​

As outlined in #1305, we are considering an option to run the hydra-node in a more lightweight mode, without requiring the full cardano-node. This feature is particularly relevant when considering the use of Hydra with the pay-per-use Blockfrost API.

To complete this component, the Blockfrost chain layer must be capable of:

• Following the chain
• Submitting Hydra transactions
• Handling relevant internal wallet queries for the hydra-node.

As an initial step, we have developed:

• A Hydra chain observer that operates in Blockfrost mode (although it has not yet been integrated into the hydra-node)
• A variant of the hydra-explorer tailored to the Blockfrost-enabled Hydra chain observer.

To achieve this, we have used a straightforward roll-forward approach via the Blockfrost HTTP API, relying on the following key API calls:

• GET /blocks/{hash}
  • confirmations: number of block confirmations
  • next_block_hash: (nullable) hash of the next block
• GET /blocks/{hash}/txs
• GET /txs/{hash}/cbor

Basically, we continuously fetch specific blocks by their hash, using the number of confirmations to indicate the block's safety (minimizing the likelihood of a rollback). From there, we roll forward using the reference to the next block hash.

While this mechanism might appear simplistic, it is highly effective for most use cases except scenarios involving exchanges.

To further optimize the performance of this new chain component, we’ve raised two new issues on the Blockfrost side to support our work:

• #209: add an endpoint to fetch all transaction CBORs in a block, reducing the number of API calls needed to retrieve Hydra head observations from block transactions
• #67: implement concurrent fetching to fully leverage parallel requests and enhance performance.

Moving forward, our next objective is to enable the hydra-node to publish Hydra scripts via Blockfrost, as outlined in #1668.

Spike: Raft-based networking​

Last month, we developed a new test suite #1532 to assess the resilience of our network stack. Now that this is in place, we can start exploring different approaches to achieve our goal of a crash-tolerant network layer.

Recently, we stumbled upon this fairly old research paper that explored various consensus protocols used in blockchain space. It reminded us of the correspondence between consensus and broadcasts:

The form of consensus relevant to blockchain is technically known as atomic broadcast

Additionally, it mentioned at least one early permissioned blockchain that achieved crash tolerance of t < n/2 by using etcd with its off-the-shelf Raft consensus algorithm. This prompted us to explore the possibility of replacing our custom network stack with this (arguably overkill) alternative in an experiment #1591 to evaluate its performance.

The GitHub issue contains all the details, but it turns out that this idea might not be as exotic as we initially thought! Implementing broadcast functionality as put requests to the etcd cluster works well when combined with an outbound, persisted buffer that handles failed writes while disconnected (or only connected to a minority). Additionally, the revision mechanism of etcd and storing the last known revision on disk allow us to create a hydra-node that is fully resilient to crashes and network failures, as demonstrated in these fault injection test runs.

Even the performance of this early prototype matches or exceeds our current implementation, especially when multi-threading performance is available. For example, using a low-powered container hosted on GitHub, we observed average confirmation times on the three-node benchmark decrease from 20ms to 100ms, while on a desktop machine with 8+ cores, the same benchmark improved from 100ms to 50ms. Since the starting numbers of the released version also vary widely between machines, these results are only indicative.

Incremental commits​

The incremental commits feature is being developed, and we demonstrated the envisioned user workflow during the monthly meeting.

Notably, users will first need to lock their UTXO in the Hydra deposit script. Once we have a signed snapshot on layer 2, the increment transaction will consume this output. Eventually, this will make the specified UTXO part of the head UTXO state on layer 2. In case of any issues, any Hydra node will be able to post a recovery transaction to unlock the UTXO.

The hydra-node API includes endpoints for depositing and recovering UTXOs, providing a convenient way to build and post these transactions. We aimed to create a user-friendly experience for these actions.

Next steps involve implementing comprehensive on-chain security, along with documentation and tutorials, to ensure our users understand how to use this new feature. This is essential since many builders on Hydra have requested this feature.

Links​

The monthly review meeting for September 2024 was held on 2024-09-25 via Google Meet, presenting these slides and this recording.

This is a monthly report on the progress of the 🐲 Hydra and 🛡 Mithril projects since July 2024. This document serves as both preparation for and a written summary of the monthly stakeholder review meeting, announced on our Discord channels and held on Google Meet. This month, the meeting took place on 2024-08-28, using these slides and you can see the recording here.

Mithril​

Issues and pull requests closed in August

Roadmap​

Here is an update on our current roadmap:

• Cardano transactions signature/proving MVP #1457: the feature is finalized and the last step to completion is to release on the mainnet.
• Cardano Stake Distribution certification #955: the feature is finalized and the last step is to release on preview, preprod and mainnet.
• Mithril signature diffusion with Cardano network layer PoC #1837: the proof-of-concept is completed.
• Decentralization of signature orchestration #1777: we have started implementing the feature with some preliminary works.
• CIP for Mithril signature diffusion through Cardano network #1775: the draft CIP has been published on the Cardano foundation CIPs repository and is under review.

Distributions​

We have released the new Mithril distribution 2430.0. This distribution includes several critical updates and enhancements:

• Support for Cardano node 9.1.0
• Support for Cardano transactions certification in release-preprod and pre-release-preview
• Bug fixes and performance improvements.

Future distributions​

We plan to release new distributions in September following the Chang hard fork:

• 2436:
  • Activation of the certification of Cardano transactions in the release-mainnet network
  • Activation of the certification of Cardano stake distribution in pre-release-preview, release-preprod, and release-mainnet.

Protocol status​

Cardano stake distribution certification​

We have completed the implementation of the Cardano stake distribution certification with Mithril and are currently preparing for the rollout of this feature to production:

• We have implemented a new type of data certified for the latest snapshot of the Cardano stake distribution done by the Cardano node at the end of each epoch.
• Under the hood, we use a Merkle tree representation of the stake distribution and sign its Merkle root.
• The Mithril clients (library, CLI and WASM) have the capability to download and verify a snapshot of the Cardano stake distribution.
• We are preparing to activate the feature in the release-preview, release-preprod, and release-mainnet networks with the release of the next Mithril distribution.
• We have created documentation and some example code implementation for developers.

Hydra​

Issues and pull requests closed in August

Notable updates on our roadmap this month include:

• Incremental decommits, Conway support fixes, and more were released in 0.18.0
• New landing page, SSL support, and bug fixes around transaction pruning were released in 0.18.1
• Incremental commits (coming in a 0.19 release) have pivoted to a deposit-based scheme based on open discussions (thanks @GeorgeFlerovsky!)
• Ready to shift ledger to Conway when hardfork is complete #1338
• Settled on Partial Fanout as a good solution to a few known problems
• Initial work on a suite of network-resiliance tests #1552
• The Hydra Doom demo yielded several ideas that need to be refined into features

Hydra Doom​

At the Rare Evo event in Las Vegas, we showcased the Hydra Doom demo, a technology demonstration of Hydra where users could play the 1993 id software game Doom and for each frame of the game, a smart contract transaction was submitted to a Hydra head which recorded game state, user inputs, and validated game state transitions (as a proof of concept, using only some minor logic pertaining to the movement of the player and dying in-game).

People could connect remotely via https://doom.hydra.family or in person on two custom-made arcade cabinets we had constructed for the occasion. Each game session contributed 35 transactions per second. In aggregate, we processed 106 million transactions with Hydra during the 78 hours of operation, which is more than the entire history of Cardano mainnet. The peak was around 7,200 tps.

The demo was very positively received on social media and seems to have rekindled the flames around the Hydra narrative both internally and externally, which hopefully bodes well for both future adoption interest among builders, as well as the prospects of securing funding for the continued development of Hydra from Intersect.

Network tests with fault injection​

We have set up a simple yet effective workflow capable of simulating a Hydra cluster under high transaction loads and packet loss network failures.

With this, we aim to consistently reproduce and resolve issues that are causing the head to become stuck, as outlined in #1436.

The workflow specifies a set of high-load scenarios, each of which spins up the same Hydra cluster on devnet used during the getting started tutorial and injects network faults using Pumba.

The idea is to allow and encourage everyone to experiment with this workflow and collect summary results, which contain details about the performance within the head. Because these tests are performed with the hydra-node being a 'black box,' it also allows us to explore alternative network stacks as a whole.

New landing page​

For the Rare Evo event with the public Hydra doom demo (see above), we also figured it would be a good idea to improve the first impression new visitors get from the https://hydra.family website.

The marketing team at IOG collaborated on creating new content and design for the landing page. In addition to highlighting the key features and explaining the importance of the Hydra Head protocol, the updated landing page also includes an analogy of how Hydra works, using a cargo plane as an example.

The original design included an alternative color scheme using Teal as the primary accent color and Plum as the secondary. An example is included below. Do you think a new color scheme would be appealing or do you prefer the Purple main color as-is? Shoot us a message on Twitter (X) or #ask-hydra on Discord with your thoughts.

Bug fix: layer 2 transactions during decommits​

After releasing the decommit feature that enables users to transfer funds from layer 2 to layer 1 while the head is operational, we quickly identified a bug. When a decommit is pending ( funds have not yet been withdrawn from the head) and a user attempts to create a new layer 2 transaction, the hydra-node attempts to re-apply the pending decommit to the local ledger state, resulting in an error.

We promptly realized that any pending decommit needs to match with the decommit in the new snapshot and should be preserved in the next snapshot(s) until it is observed. Without this fix, our decommit feature would be extremely fragile, so we made sure to quickly draft a failing test and do a proper fix.

This is a pull request that fixes this bug and we also demonstrated this fix in one of our monthly meetings.

Conclusion​

The monthly review meeting for August 2024 was held on 2024-08-28 via Google Meet, presenting these slides and this recording.

This month we saw demonstrations from both projects about new features and improved testing capabilities. Optimization results and test results underline the teams' commitment to evidence-based engineering. The Hydra Doom demo was a great success and we received good feedback from users and use cases.

While roadmaps have not moved much over the last couple of months, but more re-orientation is likely as plans for 2025 are coming together in the working groups and other Intersect committees.

This is a monthly report on the progress of 🐲 Hydra and 🛡 Mithril projects since June 2024. This document serves as a preparation for and a written summary of the monthly stakeholder review meeting, which is announced on our Discord channels and held on Google Meet. This month, the meeting was held on 2024-07-22 using these slides and you can see the recording here.

Mithril​

Issues and pull requests closed in July

Distributions​

2423.0​

We have released the new Mithril distribution 2428.0. This distribution includes several critical updates and enhancements:

• Support for Cardano node 9.0.0.
• Database cleanup and optimization once per epoch.
• Bug fixes and performance improvements.

Future distributions​

We have planned to release two new distributions in August:

• 2430 (before the Chang hardfork):
  • Activation of the certification of Cardano transactions in the release-preprod network.
  • Support for Cardano node 9.1.0 on all networks.
• 2434 (after the Chang hardfork):
  • Activation of the certification of Cardano transactions in the release-mainnet network.

Protocol status​

Here is the status of the Mithril protocol on the Cardano mainnet:

Transaction verification in Nami​

As Blockfrost now supports the Mithril endpoints within their API, they have presented an implementation example.

• The demo shows transaction verification within the Nami light wallet (using Blockfrost as its backend).
• This allows the light wallet user to verify that their transaction is on-chain with the high certainty provided by Mithril security.
• While it is just an experimental implementation, it might be accepted by the Nami upstream in the future.

Transaction certification​

We have continued working on the implementation of the Cardano transaction certification with Mithril and are currently preparing for the rollout of this feature to production:

• We are finalizing the minimization of the signer footprint on the SPO infrastructure.
• We are preparing for activating the feature in the release-preprod network with the following release.
• We are preparing thorough documentation about the mechanism used to certify the transactions
• We have selected final parameters for signing the transactions:
  • 100 blocks from the tip of the chain (~30 minutes).
  • 30 blocks between snapshots certifying the transactions (~10 minutes).
  • With these parameters, we can consider the transactions final with very high probability.
  • These parameters are hot parameters that can be adjusted without requiring a new distribution

Decentralized Message Queue CIP​

We have been collaborating with the Cardano networking team to design a mechanism for implementing a decentralized message queue to decentralize the signature diffusion from Mithril signers to Mithril aggregators based on the Cardano network layer. The idea is to create a separate process for the decentralized message queue implementation:

• A new dedicated process (aka "Mithril network node") based on the Ouroboros network stack
• More efficient handling of resource consumption (network and CPU) and reduced impact on the Cardano node performance and availability
• Enhanced security with no impact on the Cardano node
• Different release pace for both nodes
• Both nodes will be bundled in the Cardano node distribution release
• A hard fork for SPO registration of the port of their Mithril network node is likely needed.

Hydra​

Issues and pull requests closed in July

Notable updates on our roadmap this month are:

• Completed incremental decommits, to be released separately as 0.18.0 after some remaining cleanup tasks and documentation updates
• Release 0.19.0 will be incremental decommits and, depending on the hard-fork date of Cardano mainnet, switching to Conway on the Hydra L2 ledger (see March 2024 for more details)
• Refined scope of "what is Hydra V1" through the Hydra working group
  • Added SDK for wallet integration with Hydra #1509 to provide a better developer experience
  • Decided to implement Directly open heads #1329 and Partial fanout to overcome our known issue & limitations and supersede several other 💭 ideas.
  • Prioritized Optimistic head closure #198 higher based on user feedback.
  • Declared React to protocol parameter changes #195 as not essential for a V1.

Incremental decommits completed​

This month we finally finished the implementation of the Incremental decommit #1057 feature.

As evident from the GitHub feature description, the number of comments and linked pull requests, this was a complex change.

Last month, we discussed why versioning the open state on-chain is necessary and the subsequent changes to the specification and implementation. After addressing these in hydra#1473 and in hydra#1474, a thorough review and requiring tests for all changes (compared to master) in hydra#1483 concluded in a last cleanup PR, which made the feature consistent in specification, implementation and documentation.

In the monthly review meeting we demonstrated the feature and the video below also shows how it works using a small extension to the hydra-tui:

Repository moved​

This month, the Hydra repository was moved to the cardano-scaling GitHub organization: https://github.com/cardano-scaling/hydra.

This allows for a more flexible use and sharing of GitHub runners between the main repository and for example hydra-explorer.

Most links should be automatically redirected, but if you encounter broken links, just reach out on Discord or create an issue 🙏 https://github.com/cardano-scaling/hydra/issues/new/choose

Working group updates​

There were two Hydra working group meetings this month and multiple people showed interest in joining. Current member list can be found in the charter and meeting notes of all meetings are kept in the repository too.

The working group started work on the first item "What is Hydra Head V1" and gathered valuable feedback in discussions between fellow layer 2 architects and Hydra users.

Notable inputs were the need for an SDK, wallet integration and the ability to operate without a full cardano-node, which got captured in hydra#1509 and hydra#1305 respectively. Even brand new ideas emerged from these discussions like "resumable Hydra heads".

The immediate goal is to achieve a common understanding of what must be included for a Hydra Head V1 and consequently when/how this could be audited?

Discussions also revolved around discoverability of Hydra heads, specifically, but also general interoperability with off-chain ledgers (any layer 2). In that regards, the group is considering to work on a CPS/CIP in the mid term.

Conclusion​

The monthly review meeting for July 2024 was held on 2024-07-22 via Google Meet, presenting these slides and this recording.

Despite the likely impact of summer breaks on attendance, it was great to have Blockfrost demonstrate their experiments using Mithril certificates delivered through Blockfrost in a forked Nami version. This demo clearly illustrates how light clients can benefit from Mithril verifiable transactions.

On the Hydra side, we are very pleased to have successfully merged and prepared the incremental decommits feature for release. Although this process took some time, it marks the first substantial feature not originally covered by the Hydra head paper, demonstrating the protocol's capacity to grow and meet user needs. We appreciate all the valuable input from the Hydra working group regarding the features we should develop next and are excited to see the growing community interest in Hydra's future.

This is a monthly report on the progress of 🐲 Hydra and 🛡 Mithril projects since May 2024. This document serves as a preparation for and a written summary of the monthly stakeholder review meeting, which is announced on our Discord channels and held on Google Meet. This month the meeting was held on 2024-06-24 using these slides and you can see the recording here.

Mithril​

Issues and pull requests closed in June

We have released the new Mithril distribution 2423.0. This distribution includes several critical updates and enhancements:

• BREAKING changes in the Mithril client CLI:
  • The deprecated snapshot command has been removed
  • It has been superseded by the cardano-db command
• Bug fixes and optimizations.

We have also created a developer blog post about the removal of the Mithril client CLI 'snapshot' command.

Additionally, we have released a draft version of the Mithril threat model analysis. We expect to receive external feedback and contributions before considering it final.

Transaction certification​

We kept working on the implementation of the Cardano transaction certification with Mithril and made good progress with the development:

• Transactions are now retrieved with the chain sync mini-protocol using the Pallas library
• The performance of the prover route of the aggregator has significantly improved.

Low latency certification​

The transactions are imported using the native chain sync mini-protocol of the Cardano node. This has been made possible with the Pallas library, which fully supports this mini-protocol. This allowed us to retrieve transactions much closer to the tip of the chain and at more frequent intervals. To achieve this, we have enhanced the transaction importer to handle chain rollbacks, where some blocks previously recorded may be discarded at a later time, with an increased probability as we get closer to the tip of the chain.

We are currently calibrating the system parameters (depth from the tip and certification pace) before activating it on the mainnet.

Increased prover performance​

We have identified some bottlenecks in the computations of the merkelized proof for transaction membership of the transaction set as well as in the database access. These bottlenecks have been fixed, allowing us to achieve a new order of magnitude in the performance of the prover route. It is now able to consistently deliver 10,000 transaction hashes certification per second.

Mithril/Cardano integration​

We have been working with the Cardano networking team to design a closer integration of Mithril into Cardano. In particular, we have focused on decentralizing the signature diffusion from Mithril signers to Mithril aggregators based on the Cardano network layer.

Here is the current roadmap:

• Assessing Cardano network bandwidth capacity to support Mithril signature diffusion
• Designing a node-to-node mini-protocol for signature diffusion
• Designing two node-to-client mini-protocols for signature submission and notification
• Preparing the submission of a CIP draft to the community.

Hydra​

Issues and pull requests closed in June

Here is a high-level overview of the team's work over the past month:

• Incremental decommits. We continued our work on incremental decommits, focusing on investigating adversarial attacks on the head script contract.
• Specification update. We updated our specification with the latest changes following our exploration of adversarial attacks. These updates aim to enhance the security of the protocol.
• Agda specification for the Hydra protocol. Work has begun on writing an Agda specification for the Hydra protocol. You can follow the progress here.
• Incremental commit. We discussed various options for the incremental commit work, including potential designs for a hydra-ledger to improve system efficiency and security.
• Node compatibility. The node has been successfully updated and is now compatible with cardano-node version 8.11-pre.
• Website. We updated our website tutorials and documentation to enhance expressivity and ease of use.
• Committing internal wallet UTXOs. We made breaking changes to the hydra-node API /commit endpoint #1463 to remove the check that prevented committing UTXOs from an internal hydra-node wallet.

We have not planned the next release, so our roadmap remains largely unchanged. Our current focus is on incremental decommits and commits and specifying our protocol in Agda.

Incremental decommit​

We were about to release the incremental decommit feature when we discovered a bug during a demo. This bug related to handling the decommit snapshot in close/fanout. When one decided to decommit funds from the head and everyone signed the decommit snapshot, the same snapshot was used to close/fanout. If this snapshot contained some UTXOs to decommit but one already decommitted funds from the head, the fanout would distribute the decommit funds again.

This issue was significant, and although we have many tests around the decommit feature, we had not checked user balances upon head closure.

We have implemented a fix that involves adding versioned snapshots. This allows us to distinguish between two cases:

• The decommit snapshot was signed, but the decommit was never observed
• The decommit snapshot was signed, and the decommit was also observed.

When we observe the decommit transaction locally, we increment the version in the local state. This allows us to compare the snapshot version against the expected local version on-chain.

We also needed to keep extra data in the Close redeemer to successfully check for a valid snapshot signature while altering the close datum to remove the UTXOs to decommit if the versions do not match. This ensures that the fanout behaves correctly, as it is only concerned with the close datum and the actual transaction output hashes.

The remaining work involves updating the specification with the changes in the implementation. We expect to release this feature in the coming days.

Community​

Subbit.xyz​

Algy Wallis presented 'Subbit', a layer 2 solution focused on subscription-based services, aimed at addressing the complexities of building blockchain products. Subbit is designed to be easier to integrate and manage compared to layer 1 solutions, reducing latency and transaction fees. The system allows providers to verify consumer IOU notes, facilitating periodic payments and withdrawals with minimal on-chain interactions. This model mirrors Bitcoin's Lightning Network and aims to enhance scalability and interoperability for subscription services on Cardano.

Hydrozoa​

George Flerovsky introduced the Hydrazoa proposal, currently in Catalyst Fund12. Hydrazoa aims to enhance the existing Hydra framework by adding dynamic capabilities for incremental commits and decommits, along with the ability to add or remove participants in state channels while they are still open. The goal is to simplify layer 1 scripts and lower costs, with fallback mechanisms similar to Hydra's in case of layer 2 consensus failure. George highlighted the differences and potential integrations with Hydra, emphasizing a more flexible and simplified approach to state channel management.

Technical working groups​

Reza Baram provided an update on the Hydra working group, which recently had its first meeting to discuss its vision and collaborative processes. The primary goal is to create an environment for sharing knowledge and working towards scaling Cardano beyond just Hydra. The group is currently focused on defining and delivering Hydra version one, prioritizing incremental commits and robustness improvements. Feedback from the community and working group members will be sought to refine and enhance the roadmap, ensuring effective scaling solutions for Cardano applications. The Hydra working group is open to new members and contributions from the community. Read more here.

Conclusion​

The monthly review meeting for June 2024 was held on 2024-06-24 via Google Meet, presenting these slides and this recording.

This is a monthly report on the progress of 🐲 Hydra and 🛡 Mithril projects since April 2024. This document serves as a preparation for and a written summary of the monthly stakeholder review meeting, which is announced on our Discord channels and held on Google Meet. This month the meeting was held on 2024-05-28 using these slides and you can see the recording here.

Mithril​

Issues and pull requests closed in May

We have released the new Mithril distribution 2418.1. This distribution includes several critical updates and enhancements:

• A breaking change introduced in the Mithril client / Mithril client CLI:
  • The certificate chain structure has been modified to remove coupling with the immutable file number
  • The client must be updated to verify the certificate chain
• We have switched the memory allocator to jemallocator on the signer and the aggregator to avoid memory fragmentation
• We have enabled the BLST portable feature by default to benefit from runtime check of intel ADX instruction set.

Also, we have started the process of removing the deprecated snapshot command from the client which is superseded by the cardano-db command. This change will be effective with the release of the next Mithril distribution.

Transaction certification​

We have kept working on the implementation of the Cardano transactions certification with Mithril and we have achieved a new milestone with running the certification in a test network (aka testing-mainnet) which operates on the Cardano mainnet.

This allowed us to refine our roadmap with clear objectives before the MVP can be released:

• Keep working on the low latency implementation of the certification
• Optimize the aggregator prover route to reach good throughput
• Optimize the signer footprint to limit the impact on the SPO infrastructure
• Optimize the signer and aggregator warmup to avoid network instability/disruption once the MVP is released.

A first optimization of the prover route has been implemented, drastically improving performance. By implementing resource pooling on the Merkle tree that signs transactions by block range, we have achieved a 100x increase in throughput.

We have also implemented a pruning mechanism on the signer that retains only the transactions needed to compute upcoming signatures. This has reduced the storage requirements from 32GB to 100MB.

In/Out SPOs dashboard​

We have added a new page to the Mithril explorer that lists the newly registered and recently de-registered signers in the Mithril protocol.

The protocol insights dashboard​

A new Mithril protocol insights dashbord has been released. It provides metrics such as the number of SPOs and total stake involved, the daily number of Cardano database restorations, and the breakdown of running signer versions.

Feel free to request some new metrics on the dashboard!

Hydra​

Issues and pull requests closed in May

This month, Hydra version 0.17.0 was released which includes

• breaking changes of the /commit endpoint, yielding full flexibility when committing from scripts (see this how-to for details)

• new GET /snapshot/utxo HTTP endpoint

• detecting network protocol mismatches

• Full release notes and a list of delivered features

Besides corresponding features being released through 0.17.0, the roadmap did not change much this month. Current focus is on incremental decommits and commits, while we also wait for the Conway hard-fork to happen before we can switch the L2 ledger also to Conway (see March report for more details).

Transaction trace testing​

In the process of implementing the incremental decommits feature, we constantly considered how to ensure our test suite covered all potential scenarios. For example, what happens if we request a decommit, have a signed snapshot, but never post a decommit transaction? Or how do we handle close and fanout with decommit UTXO in the snapshot? There are various scenarios we are interested in testing, and we quickly realized that we would need to test at different levels to achieve the coverage we expect, especially for our on-chain code.

Our mutation test suite was also very helpful in testing whether our on-chain code produces the expected errors when manipulating the transaction to make it invalid. However, this framework tests individual transactions, and we needed to test if multiple transactions in sequence produce valid results.

To address this, we decided to utilize quickcheck-dynamic once again to model simple snapshots, which are used to construct Hydra transactions. Our hope was that if the model is simple enough, we would be able to reason about it without delving into many details and decide if a particular UTXO/snapshot should be valid in the current transaction context or not. In the quickcheck-dynamic world, our 'Action' contains snapshots, which are then used to construct different transactions, evaluate, and observe them. Our model keeps the extra information we need to evaluate these transactions, and all snapshots are signed by everybody to keep things simple.

This test suite has already exposed some bugs we had and enabled us to reason about our code using a simpler model without burdening us with real-world details, which are more involved and harder to reason about.

Hydra-related Catalyst proposals​

Catalyst Fund12 is in full swing, with over 30 proposals this round mentioning 'Hydra' in one way or another. While some directly draw inspiration from Hydra and tackle similar challenges, others aim to enhance transaction processing scalability but take different approaches (such as rollups and sidechains). Additionally, some proposals focus solely on scaling payment transactions, offering refreshingly simple solutions.

After a quick screening, we also engaged with some of the proposal authors this month to address questions about Hydra, discuss similarities, and explore opportunities for mutual learning and collaboration:

Use cases​

• Wine Supply Chain Tracking and Reporting System. Great use case that requires the seamless integration of recording data as transactions off-chain, with resulting states and tokens accessible from the Cardano layer 1. The team is already developing a proof of concept using Hydra, and this proposal aims to advance it into a minimum viable product (MVP).

• Ikigai + MLABS - Grabbit / Hydra auctions: Running auctions within Hydra heads is a popular use case, allowing for fast and cost-effective bids on layer 2, while the final purchase of auction items seamlessly occurs on the Cardano layer 1. This proposal, in particular, will utilize incremental commits and decommits to facilitate fully-backed auctions on layer 2.

Using / extending hydra​

• Hydra-Enabled Micropayments System. The same team behind the wine supply chain tracking system wants to also provide a decentralized payment system that offers great point-of-sales experience using Hydra at its core. While they aim to use Hydra Head as a core component, this would also be a great use case for the more payment-focused variants in the long run (see below).
• Open-source Hydra Head L2 Web-based Explorer and API. Improving the visibility of what's going on in a Hydra Head is a great idea. Besides exploring what heads are available on the network, this would make transactions inside a head more visible and is very much related to this idea on our roadmap.
• Cardano Layer 2: Hydrozoa protocol for lightweight and flexible Hydra Heads for Cardano. An exciting re-interpretation of the core Hydra Head protocol and ambitious simplifications of roadmap features like directly open heads, incremental de-/commits and optimistic head closure. This was also discussed in the past with core contributors, and we would love to explore how those protocol variants can find their way into the upstream Hydra Head protocol.

Alternative scaling approaches​

• Subbit.xyz : Cardano's featherweight L2. Refreshingly simple take on how to process payments off-chain. Very much inspired by lightning payment channels and maybe even Chaumean eCash systems. Having payment channels that work in such a simple and flexible way is exciting and could be a great basis for any payment channel networks, ie Cardano Lightning.
• µgraph: Instant, Untraceable Payments in Cardano. Ambitious, but promising, take on what the Hydra tail could be – an asymmetric construction that benefits individual users by using zero-knowledge proofs to ensure correct operation of the node.
• Anastasia Labs Cardano Layer2 - Midgard. A layer 2 construction, similar to Hydra Head, that seeks to utilize the Cardano ledger isomorphically, but operates with a limited validator set and employs a distinct consensus mechanism to offer an experience akin to sidechains. We always wondered what you would get if you ran a Hydra Head with a non-total consensus and checkpointing state on layer 1!

Technical working groups​

As the age of Voltaire unfolds, it fosters the formation of community alliances and member-based organizations. At the same time, there's a popular belief that governance over core Cardano projects should become more open, with decisions regarding the inclusion or exclusion of elements on the roadmap of core infrastructure being transparent.

The current strategy devised by Intersect and its members to address this involves establishing technical working groups that operate openly and transparently on roadmaps and high-level backlog items. This approach mirrors what we're already doing here (see also last months starmap update section), but with a more concrete democratic approach to ratifying a roadmap. Presently, roadmaps for both Hydra and Mithril projects are expected to be governed in this manner.

Specifically for the Hydra working group, initial steps were taken this month by drafting a barebones charter and brainstorming first agenda items to engage potential members and get first sessions going as we gather more interested stakeholders. The charter and any future outputs of the working group will be hosted on cardano-scaling/wg-hydra and we also registered the working group at Intersect.

Conclusion​

The monthly review meeting for May 2024 was held on 2024-05-28 via Google Meet, presenting these slides and this recording.

May was a short month because of the public holidays and some people being off, but we still made good progress on both projects. The new dashboards for Mithril will visualize existing participation and surely garner more interest in the protocol. We are also excited about the progress on the Cardano transactions certification with Mithril, especially as multiple projects building bridges or layer 2s are keeping a close eye on this feature.

While we are busy on the Hydra side wrapping our head around incremental decommits and ensuring they are correctly implemented through transaction trace testing, we are also excited about the upcoming Catalyst Fund12 proposals and the discussions coming along with it as others are also looking into building similar, yet different scaling solutions and tackle the same challenges.

We are also making progress on making Hydra and Mithril less IOG-driven, but more community-owned through working groups and Intersect. We are excited to see how this will turn out.