Skip to main content

September 2023

· 9 min read
Sebastian Nagel

This report summarizes the work on Hydra since August 2023. It serves as preparation for the monthly review meeting (see slides and recording), where the team updates project stakeholders on recent developments to gather their feedback on proposed plans.

Roadmap​

This month, the roadmap has not changed much as the team mostly clarified upcoming features.

The roadmap with features and ideas

The latest roadmap with features and ideas

Notable updates​

Cardano Scaling workshop in Nantes, France​

From 2023-09-12 to 2023-09-14, most of the Scaling Tribe members, that is the two Hydra and Mithril teams, met for a three-day workshop in the beautiful French city of Nantes. This workshop was run following the Open Space technique which makes room for every participant to experiment and focus on what's of interest for them at the time while being engaging for the whole team. Here is a selection of topics covered that were most relevant to Hydra.

Aiken validator experiment​

As part of thw workshop, we explored how hard it would be to implement one of the Hydra validators in aiken. After roughly two hours of work we had a hydra-node running, tests passing and benchmarks of verifying one step in the Hydra protocol using aiken.

The hydra-node just uses the UPLC binary and its hash to construct transactions, we could rely on the same interface, with the difference that the script binary gets produced by aiken. Aiken does produce a socalled blueprint plutus.json containing the contract. We staged that file into git and embedded it into a Haskell interface to be able to reference the compiled script as SerialisedScript from hydra-node.

Next, we ported the simplest contract we have, namely the Commit validator, to Aiken. This was fairly straight-forward and we test were passing after some fiddling with internal representations of data strctures (plutus-tx is encoding triples as Constr data on-chain).

The results? A significant reduction in script size, cpu and memory usage on collect and abort transactions - or in different words, doubling the supported number of parties in a Head. 🎉

Shallow cardano-node experiment​

Running a hydra-node requires running a cardano-node, and running a cardano-node requires synchronisation with the chain and about 150 gigabytes of data to be downloaded and verified, which is a significant barrier to quickly spin-up hydra-node and Hydra Heads. Given a Hydra node does not need to look at the history of the chain, a Shallow cardano-node that can run without the full database of immutable files would be fine, assuming we can retrieve a recent state of the chain with the same security guarantees than a fully synchronised chain. This is exactly what Mithril can provide!

We ran a small experiment, trying to start a cardano-node manually removing all but the last files from the immutable DB and it worked as long as one does not query the node for a block too old in the past. This lead us to envision an improved user experience for installing and running a hydra-node, by packaging it with a cardano-node and a mithril-client to retrieve only the minimum needed state and be able to start a fully functioning hydra-node in minutes.

Other topics​

  • We continued working on Kupo x Hydra integration, benefiting from the presence of Kupo's creator and former Hydra team member, Matthias Benkort
  • We brainstormed possible synergies between the Hydra and Mithril projects, something that solidified into the desire to explore this concept of "Shallow node" and its integration within Hydra
  • We discussed and clarified our new Security Vulnerability Disclosure Policy
  • Discussing the idea of a joint documentation effort, we realised the existing documentation of both teams could be improved, for example by following a Standard documentation framework.

Hydra development​

Issues and pull requests closed since the last report

This month, the team worked on the following:

Network resilience to disconnects #188​

If a Hydra node crashes, it can recover its state from disk thanks to the event-sourced persistency mechanism in place. But if a peer inside the head sends messages while the node is down, these messages are lost for this node and this can well lead to the head being stuck.

For instance, in the following picture, we see that if Bob was down when Alice acknowledged snapshot 10, then it will wait forever for this acknowledgement and will refuse to move forward with the head, making it stuck.

Head stuck because of lost message

Head stuck because of lost message

Hydra is designed to deal with non-cooperative peers in a very safe way: close the head! And this situation could be generalized as a non cooperative peer situation. So the safe move is to just close the head.

But practically speaking, closing the head every time a node restarts or a network connection is interrupted can lead to an operations nightmare. So the question we're exploring is, can we distinguish between non cooperative peer or just transient communication issue?

  • ADR 27 gives details about our current strategy
  • #1074 is a first implementation of this strategy
  • #1080 should specify the resulting network protocol so that others could implement compatible hydra nodes

Incremental commits and decommits #199​

In the beginning of this month, some progress on incremental commits and decommits was made. An early experiment about the off-chain / node logic showed that we can re-use the /commit validator and motivated design work on the feature items.

As mentioned above, the protocol extension was split into two items Incremental commits #199 and [Incremental decommits

1057. Both items contain

high-level description, interaction outline and a first design of the on-chain validators which is currently getting discussed by the Hydra contributors and researchers.

Especially the incremental commit is not trivial as the interaction between layer 1 and layer 2 is quite intricate. Observing a deposit on the Cardano mainchain may not be seen by all parties at the same time and this should not lock up the layer 2 process of transactions, while premature usage of committed funds is to be avoided. On the other hand, each step in the protocol needs to be secure in presence of an adversary and unlocking of committed funds must always be possible.

Once this validator design is clear, we can get started implementing one or both of the features to enable more flexible usage of the Hydra head protocol.

Refactored chain state #1049​

Following up the implementation of ADR#24, this marks the first enhancement in a series of refinements, and our goal was optimization.

We eliminated the need for recursive types in persisted chain states, transitioning to a non-empty history model, preserving full backward compatibility with existing systems.

Importantly, throughout this refactor, we've kept the interface between the chain and logic layers entirely unchanged. This ensures a smooth transition and seamless integration with existing components.

One of the notable advantages of this refactor is a reduced persistence footprint. We now store individual points of the chain state, enhancing storage efficiency.

Community​

Open source contributions​

This month some open source contributions were received, and the team was able to contribute back to one project:

Current scaling use cases being tracked​

Hypix​
  • An update of hydraw with improved usability to stimulate user engagement
  • A showcase for Hydra, with an option to productize
Book.io​
  • A Web3 marketplace for tokenized digital eBooks and Audiobooks
  • Requires NFT minting and distribution at scale (for each copy/instance of a book)
  • Opened a discussion to investigate possible approaches to scaling their operations on Cardano
Midnight​
  • DUST airdrop and side-chain operation
SingularityNet​
  • A Web3 marketplace for AI services offered and consumed through a pay-per-use API
  • Discussion ongoing about porting an existing Ethereum solution to Cardano
  • A chance to build on previous Hydra Pay efforts and upgrade the design to support unidirectional micropayments channels
SundaeLabs​
Emurgo​
  • Initial discussions to evaluate building a payments channel network using Hydra
  • Could leverage work done on Hydra Pay by Obsidian

Conclusion​

The monthly review meeting for September 2023 was held on 2023-09-20 via Google Meet with these slides and the recording.

This concludes yet another fully-packed month, this time not only focusing on Hydra but also encompassing Mithril and the broader goal of supporting Cardano's scaling effort to the best of our ability. Meeting and working together in person demonstrated once more, if this were needed, how much a small group of highly motivated people can achieve. It also fueled our desire and consolidated our vision of being a small group of engineers with a clear purpose, scratching our "own itch", contributing to the whatever projects and tools is needed and best for the job at hand, relentlessly prototyping and experimenting collectively to find the best solutions solving actual people's problems.

It is also worth mentioning that Catalyst's Fund 10 outcome shoudl result in more contributions to Hydra (and hopefully Mithril!) in the near term.