Skip to main content

· 6 min read
Sebastian Nagel

This report summarizes the work on Hydra since December 2022. It serves as preparation for the monthly review meeting, where the teams update major project stakeholders on recent developments to gather their feedback on the proposed plan forward.

Roadmap​

Looking at the roadmap, there are just a few things to report this month:

The latest roadmap with minor changes, which needs a reflection of the latest objectives.

Development​

Issues closed since last report

This month, the team worked on the following:

  • First write-up of Hydra spec online. The team has been aggregating the latest definitions of on- and off-chain semantics into a new technical specification. The latest version can still be found on Overleaf, before its integration into the core Hydra repository. Should you have feedback, please send it our way through one of the communication channels.

  • Closed more gaps in the Head Contract. The team addressed two more gaps in the on-chain scripts based on discussions on the specification.

    • All validators are authenticated now and ensure contract continuity by checking for the state tokens in the output (or getting burned).

    • Abort now fully reimburses all committed UTXOs, even if two parties would have committed the exact same output (quite a theoretical attack vector).

    This work is captured in #452, but this work item is taking quite a bit longer than expected. Also, it's a single feature item, lacking some rationale on certain requirements, so we intend to break this down into smaller pieces over the next days and weeks.

  • Mutation test framework improvements. While closing gaps in the contracts, the team realized that some of the mutation tests were correctly tripping validators and making transactions invalid, but not always for the right reason. In fact, some of the mutations were ‘too harsh’ and made the even transaction not even pass phase-1 validation. The team addressed this by introducing a first (naiive) way to assert the right cause of the failure by checking strings in the validator log. #679

  • Add headId in API and TUI. The team not only added the unique headId to the appropriate server outputs and the TUI #678, but also added sequence numbers and timestamps to produced outputs #618. This feature was asked for by users and simplifies identifying of heads and integration with the hydra-node.

  • Hydra explorer experiment. Some of the team used the holiday season to conduct some experiments on summarizing Hydra Heads observed on a network. This quick hack already demonstrates the value of a Hydra explorer as a tool to measure adoption by open heads on a given network. Naturally, it would reuse code from the hydra-node and challenge the architecture used in it for tracking multiple heads. Also, it will drive discussion about the versioning of hydra-plutus and the possibility to track multiple versions of the Hydra protocol on the chain (hashes/addresses change in each version).

Hydra explorer first experiment UI

Community​

  • The Hydra for Voting project, which got kicked off in December is picking up steam with deep dives into Catalyst voting and Hydra technology. Discussions are currently held in various settings and the team is continuing to develop a picture of what is achievable.

  • Hydra for auctions lite paper. IOG is collaborating with MLabs on exploring how Hydra can improve auction use cases. At this stage, the teams discussed multiple approaches on structuring the problem space with various Hydra topologies and specific trade-offs. The full case study was published on Essential Cardano. The teams plan to upstream the findings back into the use case section on the Hydra website from which this project originated.

Themes for 2023​

After kicking off planning in a workshop and reflecting on what Hydra achieved in 2023, the team has progressed in fleshing out relevant themes and objectives for this year to position Hydra as a sustainable open-source project in the age of Voltaire:

  • A mainnet mature application

    • Why? The core contributors should be able to use the protocol on mainnet, so it should be maintainable and mature enough to lock some ada in a Hydra Head using a reference DApp (eg, hydraw). By creating and growing this DApp, the team will dogfood features and hence improve additional usability.
    • How to measure? Number of heads on mainnet > 0 and core contributors feel confident to lock at least 100₳ in a Hydra Head on mainnet.
    • Next steps:
      • Publish Hydra Head V1 specification
      • Close gaps in implementation and release 0.9.0
      • Create a request for proposals (RFP) to audit the specification and its implementation

  • Increase adoption

    • Why? The team wants to showcase what is possible with Hydra through benchmarks and lighthouse projects, enabling use cases and reducing friction to use Hydra. Also, ensuring interoperability through open standards and reference implementations.
    • How to measure? Number of third-party created heads on any network > 0
    • Next steps:
      • Build a basic Hydra Head explorer to measure adoption
      • Benchmark off-chain performance (of a selected scenario)
      • Demonstrate Hydra for payments to potential users
      • Support Hydra for voting project
      • Hydra for auctions project exploring a delegated voucher auction

  • Sustainable open-source-development

    • Why? The team wants to scale Cardano in the long run. Hence this project should not be owned by a single entity, but by the community and the Cardano network itself. It is thus essential to make contributions to the project possible and ultimately it should be easy to become a contributor.
    • How to measure? Number of contributors on GitHub
    • Next steps:
      • Open up monthly reviews
      • Add tutorials to the website
      • Publish and maintain a use-case-centric roadmap and feature map

Conclusion​

This monthly review was the first public one with not only stakeholders from IOG and the CF, but also collaborators of the latest projects. This was the first step to a more open and transparent development process.

The teams showed some classic demos – like the ‘hydraw’ application which has been running on the same Hydra Head for four weeks now – and some new developments on the Hydra explorer and minor improvements to hydra-node API and TUI.

It was very interesting to bring various teams together to exchange ideas and thoughts on the individual roadmaps for the new year. The teams also shared the relevant themes and objectives of the Hydra project for 2023 (as also listed above). They will now make sure to reflect these steps into this concrete roadmap such that it will contain marketable features bringing us closer to these goals.

· 4 min read
Sebastian Nagel

This report summarizes the work on Hydra since November 2022 and also serves as a preparation & write-up of the monthly review meeting, in which we update major stakeholders of the project on recent developments and gather their feedback on our proposed plan forward each month.

Roadmap​

Looking at our roadmap we can report the following updates:

Notable changes​

Latest roadmap with drafted 0.10.0 version.

Development​

Issues closed since last report

This month we have been working on:

  • Fixed AcquirePointTooOld errors by changing the way the internal wallet initializes its state #621. See the PR description for details. This did not strictly happen this month, but we have not mentioned it last time.

  • Closed first gap in Head Contract: We collected gaps between specification and current implementation in #452 and started now closing them. This first issue, was to prevent DoS / stalling of heads, by bounding the lower and upper transaction validity time of the close transaction. As always, time handling is hard and this required detailed discussion in the form of an ADR. See ADR21 for more details.

  • Reduced cost of commits by at least 30% using reference scripts also in the commit transaction. We only used reference scripts in abort so far. This reduces the min ADA cost of commit because the transactions are roughly 5.2KB smaller.

    Before:

    After:

  • Use nix flakes & build docker images using nix: Triggered by a migration to Cicero for a nix-based CI, we started using nix flakes for nix build and nix develop instead of the legacy nix-shell in #646. While we ended up sticking with Github Actions as the only CI and a Cachix cache, we extended this work to build our docker images more efficiently using nix in #654.

    Image sizes went down between 20-50%:

    While docker CI builds on master went down to 1-10min (from 10-30m+)!

Hydra V1 Specification​

Finally, we bit the bullet and started a write-up of the specification. Over the last couple of months we had worked with marked up versions (see below) of the original paper and a GDoc to discuss and review the protocol as it was implemented.

As we also updated the security properties and proofs, we started aggregating a LaTeX document on overleaf. While it holds a similar background and preliminaries as the original paper, it only specifies a single version of the protocol with no variants, less modularization, as well as explicit assumptions and important notes for implementors.

We aim to be publishing the spec as part of the main hydra repository soon.

Community​

  • Hydra for Payments project completed the first phase of this work stream by tagging version 0.1.0. This open source project conducted by Obsidian Systems and sponsored by IOG created a great starting point for developers to build services/APIs of payment channels using Hydra Head. This month we reviewed the improved documentation and are looking forward to see this work applied to eventually build fast & cheap payments for light-wallet users and power payment channel use cases from business-to-business (B2B) in the short-term.

  • Hydra for Voting project kicked off this joint project between IOG Research, Catalyst & Hydra Tribes, and the Cardano Foundation. The goal is to assess suitability of Hydra Head to support voting use cases by implementing parts of Catalyst voting system on Hydra Head and benchmark execution at scale. The kick-off meeting took place on December 15.

  • Blog post by CF: To round off the year, Matthias wrote a nice retrospective blog post about Hydra at the CF. Besides summarizing our progress made in 2022 it also gives an outlook where Hydra is headed for in 2023.

Conclusion​

Although Hydra is in essence an open source project right now, we would love to make it "even more open source" in the coming months. Our vision of Hydra requires to have the builders of the Cardano ecosystem to be part of this journey. For that matter, we want not only to open up these monthly reports, but the review meeting backing it as well!

This, besides our engagements with the community above, will allow the project to grow beyond IOG & CF contributions. To become the true Cardano open source project we believe will be worthy to serve as the scalability component required by the Cardano network.

· 9 min read
Sebastian Nagel

Introduction​

This report summarizes the activities of the Hydra team since October 2022 and also serves as a preparation & write-up of the monthly review meeting, in which we update major stakeholders of the project on recent developments and gather their feedback on our proposed plan forward each month.

Roadmap update​

Looking at our roadmap we can report the following releases and updates:

Release 0.8.0​

  • Long overdue feature release adding persistence to the hydra-node
  • Backup & restore the state of a Hydra Head #187
  • Improve user experience following hydraw experiment #518 from being only a discussion to a (to-be-groomed & planned) feature
  • Full release notes

Release 0.8.1​

  • Follow-up release addressing user feedback on persistence from 0.8.0
  • Allow clients to see latest state after restart #580
  • Bug fixes of following chain state #599
  • Full release notes

Notable changes​

  • Impact mapping from workshop (see below) and reviewing the starmap above had us discuss whether the scope for 1.0.0 is correct or not
  • Need to be conscious of scope creep vs. getting something audited
  • Focus on getting the spec done, 0.9.0 tagged and an audit under way now → minor re-ordering in priorities
  • Iterative releases afterwards still - even while getting audited, non-contract changes first
  • Promoted https://github.com/input-output-hk/hydra/issues/635 from being only a discussion to a (to-be-groomed & planned) feature

Latest roadmap with 0.8.0 and 0.8.1 already released and slight re-ordering on 0.9.0

Development​

Issues closed since last report: Issues - input-output-hk/hydra

Besides the things in the releases above (see roadmap update), we have been working on:

  • Flaky TUI CI tests: We have sunk quite some time again in investigating cryptic CI failures in our TUI end-to-end tests. They have been crashing abruptly without any information and thus hard-to-debug. This investigation was unsuccessful and we realized at some point that the TUI tests are not that important anyways - we have API-level end-to-end tests. So we disabled these tests for now.

  • Published hydra-cardano-api to CHaP: With the Cardano Haskell Packages (CHaP) becoming available now and wanting to be a good citizen, we pushed for getting our flavor of the cardano-api also published there. This is non-trivial though, as we are at an integration point even "further up" than the cardano-node #504

  • Implemented ADR18: While the first stints on persistence already make it possible to restart hydra-node without needing to close a Head, we have implemented this in a nicer way now only keeping a single state (ground truth) for both L2 and L1 information #541 of the Head protocol and the specification, we kept discovering bigger and bigger issues and hence realized the need for a discussion on transaction validity in context of closing / contesting Heads. #615

Formal verification & specification​

An important part of the project right now is the formalization and audit preparation:

  • Internal audit has started with involvement of two persons from IOG IT team, each one addressing different part of Hydra:

    • One auditor is working on the project as a whole, targeting potential vulnerabilities with the off-chain code, the infrastructure, dependencies, etc. This has not lead to any significant issue nor action plan so far.
    • Another auditor is specifically targeting the formal specification and the on-chain code to identify vulnerabilities in the protocol itself.

  • Discussions with researchers have led to the development of a joint Coordinated Hydra Head V1 specification defining formally the protocol as it is actually implemented.

    • Work on this document has already allowed us to identify gaps
    • It is the basis on which BCryptic's analyst is working to audit Hydra protocol
    • It allows us to make explicit a lot of implicit assumptions that are in the code but not in the original paper and "Shape" the language used to describe the protocol

  • We have drafted an RFP for submission to external auditors https://github.com/input-output-hk/hydra/blob/audit/rfp/security/RFP.md defining the scope and targets of the audit

  • We are still unsure about whether our approach is the right one as having a "formal specification" in a manually checked document spanning a dozen pages seems quite brittle.

    • Some progress has been made on "formalizing" properties to be automatically "QuickChecked" using an executable model of the expected behavior of the system but this approach seems more suited for "team-internal consumption", e.g. building confidence within the core committers and contributors

    • Ideally, we would want a proper formalization of the protocol, using an existing theorem proving/model checking framework, through which properties could be asserted. Quite a lot of work has already been done in blockchain space, including some work on Lightning and TLA+ or some other state-machine/temporal logic based language appear promising:

    • This effort should be done in accordance to the DApps certification process

Product​

Most updates on the product side of things have been addressed in the starmap update above.

  • Hydra for Payments lighthouse project: Latest demonstration shown by Obsidian Systems on how to use payment channels in a "light way" from a web frontend. The API evolved and we were able to use the preview testnet now. The project is progressing very nicely and we are optimistic to close it successfully soon with improved documentation and usability.

  • NFT auctions lighthouse project: After doing surveys and interviews, MLabs is currently concluding the discovery phase with a light / white paper on how NFT auctions could be implemented using Hydra (today or in the future with more features).

  • Drafting Voting on Hydra project: Work has started on building a Proof-of-concept for voting on Hydra targeting Catalyst’s use case but with an eye towards building a generic solution suitable for large scale voting systems based on Hydra Head

    This development should be undertaken jointly with the Cardano Foundation and SundaeSwap who are also interested in building such a system and recently demonstrated their capacity of doing voting via the Cardano Summit voting system.

  • TxPipe demonstrated integration with demeter.run: Without our help nor even knowing of this effort, TxPipe has recently shown an early prototype of hydra-nodes instrumented via their demeter.run platform. This has come a bit as a surprise and is exactly why we love open source ❤️ → Video

Team & open source​

Some notable developments this month have been:

  • Renamed the repository: from hydra-poc to hydra! This was revealed in the summit presentation (see below) and should indicate that the project is not only a Proof of Concept (POC) anymore, but has become more - as also demonstrated by the various early adopters and demos lately. 🐲

  • Inner source (IOG) contribution: The education team has been working on a Hydra Tutorial and we have been involved in reviewing and trying it out. This is a great effort and will help people get started with Hydra. Thanks folks 💚

Cardano Summit & workshop​

This month there was also the Cardano Summit, this time organized by the Cardano Foundation and the content was even voted on by the community! We have been both, invited and nominated as panelist and speakers and this makes us extremely proud! In general, the reception of Hydra seems to be very positive in the community from what we could experience first hand on the summit.

Agenda items we participated in:

After attending the summit, we also used the fact that the whole team is in one location for a team workshop. We booked a coworking space and spent 3 days together. Not much coding & hacking this time, but we are happy to have produced these results:

  • Timeline / year in review: As we had new team members with us, the request was to give a recap of how the Hydra project evolved over the last two years. The whole timeline can bee seen below, and this also sparked the idea of creating a "Year in review" blog post, which is currently in preparation.

  • Retrospective: In-person workshops are the perfect place for doing retrospectives to reflect on what was good, bad and collecting ideas & actions in how to improve our work environment and processes.

    Retrospective board from 2022-11-22

  • Impact map: Conscious about the fact that the project got defined about one year ago in a first project plan, and inspired by timeline and (short-term) ideas, we also set off to reflect on the Why, How and What of this project.

    After reviewing the project vision, we reached for the tool of our choice to ideate on tangible objectives / deliverables - impact maps! Also, we felt the need to use a different goal this time. One that is reflecting more closely (or, in fact, broadly) what our current mission is. So instead of the % of Cardano transactions are done on L2, we set a new goal: being the number one DApp on Cardano (by all known metrics: TVL, traffic, volume etc..) Fundamentally, both goals illustrate the same idea, but the latter better supports the narrative that Hydra is also just a DApp (not a network upgrade) and needs usage & adoption to reach our vision.

    Below you see the result of our session. We checked back to the old impact map after creating this one, and many things are still relevant / similar on the new sample (it's never complete!) - some of the deliverables we even achieved.

    Impact map with new goal

Conclusion

Reflecting on our original plans (about a year ago) and our current roadmap we can say we are behind our anticipated schedule. However, recent developments on the lighthouse projects and community members contributing & using Hydra are convincing us that we are on the right track. The summit paints a similar picture and we feel reinvigorated by the great people we met, discussions we had and relationships we created & solidified on the summit and during our Hydra workshop.