2021 year-end review
CoBox aims to offer users a ‘peer-to-peer cloud’ infrastructure to enable collectives, cooperatives, freelancers, artists and SMEs to securely back-up each others’ files. It is built on the peer-to-peer protocol Dat (now called the
hypercore-protocol) allowing real-time replication of distributed file archives. CoBox enables multiple users to add and modify files on a shared distributed filesystem, coupled with an encryption scheme such that backups can be securely hosted on remote devices by partner organisatons or individuals without them being able to access another’s’ private data.
Following an intense 2019/2020 accelerator-fuelled initial sprint, the team announced our ‘Sprout’ Alpha V1 release and published our first prototype alongside a year-review zine. Since our little ‘Sprout’ entered the world a lot has changed for the project. This document is a reflection on our second end-of-year, accompanies by an improved ‘Seedling’ Alpha V2 release, along with some thoughts on what the future might hold.
CoBox has butted up against several challenges which have made continued development incredibly difficult. Many core dependencies of CoBox in the
hypercore stack are under active development and our development team has struggled as the technical ground on which we’ve built continuously shifted beneath our feet. Furthermore the timeline of their developments has not been as transparent as it could have been, which may have enabled us to adjust our own development process to accommodate downstream improvements. The pandemic made it an on-going challenge to conduct truly peer-driven design, its much harder to run workshops and read people’s reactions through video calls. Furthermore, a significant amount of additional funding is required to get CoBox to a state whereby we would feel it adequately meets the needs expressed by our partner projects. Finally, several core team members have become involved in other research and projects. As a group of disparate individuals, with diverse personal priorities and lacking project-specific funding for now, we will likely not take this forward as a crew. Some of us, however, might still return to it in the future, should individual timelines allow. As such, following the ‘Seedling’ release, we are wrapping up active development in the project.
Its been a trip, we’ve really enjoyed the process, and we have learned a great deal about the challenges decentralised tools and decentralised infrastructure face.
Once again with love,
Driven by Peer-Design
The principle of mutual aid sits at the heart of CoBox. In our development practice, this means basing our design and development decisions on the real needs of our user-community. Based on early need-finding work, our initial plan for this year’s sprint was to create a registry to allow groups to discover, connect and create agreements with one another for backing up their data. Our first lesson learned was that user-centered design means being prepared for unexpected results!
Following further research, users and groups participating in our workshops were not so interested in being match-made with organisations previously unknown to them, already being part of existing networks that could provide the support needed. Instead what concerned them more was their own and other groups’ capacity to maintain consistent and reliable infrastructure and data backups under the stress and strain of their daily lives and work, particularly when maintaining backups was not a core part of their group’s mission.
A priority for mutual backup was therefore that groups can easily verify the ‘health’ of their backups - both of their own and of those they host for others. Only with this assurance would groups be comfortable trusting in the safety of their data, or in taking responsibility for the safety of data belonging to others, such that a practice of mutual backup would be pursued.
Throughout the project, we’ve drawn on great insights and ideas from the organisation and projects involved in the co-design process. While limitations in our capacity, both operational and developmental, meant we haven’t been able to implement as much as we would have hoped in terms of features that enabled CoBox to meet people’s needs, we remain committed to the methodology and highly recommend other projects pursue peer-driven-design as a means to build relevant and meaningful tools.
For new backup-health data to be easily accessible to users, we needed to add additional local databases to store and update network-related data, such as the last time a given individual (of one’s own group, or a ‘partner’ group tasked with backing up some data) was seen online, whether they are online now (i.e. whether they are backing up data as it is being written in the present) and the last time one’s own copy of the data was fully synchronised with the remote copies made.
Accessing the desired information about peer-status and synchronisations from the underlying DAT(/Hyper) protocol took longer than anticipated due to the nature of the protocol and limitations imposed by
multifeed. Having established an effective local store of networking data, we embedded new features in the UI enabling users to survey live the health status of their data as they connect and disconnect with remote peers. Furthermore we added a settings page allowing users to individually configure status alerts based on how many and how up-to-date backups are.
A second major feature developed this year was to introduce an experimental implementation of DIDs (decentralised identities). This makes use of
libsodium’s key derivation function and our CoBox
parent_key, along with a randomly generated seed, to create dynamically generate unique keypairs. Initially these are ed25519 signing keypairs, and can be easily converted into curve25519 keys for encryption. Multiple identities can be generated, made use of, and destroyed through the API. Derivation seeds are persisted to disk in a leveldb so the identity can be dynamically regenerated. This allows users of CoBox to experiment with being a different identity in different contexts. The existing export / key backup functionality needed rewriting after introducing DIDs. Challenges remain for whether our approach is a viable long-term way of addressing DIDs.
We did not have the time or funding to fully embed this feature into the UI or CLI, and as such while implemented in the back-end, this feature remains inaccessible.
The rapid timeline for delivery during EU LEDGER led to the accumulation of some technical debt. Much of this has been addressed as best is possible while many of the underlying APIs and core data structures remain under development and are subject to breaking changes in updates. A breaking change has been introduced in
@coboxcoop/server after implementing password protection of the parent key (see Security Review).
2021 presented us with the opportunity to receive two hour-long sessions with Least Authority, during which core modules of CoBox received a security review and we were recommended some optimisations that would improve the system’s resilience to attack.
For example, CoBox originally stored private keys in plain text on the local file system. While these were never exposed over the web API to any local client, a security vulnerability remained in the existing system whereby malware installed natively in the operating system would be able to access these. Following the review, we implemented on-disk encryption of the keys using password protection. The primary parent key - from which all signing and encryption keypairs are derived on the fly when the app starts up - is now encrypted using a password. This security optimisation makes use of the industry standard for password hashing,
bcrypt. Introducing this significantly altered the flow of the application, with a new authentication step required before loading any CoBox keys (and their corresponding
hypercore). Controllers requiring authentication are protected using a server-side session.
Further work is needed to address other recommendations made.
By eliminating the need for servers and instead using very low-power ARM devices - CoBox contributes to a massively reduced carbon footprint.
One of the major challenges CoBox and any similar project faces going forward is the construction and maintenance of reliable hardware - no easy feat. Our hardware prototypes have faced such challenges. Many projects have tried - and failed - at building decentralised server architectures. We invested significant time into both the software and hardware component of our project, and ultimately as a small team spread ourselves pretty thin. Any new project attempting to tackle seeding a decentralised cloud infrastructure has to engage positively with the long-term responsibility of its maintenance and reliability. Despite our best efforts and willingness, the timing for us simply wasn’t right!
Following our initial accelerator funding from the European H2020 Ledger grant, we scoured the funding landscape for new opportunities and submitted multiple proposals. For better or worse, CoBox sits amongst a quiet and underfunded niche of software projects that aim to create an alternative digital infrastructure that is based on principles not profit, and staunchly resisting unicorn goals. Likewise, in spearheading primarily social innovations in technology, CoBox is a downstream project that is built upon underlying deep-tech innovations, rather than fundamentally driving the latter ourselves. We did, however, secure support from the Prototype Fund, a project of the Open Knowledge Foundation who prioritise open-source and early-stage innovations to promote experimentation in tech. We are grateful to the Prototype Fund for their support, and for their efforts to keep the community spirit alive throughout lockdown.
Off the back of an intensive first year, and subsequent drought in funding, Magma members supplemented work on CoBox by collaborating on other projects within the p2p community. Other members graduated on to new and exciting things: prioritising a more geographically and politically local focus during lockdown, joining a radical Blockchain-startup bringing secure data-sovereignty to the new digital age, and returning to university for deeper explorations of complexity and systems.
For anyone interested in taking CoBox forward, there is much needed work. Priorities would be fixing some underlying issues with our encryption scheme, tackling the challenge of duplicated data, perhaps by integrating
hyperspace’s native multi-writer modules (released two years too late!) among other factors. If you would like to contribute, or even pick up the project and run with it - please get in touch. We will be happy to support you in whatever way we can!