How Arweave and Bundlr Are Building the Modern-Day Library of Alexandria

The Library of Alexandria housed nearly half a million historic scrolls ranging from literary works and philosophical dissertations to scientific research. It contained the entire knowledge of the ancient world, only to lose it all to multiple fires that eventually led to the slow degradation and eventual irrelevance of the library. Despite the numerous challenges the library faced over time, its main problem is one that still exists in today’s digital world: the over-centralization of information.

Centralized Information & Archives

Today, our web infrastructure is highly centralized and fragile, resulting in a handful of large technology companies and organizations storing most of our information and data in relatively few server farms. As a result, information can easily be restricted, altered, or inadvertently lost. There are many ways for these effects to be permanent, including the destruction of data centers caused by war or natural disasters, hackers gaining access to databases, or governments removing certain information that contradicts their narrative. Aside from these external risks, information loss can also occur when web links end up rotting. Over time, web links can fail to work and can deliver a “404 - Page not found” error, due to companies shutting down or domains not being renewed. In a 2014 study, researchers found that nearly 50% of all hyperlinks in Supreme Court opinions led to content that no longer existed on the internet or had been altered. Harvard Law researchers analyzed a sample of 553,693 New York Times articles and found that 25% of all links were completely inaccessible. The data showed that link rot worsened over time, claiming 6% of links from 2018, 43% of links from 2008, and 72% of links from 1998. Link rot results in the permanent loss of internet history and demonstrates that the majority of information on the web will not stand the test of time with our current web infrastructure.

The Internet Archive (IA) exemplifies the risks with centralized archiving. The IA is the largest internet archival organization, offering millions of free books, movies, websites, and more. One of the organization’s projects includes the Open Library, which is a virtual library that allows users to borrow digital books. In response to the Covid-19 pandemic, many libraries were closed, which prompted the IA to implement a plan that allowed people access to any book in its database, without any restrictions. Many publishers were quick to call this an act of public piracy. As a result, four major book publishers filed a lawsuit against the IA, starting an ongoing multi-year legal battle. If the court ends up siding with the plaintiffs, the IA will be forced to remove 1.4 million books on its virtual shelves and put a halt to their book-copying practice. Although this incident could be classified as IP theft, it highlights some of the risks centralized storage entities face due to government coercion. Our modern-day archival infrastructure is fraught with vulnerabilities that could result in an outcome similar to that of the Internet Archive or Library of Alexandria, so finding an alternative storage system is vital.

To solve this problem, Arweave has created a decentralized storage platform that functions like a shared permanent hard drive. Using its Blockweave, a blockchain-like structure, Arweave immutably stores multiple copies of the data uploaded onto the network’s miners. The miners are distributed globally, which ensures that no single point of failure results in data loss. Arweave’s “pay once, store forever” model makes it simple and straightforward for users to secure their data for centuries using its endowment fund. The endowment funds are derived from a portion of network transaction fees, which is used to pay out Arweave miners over time, guaranteeing at least 200 years of storage.

Archives serve as witnesses to the past, providing future generations with the historical records and information needed for society to learn from the past. Using Arweave’s decentralized storage network to archive information permanently for future generations – without the risk of data being altered, lost, or censored – could prove revolutionary for the development of humanity.