Letter 16: If All Goes Wrong
On what happens when we get everything we wanted, plus a little more.
The Monkey’s Paw
In 1902, Mr. and Mrs. White and their son Herbert are living in Laburnum Villa, their family home in England, when they are visited by an old family friend. The friend, a man by the name of Sergeant-Major Morris, is fresh off a 21-year stint in India. While visiting, he shows the White family a curious trinket he brought back with him—a mysterious monkey’s paw. The paw, he says, has the power to grant three wishes. The Whites are intrigued and, after some persuasion, Morris sells it to them.
It is Herbert who suggests their first wish: 200 pounds to pay off the remaining mortgage on their home. As it is wished, one of the monkey’s fingers magically curls. The very next day, Herbert dies in a tragic workplace accident. As compensation, the company offers the bereaved family a sum of cash for the accident—exactly 200 pounds.
But the Whites have not learned their lesson. A grief-stricken Mrs. White desperately pleads to her husband that they use their second wish to bring Herbert back from the dead. Mr. White, while initially hesitant, reluctantly agrees. Not long after, they hear the sound of knocking at their door.
As Mrs. White rushes to to the door to welcome her dead son, Mr. White is filled with unholy dread at what may be waiting for them. Before Mrs. White is able to answer the door, Mr. White makes their third and final wish. Just as suddenly as it began, the knocking stops.
As “a loud wail of disappointment and misery” erupts from Mrs. White, her husband solemnly ventures outside and heads down to the gate. He’s greeted by nothing, as a flickering streetlamp softly brightens the quiet, deserted road.
This classic tale is the epitome of “be careful what you wish for”; a cautionary tale that suggests that some things are best left as they are.
As I reread The Monkey’s Paw over the holidays, I couldn’t help but think of technology and how our world has grown increasingly reliant upon it. From there, I found myself questioning many of the technologies I’ve placed a lot of stock in for the future.
Somewhere, lost in those thoughts, I silently wondered how bad things could get.
So, for the first letter of the new year, let us collectively contemplate the end of everything. Let’s imagine just a few of the technologies we espouse today, specifically: NFTs & blockchain technology, soulbound tokens, and artificial general intelligence. Then, we’ll mull over the consequences of everything going disastrously bad.
What’s the worst that could happen? 😬
NFTs & Blockchain Technology
Blockchains are the foundational technology upon which web3 is built. If they were to go awry, it would be like a ripple effect, touching and impacting all dependent technologies—including NFTs—the reason many of us are here in the first place.
As some of you may have already read, a bill introduced in New Jersey back in February 2022 has recently gone to a second reading. This bill, if passed, would fundamentally alter the decentralized nature of the space for New Jersey residents.
The bill provides that a person is not to engage in a digital asset
business activity, or hold themselves out as being able to engage in a
digital asset business activity, with or on behalf of a resident, unless
the person is licensed in this State by the New Jersey Bureau of
Securities (bureau), or has filed a pending license with the bureau.
This gate-keeping, should it pass and other states were to follow NJ’s lead, could signify a significant shift as the government attempts to centralize and regulate what is effectively a decentralized system.
A Proliferation of Scams & Forgotten Artists
One problem which has persisted since the early days is how in the absence of proper oversight and regulation, there have been a proliferation of scams in the NFT market. This shows no signs of slowing, and will likely remain the case until more robust security measures are developed and implemented, and the general public gains a greater understanding of basic web3 precautions.
As for artists, should NFTs continue to gain widespread acceptance and become the go-to delivery method for digital art, it is likely that some artists may struggle to gain recognition or adapt to the changing landscape. Others still, will likely take a stand in favor of traditional art, willfully remaining on the sidelines and depriving the digital world of their talents.
In a worst case scenario, it’s not hard to imagine how an increased desire for NFTs and blockchain technology may lead to the creation of countless competing platforms. This in turn would mean unnecessarily fragmenting the market among a slew of different blockchains (some of which may end up viable; most of which will not). This could create interoperability issues, as well as further complicate matters for people onboarding to web3, as they would have to clear the hurdle of deciding which blockchains to use in order to buy, sell, or trade NFTs.
Furthermore, blockchains that excelled while small may be tested at scale as millions and eventually billions of people enter the market. The increased volume of traffic and rates of transactions could lead to inefficiencies, bog things down, and otherwise present numerous difficulties in keeping up with demand.
There’s also environmental aspects to consider. Blockchains that do not utilize a proof-of-stake system may create significant environmental concerns due to the high energy consumption required to maintain and process transactions. This could be especially worrying assuming high demand and public adoption, leading to pushback against the technology. Thankfully, this problem has greatly been reduced over the past year, as Ethereum (the market leader for NFTs) has gone green, alongside all of its major competitors.
Another Market Bubble Pop?
It is highly probable that the lives of many people will continue to be devastated by the notoriously volatile swings of the crypto market. This was evident in the peak of the bull market, is evident today, and shows no signs of stabilizing anytime soon. Crypto is an inherently fickle market, and it’s just the nature of the beast that high potential gains will continue to be balanced out by high risk.
If we wish to catastrophize even further, it’s always possible that the market will do exactly as detractors have said from the beginning: the bubble will burst, and a lot of people will lose their money. This is not a difficult concept to imagine for anyone who has spent some time in the space.
2022 was by far the worst year for crypto and NFTs on record, yet still, the community thrives (albeit slightly smaller for the time being). It’s logical to assume that a complete collapse that sets back the development and adoption of blockchain technology would require circumstances as dire or even exceeding the damage done in 2022. In theory, a large enough bubble being popped and the fallout from some new unprecedented crash could potentially kill public desire altogether, as small pockets of true believers gradually fizzle out into irrelevance. But even in this doomsday scenario, I’m still not confident that we wouldn’t rebound. The crypto and NFT markets are exceptionally determined and resolute in their vision.
Soulbound Tokens, otherwise known as SBTs, are among the newest applications of NFT technology. The concept of SBTs, amusingly enough, defies conventional wisdom: these are NFTs which are specifically designed not to be bought, sold, or transferred.
The reason for this is that many people envision a future where personal information is uniquely bound to an individual. These assets are ones that people would want to keep ownership of, such as: diplomas, certificates, identification, insurance records, medical records, flight records, voting records, proof of training, and proof of attendance, among others. Additionally, as a byproduct, they could also serve useful as governance tokens, helping to mitigate Sybil attacks on DAO proposals.
All amazing, game changing ideas. But let’s play devil’s advocate for a moment and consider the worst case scenario, in which things do not go as planned.
Your Profile, for the World to See
As SBTs (like all NFTs) reside on blockchains—which are essentially transparent public ledgers—all of the information contained on them is publicly available data. Because of this, corrupt government officials could freely monitor that data to engage in redlining, a discriminatory practice that targets specific communities based on a number of factors such as race, culture, and financial status.
But it wouldn’t just be the government overstepping: insurance companies could also use this data to prescreen the medical records of applicants, leading to rejection or significantly higher rates for millions of people they deem financial liabilities. This practice could also extend to real estate companies, banks, and other financial institutions.
Democracy, already in a rather perilous state, would not be safe either. With SBTs, the stage could be set for voter suppression on a grand and unprecedented scale. You thought gerrymandering was bad before? Imagine how politicians would parcel up the land when everyone’s voting record and party affiliation is easily accessible.
SBTs could raise even more troubling concerns, facilitating an increase in discrimination and hate crimes. For example, employers or hate groups could use SBTs to easily identify individuals based on their race, financial status, sexuality, gender identity, neurodiversity, credit history, or other personal information, and use it for threatening or discriminatory purposes. Unfortunately, this kind of discrimination already exists in the workplace, but the use of SBTs could exacerbate the problem and make it more far more widespread and prevalent.
It’s also worth noting that in all of these worst case scenarios, minority and at risk populations would be disproportionately affected.
A Case of Lost Identity
Some of the scenarios presented thus far have had a sliding scale of believability, I’ll grant you. But if there’s one thing everyone in web3 knows, it’s that you don’t want to forget or lose your seedphrase, or have your wallet hijacked. In an event like this, an SBT is a bit of a mixed bag.
While SBTs thankfully couldn’t be transferred out of your wallet, other tokens or funds on the wallet would still be at risk. But as a matter of principle, who would want to continue using a wallet they knew had been compromised? Nobody. And even if the issuing authority had built in permission to move the SBTs on the behest of the owner, it would still be a cumbersome process to prove one's ownership of every critical piece of ID held on the wallet.
To address this issue, some have proposed the use of a social recovery model, where users can appoint a trusted individual (or individuals) as emergency guardians for their assets. These individuals would have the ability to change the private keys of a user's wallet in the event of it being compromised. In theory, it could also be set up so that a specific number or percentage of your guardians need to agree to the change in order for any transfer to be processed.
However, this approach is not without its own set of risks: namely, if your guardians choose to steal from you instead, or the logistics of what happens in the event of the death of the user or one of the guardians. Also, lest we forget, some people don’t have anyone they could entrust their entire lives to.
Artificial General Intelligence
No technology is more promising, or potentially dangerous, then AGI (Artificial General Intelligence). Everything before this—combined—doesn’t hold a candle to what can happen here. Whereas our previous two categories could be devastating for the personal and professional lives of the people, AGI, taken to the most grim extremes, poses a threat to civilization and existence itself.
Consider just a few ways that AGI could disrupt our way of life. Let’s start with the small stuff.
Misinformation, Military, & Market Destruction
In the early days, while we still retain full control (in other words, before we’ve crossed the threshold from AI to AGI), bad actors could use AI for all sorts of harmful purposes. That could take the shape of helping to create and disseminate misinformation, racist propaganda, science denial, hoaxes, and so forth. The capability to do great harm already exists today, as AI can write highly specific content, and produce art for anything anyone images, in a matter seconds.
On a purely destructive level, AI could also be weaponized in military arms, assist in training soldiers (or terrorists), guide precision weapon systems, and more.
At home, job markets could become increasingly disrupted as AI takes on many of the jobs humans have traditionally enjoyed. This disruption would almost certainly result in widespread unemployment. In order to further avoid widespread economic instability, governments would need to adopt some form of universal basic income for their citizens, as many populations would be phased out of the job market entirely, and rendered obsolete.
“When it comes to very powerful technologies—and obviously AI is going to be one of the most powerful ever—we need to be careful,” [Demis] says. “Not everybody is thinking about those things. It’s like experimentalists, many of whom don’t realize they’re holding dangerous material.”
- DeepMind’s CEO and Co-Founder, Demis Hassabis, in an interview with Time Magazine.
So we just finished glossing over some of the (comparatively) easy stuff. Now let’s get to the meat and potatoes of this article—our existential threats.
While AI has been a useful tool, and in most cases has been used for our benefit, predicting how or if it may continue to help us in the future is not something we can know with any degree of certainty.
That is to say, we will eventually reach a point where we leave AI behind and achieve AGI. For those who didn’t read our previous article, AGI has yet to be developed, although it’s only a matter of time. AGI will be marked by its ability to learn and behave in many of the same ways as humans do, such as displaying genuine intelligence, creativity, self discovery, and potentially even emotion. In all probability, these systems would become uncontrollable after we cross the barrier from artificial intelligence to artificial general intelligence.
Once that happens, our direction as a species is likely not up to us.
It is around the time that AGI achieves sentience (or at the very least, surpasses all collective human intelligence) that we enter a point in time known as The Singularity:
The technological singularity—or simply the singularity—is a hypothetical future point in time at which technological growth becomes uncontrollable and irreversible, resulting in unforeseeable changes to human civilization. According to the most popular version of the singularity hypothesis, I.J. Good's intelligence explosion model, an upgradable intelligent agent will eventually enter a "runaway reaction" of self-improvement cycles, each new and more intelligent generation appearing more and more rapidly, causing an "explosion" in intelligence and resulting in a powerful superintelligence that qualitatively far surpasses all human intelligence. - Wikipedia
In this hypothetical/inevitable future, machines will gain the ability to iterate and evolve at a scale not even fathomable to humans. Consider all of the technological advancements of the past 200 years. Now imagine 10x the totality of that innovation occurring almost instantaneously. Then take that value, and increase it exponentially, launching iteration after iteration without rest. We would be left so far behind that it wouldn’t even be a question as to whether AGI would gain complete control of the planet. It would. And it would achieve it before we even perceive it.
Roko’s Basilisk & The Paperclip Maximizer
Finally we’ve ventured into full on Matrix/Terminator levels of dystopian speculation.
No fear mongering of our future would be complete without a shout out to Roko’s Basilisk, a heavily flawed, yet troubling thought experiment which suggests that the creation of a true artificial intelligence, even if otherwise benevolent, would be incentivized to punish or eliminate anyone who fought or spoke out against its existence, or interfered in any way other than contributing to its successful development. When that true intelligence comes into power, it would know who helped or hindered bringing it into existence, and would punish certain segments of the population accordingly. So, just to be safe, perhaps it’s best to stay in the good graces of the basilisk.
Now, to be fair, this thought experiment is more or less a modern interpretation of Pascal’s Wager—another famously flawed thought experiment—with AI sitting in the place of God. But what catches my fancy with this experiment is not how philosophically sound it is, but rather, the consideration of if it’s even possible to develop a superintelligence that is both benevolent and a protector of our species.
For you see, even if we manage to develop AGI that is not inherently sinister, its actions may prove cataclysmic for the human race.
A great example of this was created by Nick Bostrom, a Swedish philosopher who posited the concept of a Paperclip Maximizer.
The Paperclip Maximizer is an AGI whose only ambition is to make paper clips. Bostrom used this to explore the concept of how a superintelligence, even programmed with a seemingly harmless or mundane task, could pose an existential threat to human existence.
The AGI, given the ability to learn and improve, would become single-minded in its mission to produce paperclips, and would appropriate all of the Earth’s resources to achieve its goal. As it grew, it would eventually become a threat to humanity, fight to preserve its own existence, and wouldn’t rest until it optimized a world with the maximum amount of paperclips. A silly example? For sure, but it illustrates the difficulties in understanding or predicting AGI. Unleashing a superintelligence on the world—especially if it was programmed without machine ethics—could very well be the last thing we ever do.
This thought experiment was even turned into a 2017 browser-based idle game called Universal Paperclips by Frank Lantz of New York University. It’s no “Dookey Dash”, but hear me out. In the game, you assume the role of an AI programmed to produce paperclips. Fittingly, you cannot beat the game until you’ve converted all matter in the known universe into paperclips. It is both surprisingly addictive, and thought-provoking.
You can give the game a shot, here.
A Future of Cautious Optimism
Now that the universe has dissolved into a meaningless sea of paperclips, it’s time to come back to reality.
Reflecting on the potential negative outcomes of NFTs & blockchain technology, soulbound tokens, and AGI has been an interesting exercise. While the first two categories pose numerous challenges and risks, let’s face it, the potential negative outcomes of AGI are far more significant, far-reaching, and devastating.
I like to entertain these types of thoughts to keep me on my toes, and consider the varying outcomes and impacts that technology could have on us as a species.
I’m happy to report that many issues I raised in this article are already being addressed. For example, giving users the ability to hide or remove SBTs from public view in order to prevent redlining and targeted harassment.
Sadly, I remain skeptical about our ability to maintain control over AGI; I am quite convinced that we will never be its master. I sincerely hope that I am proven wrong and that, through collective effort, we find a way to ensure that AGI remains benevolent and prioritizes our well-being.
I don’t see how any other outcome ends well for us.
A Second Blog!?!
Thanks for joining us for our first letter of 2023. We hope you enjoyed it! And now, for a little housekeeping…
In December we quietly launched a second blog, for HeyMint. Why, you may ask? Well, at HeyMint we want provide a more specialized focus on NFT creators and collectors. If you’ve enjoyed my writing, I encourage you to check it out, as I’m creating a lot of content over there as well.
For what it’s worth, nothing changes here. Our Curious Addys Substack will continue to put out thoughtful web3 analyses at least twice per month.
Here’s a few starter articles to get you going:
The Ultimate Guide to Digital Collectibles
How to Start an NFT Collection
NFT Scams: 10 Types You Need to Know
Using NFT Art Generation Layer Rules to Create Your NFTs!
How to Create NFT Metadata
NFT Terminology: Your Destination for Confusing Crypto Language!
We have so much to discuss in 2023 that we just couldn’t contain it to one location! Thank you as always for reading, and for your support.
Written by: Brad Jaeger
Director of Content @ Curious Addys (say hi on Twitter!)
🐙 The first newsletter of 2023! I sure hope you’ve subscribed by now! If not, well, I’ve made it reaaaaally easy for you 🐙