The optimal Ethereum heist: attacking the Parity wallet (part II)

[continued from part I]

A suboptimal heist?

Looking at how the attacker exploited the vulnerability shows that it was far from being the most elegant operation. First notice the delays between the two calls made to each vulnerable contract. Here is another look at the pairs of calls made for exploiting three different vulnerable wallets:

Screen Shot 2017-07-26 at 16.29.29

Pairs of calls used to exploit the Parity multi-signature wallet vulnerability against three different contracts

In the first attack, there is a gap of 10 Ethereum blocks between the calls, corresponding to ~3 minutes based on current block frequency. For the second attack, the calls are separated by 7 blocks. For the final victim, they are only 2 blocks apart suggesting that the perpetrator improved their execution over time. Still these wide gaps suggest that at least the first two attacks were being crafted manually, likely using an interactive REPL such as the geth Javascript console that allows invoking arbitrary methods on any contract. In principle there is no reason to wait between calls: as long as a miner processes them in the correct order, they could even take place in the same block. Even if the perpetrator is proceeding cautiously, waiting to confirm that the first step has achieved the intended effect, there is no reason to wait more than one block before pulling the trigger on the second one. (Note that unlike Bitcoin, Ethereum network does not have a perennial backlog of transactions or arbitrary fee hikes—unless an ICO is going on. Block timestamp is a close proxy for actual attacker moves.) In other words the attacker did not bother with automating the exploit by writing a script to make the calls in quick succession. Each victim appears to have been exploited with a hand-crafted sequence of calls.

That brings up the second mystery: choice of targets. The first successful attack is followed by several hours of inactivity, two more attacks in quick succession and finally, complete silence. There is more activity from the attacker address in days following the heist; but those are all for distributing the stolen Ether into other addresses, likely in preparation for cashing out. There are no other instances of the pair of calls to a vulnerable contract. While it is conceivable this actor switched accounts to better cover its tracks, there are no other instances of theft reported besides the original attack and the white-hat response. Assuming the white-hat group is indeed a distinct actor and not simply another front for the attacker, the surprising conclusion is the perpetrator operated on a leisurely schedule and stopped after targeting just three vulnerable contracts.

The next mystery is that 12 hour pause between the first and second attacks. That initial salvo netted a respectable 26793 ETH in spoils—over $5M. But the perpetrator was clearly not content to rest on those laurels, as evidenced by the following two encores. Why wait that long? Zero-day exploits have “half-lives,” especially when their affects are noisy; and it is difficult to imagine a more noisy demonstration of an exploit than burglarizing large sums on a public blockchain. Once the first vulnerable contract is hit, the clock starts ticking for everyone else to rediscover that vulnerability independently. In general such rediscovery poses two problems for the offense. First, other crooks will get into the game, racing the original finder to exploit remaining vulnerable targets. Second, defenders will be alerted to the existence of a vulnerability. They can craft a patch that closes the window of exploitation for everyone.

Granted that second response only applies if it is possible to upgrade an application in place without losing its state. While this is true for applications running on desktop and mobile platforms (you can patch your web-browser or operating system with nothing more than a couple minutes lost rebooting) it is not true for all platforms. For instance smart-card architectures compliant with the Global Platform standard have no concept of “upgrading” an existing application— applets can only be deleted and reinstalled, with all associated data lost in the process. (And that is a security feature: it protects applications from malicious updates, even by the software publisher.) It turns out Ethereum contracts are far more similar to smart-card applets than they are to vanilla Windows apps: Ethereum has no concept of replacing the code behind a contract with new code. In fact such a capability would run counter to the ideal of immutable contracts. If the code governing a contract can be modified after the fact, it can not be counted on to behave according to predefined rules in perpetuity. But all is not lost for the defenders: in an echo of the proverb “if you can’t beat them, join them,” white-hats can co-opt the exploit, using it preemptively to rescue funds from vulnerable contracts. This is exactly what happened during the DAO attack and again with the Parity wallet in this case.

Bottom line: once the existence of a vulnerability becomes public knowledge, the odds of successful exploitation for any one actor declines down over time. (Paradoxically the chances that any given vulnerable target will be exploited goes up; there are many more threat actors armed with the required capabilities. But they are all competing against each other for the same pool of victims.) A rational attacker being aware of these dynamics would pic off as many targets as possible in the shortest time to preempt the competition. Instead there is a 12 hour self-imposed ceasefire after the first successful exploit and only two more thefts after that. Why?

It is certainly not the difficulty of locating vulnerable contracts that is holding up the attacker. They have the exact same code; they are only differentiated by arguments to the constructor specified at creation time. That means a standard blockchain explorer can help locate other contracts sharing the same code. For example Etherscan has a web interface to search for similar contracts to one specified by address:

Screen Shot 2017-07-26 at 23.34.14

Locating contracts with identical or very similar code to a given contract

Given a security flaw in a widely used smart-contract, locating all instances of that vulnerable contract is shooting fish in a barrel. So why stop with three? The white-hat group has allegedly rescued 375K+ ether, more than twice the ~150K ETH the perpetrator managed to purloin. That is a lot of money left on the table.

Even more puzzling, the wallets attacked did not even represent those with highest balances. Given the increasing probability of rediscovery over time, the optimal strategy is going after wallets with highest balances first. But among contracts rescued by the white-hat group are three with balances of 120K, 56K and 47K respectively, each one alone higher than the take from the first victim. Why pass on these more lucrative targets when the same level of effort redirected elsewhere—doing nothing beyond copy/pasting a different contract address into the exploit sequence—could have yielded higher returns?

By all indications, the execution of this attack looks sloppy:

  • No automation for delivering the exploit
  • Long pause between the first and second wave, giving defenders precious time to organize a counter-strike
  • Worst of all, suboptimal choice of victims that fails to maximize profit given complete freedom to choose targets

Yet viewed in another light, that last one may have been a deliberate decision to optimize for a different criteria: the chances of actually getting away with the theft, walking away with the ill-gotten-gains.



The optimal Ethereum heist: attacking the Parity wallet (part I)

On Jul 19 news started circulating on social media that a critical vulnerability existed in the Parity multi-signature wallet. This was quickly followed by even more startling update that the vulnerability was being actively exploited in the wild to steal funds from vulnerable wallets. By the time the dust settled, more than $30M (at the prevailing ETH/USD exchange rates) had been stolen. Meanwhile a “white-hat” group emerged, racing to use the exploit themselves  to rescue another $85M held in other vulnerable contracts from another strike by the perpetrator, sowing further confusion around who exactly are the good and bad guys in this episode. That would have been unprecedented in any other setting— akin to a vigilante neighborhood watch group preemptively looting a bank to secure its funds, knowing that a group of professional bank-robbers were going around hitting other branches. (Except it already happened once before in Ethereum: the summer of 2016 witnessed another self-appointed white-hat coalition deliberately exploiting a known smart-contract vulnerability to rescue funds remaining in the DAO during another high-profile Ethereum contract debacle.) This post examines the nature of the vulnerability, the modus operandi used by the attacker and some unresolved questions around why they did not maximize the monetary gains from this exploit.

Reverse engineering the vulnerability

While the Parity team put out an emergency PSA, they did not disclose the nature of the vulnerability, perhaps out of the mistaken notion that doing so would facilitate additional attacks. (Ironically they also committed the fix to a public repository in Github, unwittingly 0-daying themselves.) But it turns out that the vulnerability was so blatant it would have been easy to identify from the pattern of exploits. Let’s start with one of the first pieces of information that emerged during this episode: the thief hit an Ethereum contract at address 0xbec591de75b8699a3ba52f073428822d0bfc0d7e and siphoned funds to his/her own address at 0xb3764761e297d6f121e79c32a65829cd1ddb4d32. This turns out to be all the information required to work backwards and reverse engineer the bug.

Looking at the transactions originating from the attacker address around this time, we see 2 function calls into the victim contract, closely spaced in time. In fact this same pattern is repeated for two other vulnerable contracts that were exploited:
The “value” is displayed as 0 ethers in this blockchain explorer, which is misleading— it means that no funds were transferred from the attacker to the victim as part of making this function call into the smart-contract. (That makes sense; when you are trying to rob an establishment, you usually do not send them more funds.) But if we were to look at the victim view, we would find that the second, later transaction in fact resulted in the vulnerable contract transferring 82000ETH to the attacker—more than $15M at prevailing exchange rates, not bad for two function calls:

Screen Shot 2017-07-19 at 13.38.39.png

Across all three victims, the coup de grâce is delivered in this second call, resulting in transfer of funds from the targeted contract to the attacker. We will keep this in mind while diving into the call details.

Looking at the second call in a blockchain explorer, it is a call to the execute() method of the contract. If these function arguments look familiar, that’s because the first one is the Ethereum network address of the attacker. The second one is the amount in Weis to transfer to that destination address:

Screen Shot 2017-07-19 at 16.49.59.png

Why this call succeeded in transferring funds is mysterious. The source code clearly designates the function with the modifier “onlyowner,” suggesting that the developer intended for the function to be only callable by one of the contract owners. Surely the attacker is not already an owner of the contract? (Otherwise this is just an ordinary legal dispute involving insider malfeasance, not a critical vulnerability in the contract logic.) Of course it is one thing to intend for that outcome, another to achieve it; machines can only execute code, not good intentions. But looking at the implementation of the modifier and following the call chains, everything appears to be in order. By all indications, if the caller is not one of the contract owners, the execute() function will not actually execute.

Solving this puzzle requires going back to that preceding function call before the theft. We can theorize that perhaps that first call is a case of “prepping the battlefield” by placing the contract state into a vulnerable state such that the second call will succeed. Looking at the details in a blockchain explorer provides the missing clue:

Screen Shot 2017-07-19 at 13.50.07.png

That initWallet() function is supposed to be used for initializing the contract when it is originally created. It is called by the constructor and records the set of owners, the quorum required to authorize funds transfer and daily withdrawal limit. Looking at the parameters, there is that familiar attacker address again 0xb3764761e297d6f121e79c32a65829cd1ddb4d32 at parameter #5. There is also the number 1 passed in as argument. So we can posit that the attacker called this function to overwrite the contract state, listing that address as the sole owner and indicating that approval from just one owner is enough to authorize release of funds. That explains why the second call succeeded: by the time the “onlyowner” modifier was being checked to authorize the funds transfer, ownership information had already been corrupted.

So there is the vulnerability: a sensitive function that should have been only callable internally—and only during initialization of the contract— was left exposed to external calls from anyone on the blockchain. In fairness the reason for that unexpected reachability  is subtle: the Parity wallet is structured as a thin-wrapper that delegates the bulk of implementation to a much larger, shared wallet library. The vulnerable function is part of that shared library and can not be invoked directly. But the fallback method in the the outer wrapper forwards arbitrary calls to the library, effectively exposing even seemingly “internal” methods. Calling that particular function allows overwriting the ownership information, effectively redefining who controls the funds managed by the contract. Sure enough a commit to the public repo on Github shortly after the announcement confirms this theory: the fix adds a new modifier to protect the function from being called a second time after contract is already initialized.

Making a solid case for bad language design

So that is the cause in effect. But as with most vulnerabilities, it is more instructive to search for a systemic root cause— intrinsic properties of the system that made this class of error likely. Absent root causes, every vulnerability looks like bad luck: someone, somewhere made an unfortunate error or committed an oversight that they promise will never happen again. In this case a closer look at the programming language Solidity used for authoring smart-contracts suggests that some design choices in the language increased the likelihood for these errors. Specifically, Solidity defaults to allowing all methods in a contract to be invoked publicly by default. This fails the criteria of being secure by default: public methods are exposed to hostile inputs from anyone with access to the blockchain, in the same way that a service listening on a network port invites increases attack surface. Parity developers were also quick to blame Solidity in their own post-mortem:

Fourth, some blame for this bug lies with the Solidity language and, in its current incarnation, the difficulty with which one can understand the execution permissions over functions. […] We believe one or both of two ideas would help. One would be to change the default access mode of functions to “private”, rather than the eminently insecure “public”.

Ironically the bug was introduced during a refactoring, which moved the initialization code out of the constructor and into its own function. Refactoring is “a disciplined technique for restructuring an existing body of code, altering its internal structure without changing its external behavior.” It turns out in the case of Solidity, repackaging a few lines of code into a separate function does alter its semantics: those lines of code could become externally reachable outside of the original call path. (Note that it is not possible to invoke the constructor twice; there was an implicit guarantee of one-time execution present in the original version that is missing from the rearrange code.)

Language design aside, the contract itself contains some questionable logic. First notice the complexity around managing ownership: this contract allows dynamically modifying the ownership of an existing contract, voting out members or introducing new ones. How often is that logic exercised in the wild? Is it worth introducing this complexity? If it is an edge case, there are much simpler solutions: since the quorum for membership changes is identical to that for authorizing funds transfer, why not simply launch a contract with new membership and transfer all the funds there? But adding insult to injury, Solidity does not have a notion of “constantness” for object fields, the same way that “const” modifier can be applied to C++ or Java members. Confusingly there is the concept of constant state variables, but such fields must be initialized at compile time via immediate values—in contrast to the more powerful C++ version where they can be initialized using variable inputs supplied at construction time. That makes it difficult to express the notion that specific contract properties such as ownership list are immutable for the lifetime of the contract once constructed.

Given this background on why these smart-contracts were vulnerable to theft, the next post will look at some curious details around how attackers capitalized on the flaw for profit.



Bitcoin for the unbanked: receding possibilities

In October 2016 the former COO of the Chinese Bitcoin exchange BTC-C generated plenty of controversy with a nonchalant tweet starting out with this casual statement:

“Bitcoin isn’t for people that live on less than $2 a day.”

Casually dropped in the midst of one of those interminable arguments about Bitcoin scaling playing out on social-media, this statement was appalling for the implied premise. It was a complete 180 degree turn from the rhetoric surrounding the rise of cryptocurrency, with Bitcoin always the torch-bearer. Virtual currencies are liberating forces we were told, breaking up the hold of entrenched institutions on finance that rigged markets and debased currencies. Arrayed against the forces of progress was a rotating cast of villains, different for each portrayal but largely interchangeable in terms of their unfair position: the Federal Reserve, too-big-to-fail banks requiring bailouts, government agencies asleep at the wheel clinging on to outdated regulations or worse enabling those bailouts in a case of regulatory capture, Visa/MasterCard duopoly controlling online payments. Bitcoin was going to be the new money system for  everyman, unequivocally on the side of with David against the faceless institutional Goliath. No need for banks, credit-card networks, payment processors, money-transmitter licenses. Anyone with an inexpensive smart phone running a Bitcoin wallet application could send and receive money directly to anyone else anywhere in the world. No army of middle-man circling for their cut of the transaction, no gatekeepers to decide who is worthy enough to partake in this network. Also no need to worry about the debasement of hard-earned currency by an out of control printing press operated at the behest of unaccountable bureaucrats.

Rhetorical excesses aside, there were plenty of solid use cases for Bitcoin in the early days suggesting that it could help the so-called “unbanked”— more than a billion people living in developing nations without access to financial services, subsisting strictly on a cash system. No bank accounts, no way to write checks or swipe credit-cards for purchases, no credit history, no way to finance large purchases. In Africa nascent systems such as M-Pesa had already demonstrated that one could leap-frog from a standard cash economy to using smartphone wallets directly. In a strange twist, these countries skipped entire generations of earlier payment systems including check clearing, ACH, wires, PIN debit & credit networks, skipping straight to mobile wallets while US tech companies struggle and flounder in their attempts to bootstrap mobile payments. Yet M-Pesa is still a centralized technology operated by the wireless carrier Vodafone. It is crucially dependent on the coverage of retail infrastructure, specifically kiosks where consumers can go to exchange cash for M-Pesa credits.

Bitcoin offers some compelling advantages over this model. With a decentralized system, no single actor has enough leverage to squeeze the ecosystem for additional profits. Safaricom can raise M-Pesa transaction fees arbitrarily. The closest analog to a pricing cartel in Bitcoin are miners, but they face relentless pressure to keep fees competitive: if one miner decides to charge too much for mining Bitcoin transactions into the ledger, a more efficient one will come along and gladly collect the fees from those transactions. While consumers still need currency exchanges to convert between BTC and local fiat money, that function can be served by private vendors competing in a transparent market instead of under complete control of Safaricom/Vodafone. For those concerned about monetary policy, BTC offers an attractive intrinsically deflationary model. The government of Kenya can churn out shillings and Safaricom can flood the market with M-Pesa credits just as easily as Hasbro can print more Monopoly money. But no one can fabricate Bitcoins out of thin air and cause runaway inflation in BTC prices.

Given all of these factors, it is not too hard to see why Bitcoin circa 2012 looked promising for developing nations. It is another case of the surprising last-mover advantage: instead of playing catch-up with developed countries by painstakingly building out “legacy” payment rails—check clearing, card networks, eventually NFC payments—emerging markets can leapfrog straight to the latest paradigm of virtual currency.

It did not work out that way. A quick look at Bitcoin exchange statistics reveals that the majority of trading in Bitcoin is concentrated in a few markets with existing, highly-developed financial systems, not emerging markets home to millions of unbanked consumers. (China is arguably the rare exception because Bitcoin was one of the few options around capital controls in place to prevent currency outflows, but that volume has evaporated almost overnight after the Central Bank cracked down on exchanges.) It is not to difficult to see why the rosy picture painted above did not play out. For starters, BTC itself has been an extremely volatile: from an all-time high near $1300USD before the implosion of Mt Gox to dipping below $200 in two years, doubling again within a year and then embarking on a stratospheric climb towards $3000. For all the talk of inflationary policies, fluctuations in the value of Bitcoin would make even the most irresponsible, interventionist central banker look restrained by comparison. That volatility makes it less than ideal to use Bitcoin for everyday commerce, much less enter into long-term contracts denominated in BTC.

One could argue that early volatility is just growing pains for a new-fangled currency as the market struggles to discover “correct” pricing. Alternatively it can be blamed on hoards of speculators with no actual use for BTC chasing this coveted asset simply because other people are also trying to buy it—a classic case of an asset bubble. Either way, optimists expect such speculative activity will eventually diminish in scale compared to the overall volume of cryptocurrency trading, resulting in a steady state with relatively stable exchange rates. But there is one more assumption built into this lofty vision of Bitcoin, as an democratizing force that helps millions of consumers in developing countries fully participate in markets: low transaction costs. That premise looked solid in 2012 and anchored many other presumed use-cases for Bitcoin such as paying for that cup of coffee. All of these scenarios have been called into question by recent developments. In contrast with the problem of exchange-rate volatility which may well improve as the market matures, the vision of low-cost efficient payments is becoming less realistic.

The Bitcoin fee model is unusual to say the least. Most payment systems charge costs that are at least in part proportional to the value transferred. This follows a natural assumption: someone moving large amounts of money derives greater utility from that transaction than a person moving a modest amount. It follows that they would be willing to pay higher costs for the privilege of executing that transaction. (This is an oversimplification; it is also common to have fixed costs and discounts that kick-in for high amounts.) Bitcoin throws that logic out the window, charging instead based on approximate “complexity” of transactions. That complexity is indirectly measured by amount of space required to represent the transaction on the blockchain. A transaction with a single source, straightforward redeem script (“sign with this public-key”) and single destination output takes relatively few bytes to encode. One that combines multiple inputs, complex redemption conditions (“signed by 3 out of 5 keys”) and distributes those funds to multiple destinations takes up more space. Yet complexity is orthogonal to value transferred. This is what makes it possible to move $80 million USD with a few cents in transaction fees—an astonishing level of efficiency unequaled by any other payment system available to consumers—or moving $5 while paying half that amount in fees, which is extremely wasteful.

It’s as if banks charged fees for cashing checks based on how much ink there is on the check instead of their notional value. Yet this model makes sense given that space in the blockchain is itself a scarce resource. Each new block miner for extending the ledger can accommodate exactly 1MB worth of transactions. That scarcity creates natural competition for transactions trying to get mined into the next block by providing sufficient incentives to miners.

That brings us back to the question of developing markets. Back in the early days when ambitious visions of everyone paying for their next cup of coffee in bitcoin were being bandied about, those fees were negligible. Bitcoin was poised to undercut credit-card networks for retail purchases, massively undercut Western Union for international remittances and even outdo Paypal for efficient peer-to-peer payments. It even looked like the first realistic option for micro-payments, where very small amounts of money change hands very frequently: visitors to a web site donating a few cents for each article read. Fast forward to 2017, blocks are full, memory pool—that waiting queue of outstanding transactions waiting to be confirmed in the ledger— has ballooned and transactions fees are no longer negligible. Bitcoin businesses that naturally attract frequent fund movements such as exchanges have resorted to policy changes for passing transaction fees directly to customers. The only fighting chance for micro-payments today rests on the deployment of additional overlays such as the Lightning Network, implemented on top of the standard Bitcoin protocol.

Given the status quo it would be difficult disagree with the statement that Bitcoin as it exists today has very little to offer citizens of developing nations looking for an alternative payment solution for everyday purchases. Indeed the network as deployed today is not capable of clearing a large number of small transactions. (They may still find some value in its deflationary nature, as with Venezuelan citizens hoarding BTC in the midst of their economic crisis.)

It did not have to be that way. The scarcity of space is an artificial consequence of the arbitrary 1MB limit, the relic from a tactical fix implemented in response to an unrelated problem without much consideration given to future consequences. One could imagine counterfactuals where the blocksize limit is allowed to float, perhaps increasing automatically over time or adjusting in response to demand in the same way mining difficulty is constantly calibrated for constant throughput. There are clear costs to increasing blocksize: additional space required for storing a larger ledger would place demands on all nodes participating in the network. By raising costs, critics contend that such unchecked growth may force some to give up, resulting in a less decentralized network. On the other hand, the ledger is expanding every time a new block is mined and “cost” of running full node does go up measured in raw disk space. So the relevant question concerns rate of increase. Is the increased burden outpacing Moore’s law to the point that running a full node becomes more expensive in real terms? Is the growth rate predictable enough for planning future capacity? (That is a strike against quantum leaps from 1MB → 8MB because it leaves little time for adjustment.)

Arguments for and against raising block limit are being advanced daily, as are alternatives that improve throughput while leaving that sacred parameter alone. The problem is not for lack of ideas on scaling; there are too many possibilities coupled with too little consensus on which ones to pursue. The community has been unable to agree on a single solution, precipitating the current crisis with miners and users playing a game of chicken that could splinter the network on August 1st. High fees and low transaction rates have sabotaged many scenarios for using bitcoin that seemed perfectly within reach in the past. Dashed hopes for getting the millions of unbanked citizens of developing nations onboard is just one part of that collateral damage.