10/18/24 Incident Post Mortem

Incident Summary

At 10:09AM UTC, October 18, 2024, Tapioca DAO suffered a security breach as a result of a social engineering attack targeted at a core contributor, which resulted in the loss of approximately 605 ETH and 3.1M USDC totaling $4.65M USD. These stolen funds are currently being held across several known addresses, with the primary address of the hacker being 0x69d91e56ca80f2a4d7b808b59053ea5c5505ffe2, where (as of time of writing) $3.915 million USD in cryptographic assets are being held on Binance Smart Chain.

The security breach was made possible via the attacker(s) compromising the private keys of a core contributor responsible for smart contract development. The compromised engineer had been a trusted lead smart contract engineer for Pearl Labs since inception, or for approximately two and a half years. SEAL911 identified the attackers as a North Korean group, used a "contagious interview" attack methodology to inject malware on the contributor’s computer in order to gain control of the private keys of his address to carry out the theft. The attacker(s) have been linked to several similar incidents involving contagious interview attacks by respected individuals in the space. The Contagious Interview is a tactic used by attackers, who pose either as a job seeker or recruiter, to trick the target into downloading files that appear legitimate but actually contain malware. The goal of this method is to exploit the trust typically associated with the hiring process.

The attacker utilized the control of the engineer’s private keys in order to:

Mint 315.5 trillion USDO stablecoins to drain the USDO/USDC liquidity pool.
Steal 29,669,866 TAP tokens from the vesting contract, valued at approximately $38 million at the time of the attack, to drain the TAP/ETH liquidity pool.

The 315.5 trillion USDO tokens were exchanged for roughly 3.1 million USDC from the USDO/USDC liquidity pool, and the 29.6M TAP tokens were exchanged for additional $1.6 million USD in ETH from the TAP/ETH liquidity pool, totaling approximately $4.75 million stolen, a supermajority of which are DAO owned funds.

This attack was made possible due to the admin controls of both token contracts being controlled by a single signer, with this signer address being a hot wallet. Security personnel and other contributors had previously repeatedly directed the compromised engineer to transfer the admin role of the smart contracts to the DAO’s 4-of-7 multisignature wallet after the June 2024 deployment of the Tapioca protocol. The engineer was only directed to implement single-signer controls for pause functionality to enable swift reaction to any security events, and to integrate Hypernative security monitoring. Additionally, the DAO provided the compromised engineer with a cold storage wallet, hardware 2FA key, and other security devices. Despite these directives and precautions, the engineer failed to follow or implement the directives, use any of the provided devices, and further additionally received and ignored a direct warning about “contagious” interview attacks less than three months prior.

May 7th, 2024, Task created to script the transfer of the multicall to the multisig, engineer closes it as completed

June 25th, 2024, another task created to do above task, closed as completed again.

Contagious Interview warning, July 20th, 2024, internal Slack channel

The engineer, who was compromised, both stated that the directives were properly completed and created the appearance of completion by closing task(s) on ClickUp. This was mistakenly taken as fact by the DAO due to his seniority and the absence of formal compliance measures.

Attack Methodology

The social engineering attack started when the compromised engineer was targeted through LinkedIn, and further communicated through Telegram by and with an individual posing as “Alberto Flores Galvan,” who was offering the engineer a fake job opportunity.

Attacker's LinkedIn Profile, please note: the attacker is most likely impersonating or fabricating this identity, and if real, this identity used is most likely another victim.

The engineer was asked to download a malicious .env file containing malware, leading to the compromise.

Once the attacker(s) obtained the compromised engineer’s private key, they transferred the admin ownership of the TAP token’s vesting contract, stole 29.6M TAP tokens held within it via a “rescue” function, then liquidated these stolen tokens to steal the majority DAO owned ETH liquidity held within the TAP/ETH liquidity pool. Admin Ownership within token vesting contracts in of itself is a common practice within DeFi to modify vesting recipient addresses.

The attacker then added a new address to USDO’s whitelist of minters, minted USDO stablecoins, and stole the DAO owned USDC liquidity in the USDO/USDC liquidity pool with the illegitimately procured USDO. The purpose of this whitelist was similar to Aave’s GHO “facilitators”, wherein if the Tapioca DAO elected to create new methodologies to mint USDO, they could be directly added as a minter to the contract.

Attached below are the recoverable screenshots of the conversation between the engineer and attacker on LinkedIn & Telegram. The only information redacted is the first name and handle of the engineer.

LinkedIn Conversation between Engineer & Attacker

Telegram Conversation & Info between Attacker and Engineer

To provide context for the compromised engineer’s actions, Pearl Labs began restructuring the engineering staff in June 2024, prompted by the twAML economic issue which caused the Tapioca protocol to be paused. EnigmaDarkLabs was brought in at this point to advise Pearl Labs and oversee development. Enigma then performed an analysis of development processes and recommended that both senior engineers be terminated for lack of expertise, sluggishness, and recklessness. Based on this recommendation, Pearl Labs leadership reached a compromise, one of the two engineers would be terminated, while the other, would be demoted to a junior developer role while new experienced engineers would assume the CTO and Senior engineer roles.

However, due to the timelines to employ new engineers to develop and deploy Tapioca V1.1, both engineers were retained in their existing positions to complete V1.1 under the supervision of internal security personnel while leadership carried out the restructure. As both of the original engineers' existing issues worsened, one of the two aforementioned engineers was terminated before the release of V1.1- just two days prior to this incident taking place. This termination prompted the compromised engineer to rapidly seek employment without anyone’s knowledge, making them especially vulnerable to the contagious interview attack. This, in concert with the compromised engineer’s insubordination of directives regarding contract admin controls and lack of utilization of any of the provided security devices, provided the attacker an easy pathway to carry out this simple attack.

Incident Response

The security breach was reported at 11:18 AM UTC, and within minutes, security personnel engaged SEAL911. A war room was established by 11:22 AM UTC, and the root cause was identified within five minutes: the attacker had full control of the admin role, preventing any immediate countermeasures to stop the theft. The team quickly removed liquidity from the two key Uniswap V3 pools, recovering approximately $700,000 USD.

Further investigation revealed that the attacker had 996 ETH ($2.65 million USD) in collateral under their control. With the assistance of EnigmaDarkLabs, the security team utilized an exploit to recover the ETH from the attacker before he could launder it. As a result, the DAO recovered $2.65M USD from the hacker’s control. At this time it has been advised to withhold the specifics of this exploit due to ongoing circumstances. However, as soon as possible, this will be provided as an Addendum to the Post Mortem.

Following the attack, the DAO's treasury, excluding TAP tokens, stands at ~$4.8 million, representing a 45% loss from where it stood before the social engineering attack.

Detailed Events Timeline

Arbitrum: Ownership of Tapioca Multicall (Deployer) contract gets transferred to attacker:

Oct-18-2024 10:09:07 AM +UTC

Arbitrum One Transaction Hash: 0x79a17d0848... | Arbitrum One

Call Transfer Ownership Function By Tapioca: Deployer on 0x302E9128...c61F7fDED | Success | Oct-18-2024 10:09:07 AM (UTC)

No comments yet

Incident Summary

Attack Methodology

Incident Response

Detailed Events Timeline

Arbitrum One Transaction Hash: 0x79a17d0848... | Arbitrum One

Arbitrum One Transaction Hash: 0x2ad98dd29c... | Arbitrum One

Arbitrum One Transaction Hash: 0x0bca43cfb5... | Arbitrum One

Arbitrum One Transaction Hash: 0xaf9fe4593a... | Arbitrum One

Arbitrum One Transaction Hash: 0xae9559d612... | Arbitrum One

Arbitrum One Transaction Hash: 0xe62716b847... | Arbitrum One

Arbitrum One Transaction Hash: 0x2eb546873a... | Arbitrum One

Arbitrum One Transaction Hash: 0x8cf8def40f... | Arbitrum One

Arbitrum One Transaction Hash: 0x1abb8cf0b0... | Arbitrum One

Arbitrum One Transaction Hash: 0x174c3deaf5... | Arbitrum One

Arbitrum One Transaction Hash: 0x9223b9970c... | Arbitrum One

Arbitrum One Transaction Hash: 0x76fd3a27ad... | Arbitrum One

Arbitrum One Transaction Hash: 0x0cd1a06a99... | Arbitrum One

Arbitrum One Transaction Hash: 0x8bccaabadc... | Arbitrum One

Arbitrum One Transaction Hash: 0x8b62af841f... | Arbitrum One

Arbitrum One Transaction Hash: 0x891d9be052... | Arbitrum One

Arbitrum One Transaction Hash: 0xc324014cac... | Arbitrum One

Arbitrum One Transaction Hash: 0x23816332ac... | Arbitrum One

Arbitrum One Transaction Hash: 0xa95a9561dc... | Arbitrum One

Arbitrum One Transaction Hash: 0x43fabb8a2a... | Arbitrum One

Security Measures

Web3 Security Measures:

Multisig Security Measures:

Conclusion

10/18/24 Incident Post Mortem

No comments yet