The THORChain network went offline on Thursday due to a software bug, as confirmed by the blockchain operators on Twitter.
The team noted that they are earnestly working to resolve the issue as soon as possible, adding that the outage has nothing to do with solvency.
Devs are aware of a chain halt and have identified the likely cause due to a unique transaction type (nothing to do with solvency).
An update will be posted as soon as there is more confirmed information.
— THORChain (@THORChain) October 27, 2022
Non-determinism between nodes
About four hours after the update, the THORChain team shared another update, disclosing that the issue stemmed from sources of non-determinism between nodes.
“Consensus halts in a distributed state machine are from sources of non-determinism between individual nodes and prevent the ledger from becoming corrupted,” the team wrote.
The network operators went ahead to outline the steps required in resolving the problem, which included finding source of non-determinism, releasing an update, and restarting the state machine, noting that they were close to finding the source.
Moving to another three hours, the team announced they had discovered that the issue was string manipulation.
“Turns out to be string manipulation: code was pushing a cosmos.Uint (instead of a uint64) into a string, which causes the string to get the point of the big int instead of the actual value, causing the memo string to be different on different nodes.”
When Will THORChain Come Online?
The developers revealed that they undermined the issue in stagenet because the bad memo was swapped immediately and was never written in the block.
However, it affected the mainnet because it has a queue, which does not allow the swap to synth to happen on the same block, so the bad memo was written in the block.
The team also asked THORChain node operators to stand by and get ready for instructions as the patch was being shipped immediately.
However, it has been more than 15 hours since the network went offline, and there is yet to be an update from the team on when the network will resume operations.
Meanwhile, while this outage might have happened to THORChain’s, it is not the first of its kind, as several other blockchains have gone offline repeatedly.
Solana has been experiencing several network downtimes, with the most recent being late last month due to the creation of a duplicate block.