Yesterday, the Zilliqa mainnet underwent an emergency upgrade to fix an issue that caused mining nodes to run out of memory when processing large smart contracts on the protocol.
The memory issue was detected the day before when the Zilliqa mainnet and testnet went offline. After this incident, the Zilliqa team informed the community about the network downtime via the tweet below.
Both our mainnet and testnet are currently down. Our team is quickly working to resolve this.
We will update you once we resolve it and confirm what the issue was. We apologise for any inconvenience caused!
As always, your funds remain SAFU.
The Zilliqa Team
— Zilliqa (@zilliqa) July 13, 2021
Fixed memory issue in Zilliqa’s mainnet v8.0.
Jun Hao Tan, Zilliqa’s Co-Founder and veteran Vice President of Security and Platform Engineering, explained that the issue was “the result of memory fragmentation, which failed to return memory to the kernel.”
The team on the project has resolved the aforementioned issue with the release of Zilliqa Mainnet v8.0.5 and the Zilliqa mainnet and testnet are now running smoothly.
“Zilliqa is State of the art and Difficulties are normal”
The Zilliqa team explained to the community that since the last major network upgrade of v8.0 in May of this year, the network has become unstable in some cases.
The team pointed out that v8.0 is full of features as described below.
It reduced block time, revised the consensus protocol, introduced remote status reading for Scilla, adjusted priorities for new miners, and included several other key optimizations and bug fixes.
As a result, the network became unstable several times, causing the core team at Zilliqa to “intervene several times to introduce patches, causing unwanted network downtime.”
Also, the Reddit post highlighted that Zilliqa applies the latest technologies. Therefore, it is sometimes not easy to identify major errors during the testing phase. The script was described as:
However, many of these cutting-edge innovations in engineering and development pose a significant challenge. Until these changes go live on a production mainnet system, it’s impossible to be sure that the changes made are error-free. While we may stop testing anything in production, sometimes it’s clear that the real test is always in production.
It’s nearly impossible to guarantee that a particular piece of software is bug-free, but we certainly try to ensure that. Despite these challenges, we are committed to constantly innovating and adding new features to the chain with minimal downtime and errors.
The team at Zilliqa concluded their Reddit post by thanking the ZIL community for their continued understanding and support as we strive to make the protocol better and more robust.