My Journey to Being a Validator on Ethereum 2.0, Part 5
This article is the last in a five-part series on how to run your own Eth2 validator. If you are new to this series, be sure to check out Part 1: Getting Started, Part 2: Validator Hardware Considerations, Part 3: Installing Metrics and Analyzing P&L and Part 4: Safely Migrating Your Eth2 Node.
For assistance running a Besu or Teku node, please go to ConsenSys Discord for assistance
The Merge date has been announced at the difficulty rate of 58.75 sextillion, which is expected to be around September 15th, 2022. And for the first time in many years, it seems like the date won’t be pushed.
In the seven years since the launch of Etheruem, and two since the launch of the Beacon Chain, the Merge has seemed unlikely to happen. Detractors said the Ethereum network had grown too large, the protocol too complicated, the ecosystem too dense with competing actors.
But there has been an incredible movement over the past two years to get us to this point. The Ethereum Core Developers (CoreDevs) pushed for the development of the Eth 2 testing spec as a benchmark for diverse client development. Client developer teams, technically all competitors, worked tirelessly together to move their clients towards that spec.
All these technical feats were complemented by a Cambrian explosion of incredibly supportive community work, exemplified best by the subreddit EthStaker (“Welcoming first, knowledgeable second”). These communities have supported the growth of solo stakers throughout their live streaming sessions, the tooling group Stakehouse and an excellent Discord.
So what still remains? In this post, we’re going to walk through the last steps to get your single validator ready for the Merge. The community has made the steps really clear:
- We need to run Eth1 (“Execution Client”, Geth, Besu, etc.) and Eth2 (“Consensus Client”, Prysm, Teku, etc.) software clients simultaneously on our validator. Previously, you could use an Infura / Alchemy endpoint but that’s not possible anymore. THIS IS A BIG DEAL. We’ll explain why below.
- Those same two clients—Eth1 (“Execution Client”) and Eth2 (“Consensus Client”) clients—require a cryptographic handshake, called a JSON Web Token (JWT), that allows them to communicate securely together.
- We need to set up an Ethereum address to collect our “mining” rewards…We’re gonna process network transactions now!
We will shuffle around the order to make it flow easier. First, we will tackle JWT, next set up a local Ethereum address, then set up a local Execution Client (Hyperledger Besu). But first, let’s talk about why we need to run an Execution Client.
Execution Client? Execution Client!
The biggest headline for solo stakers is the requirement that we need to run an Execution Client, previously called an Eth1 client, on our staking machine.
This is a big deal because syncing Execution Clients to mainnet has historically been quite challenging. (I crashed my MacBook Air laptop multiple times trying to full-sync Geth in 2018 and did it again a few days ago syncing Besu!). For example, you can’t use a spinning-state hard drive to sync a full Ethereum node because the input-output rate is not fast enough, it has to be a solid-state hard drive.
Knowing this, however, Execution Client developers have been hard at work optimizing the syncing process. Geth introduced snap-sync, Besu introduced Bonsai for optimized storage, and Nethermind also has a snap version of its sync. This doesn’t make syncing a total breeze: A full sync will still take a few days and requires significant hard-drive space. It does, however, greatly improve the experience of running an Execution Client on your machine.
Why do we have to run an Execution Client? Because Beacon Chain validators are the new miners on the Ethereum chain.
Validators on the Beacon Chain have been proposing blocks with no transactions because there are no transactions on the network yet. The Beacon Chain has been focused on stress-testing client interoperability, consensus methods, and workshopping research assumptions.
After the Merge, validators will have to do the work of miners on the old Ethereum chain. This means validating each transaction included in a block and updating the global state to reflect that transaction’s outcome.
And this work can’t be outsourced: The nature of decentralization requires nodes to “trust nothing, verify everything.” Despite its impressive capacity, we can’t use Infura to get our network state at the speed we need. Instead, we have to have a full, local version of the Ethereum world state on our machine. This will be provided by an Execution Client.
The two clients will use the Engine API to communicate securely between each other. The full Ethereum validator post-Merge is a super-organism, comprised of both an Execution Layer client and a Consensus Layer client. Here’s what that setup looks like (Beacon Node is the Consensus Layer client).
Source: AllCoreDevs Updates
As we mentioned before, Eth 1/ Execution Clients require significantly more memory than Consensus Layer clients. The Ethereum world state is significantly larger than the Beacon Chain since it’s been around longer and it has complicated transactions within it (Hello, DeFi Summer!). For example, the Execution Layer client Hyperledger Besu requires 650 GB for mainnet sync, while the Consensus Layer client Teku requires around 160 GB.
One final note: Validators can run their clients still using a third-party endpoint like Infura without being slashed. However, this will dramatically reduce the fees available to them and open them up to being robbed of their fees. Please read this article by Adrian Sutton to understand why.
First, we should have been updating our Teku clients regularly over the past few years. I have been a little lax on this front, but I do install updates when they are labeled “required” by the development team. I do not want Ben Edgington to be disappointed in me!
As we mentioned, I’m also going to need a new digital setup as the hardware requirements for an Execution Client (400-600GB) are significantly higher than my Consensus Client requirements (160GB). We’ll keep the 6-8GB core to run the validator. The increased storage significantly raises the costs of hosting digitally. A more cost-effective way would be running a setup locally, which I’ll do after the Merge.
Here are the options for hosting your full Merge-ready machine:
- Hosting it locally. Ethereum on ARM has done an exceptional job setting up custom Ubuntu images for Raspberry Pi. Unfortunately, there is a shortage of both the Raspberry Pi 4 Model B 8GB boards and other similar microcomputers. If you are hosting it locally and are comfortable with Docker, Eth-Docker is an excellent resource.
- Using a hosting site. Allnodes and other hosting services offer very affordable setups for Ethereum post-Merge ($5 / month) and Rocketpool’s Minipools ($10 / month)
- Using a virtual setup. For consistency with this series, this is the one I’ll go with. As noted above, this is not the most cost-effective and I’d like to migrate to a local setup later.
My previous setup is running the latest required Teku on a Digital Ocean droplet with 4 CPUs, 8GB RAM and 160GB memory. I’m adding 500 GB of extra memory and have also increased CPUs, as Teku is running rather hot on my machine (if you’re also experiencing this, you can try increasing heap size). I’ll be adding Besu 22.7.0 which will require 650 GB of storage. (A terabyte is out of my price range for now).
The ideal configuration on a virtual instance is 6-8 CPUs, 8GB RAM and 1TB hard-drive. This will help accommodate state growth as it occurs on the chain.
Now that we have a recap on setup, let’s go through the three essential things we need to prepare for the Merge:
JSON Web Token
We need a JSON Web Token (JWT) for our Execution Layer Client and Consensus Client to speak together securely.
When we’re getting ready to propose a block as a validator, we’ll need Besu, our Execution Client, to be able to securely pass Ethereum world state information to Teku, our Consensus Client.
Setting up a JWT is very straightforward with our Ubuntu instance. In a specific directory, please run:
openssl rand -hex 32 | tr -d "\n" > ee-jwt-secret.hex
This will generate a file, ee-jwt-secret.hex, that we’ll use in our Besu and Teku setup.
(You don’t need to install openssl or rand for Ubuntu as they come with the operating system.)
Set up an Ethereum Address
The next thing we need is an Ethereum address for our Consensus Layer client. This will collect the block rewards (minus the burned fees) from our block production, when it occurs.
This can be any Ethereum address. You can use your most sacred hexademical or your most recent from your latest MetaMask installation. Just make sure you provide it and you have access to the address! If you don’t, you won’t be able to collect the fees.
In our Teku configuration file, it goes under the value:
You can read more about it here.
Installing a local Execution Client (Hyperledger Besu)
First, we’ll have to install Besu on our local client. Besu documentation installation info here. (Note: If you’re installing another Execution Client, you can find excellent tutorials on Somer Esat’s Medium page. Please support client diversity!)
We need to download the latest Besu binary release from the release page here. We can do that using curl from our Linux Ubuntu instance:
> curl -JLO https://hyperledger.jfrog.io/artifactory/besu-binaries/besu/22.7.1/besu-22.7.1.tar.gz
Note: I’m using version 22.7.1 but you should download the latest stable release.
Decompress the downloaded file with the following command. If you have a different version, sub that file name in as opposed to besu-22.7.1.tar.gz :
> tar -zxvf besu-22.7.1.tar.gz
As always, for cleanliness sake, remove the tar.gz file. We now check to make sure the file has been installed smoothly by running the following command and making sure it doesn’t throw any errors:
> cd besu-22.7.1 && bin/besu --version
With this setup, we’re going to be using configuration files for both Besu and Teku. They take different formats (TOML for Besu and YAML for Teku), but it makes your command line much cleaner and easier to manage.
Here is the configuration file we’ll be adapting for our mainnet Hyperledger Besu sync taken from Matt Nelson’s excellent configuration file. You should use nano or another editor to put this configuration file into a local text file named besu-mainnet-config.toml.
Let’s call out a few of the configuration options you should be aware of / need to change.
This is the JWT we generated above. We’ll need this for both Besu and Teku. Be sure to substitute in the correct directory path for your machine.
This is the Engine API we mentioned earlier. It’s the common channel Besu, our Execution Client, will use to speak with Teku, our Consensus Client. It must match the engine-rpc-port option in our Teku client!
This is for the network you’d like to sync to. We’re doing mainnet here, but if you’re testing with Goerli or another testnet, you can substitute them here.
sync-mode="X_SNAP" # Can use FAST or FULL (FULL will use a lot of disk and months to sync!)
This is to prevent us from having an enormous node on our computer and relies on the Bonsai state management system from Besu. This will keep our state around 650 GB, although this may grow or shrink with network upgrades.
From the above option, we need to remove ‘ADMIN” and “DEBUG” as we’re exposing this endpoint.
systemd setup for Besu
Using the TOML configuration file for Besu makes starting our sync easy. We’ll create a systemd service very similar to the section “Create systemd service” from this post.
This step will make a service that will run Besu in the background. It will also allow the machine to automatically restart the service if it stops for some reason. This is a necessary step to make sure our validator runs 24X7.
Create a non-root user for the Besu service:
sudo useradd --no-create-home --shell /bin/false besu
Then, we create the service file by using the nano text editor, run the following line:
sudo nano /etc/systemd/system/besu.service
Rather than the larger command line options we did in a previous post, we’re going to insert the following:
Description=Besu Mainnet Node
Type command-X to exit, then type “Y” to save your changes
We have to restart “systemctl” to update it:
sudo systemctl daemon-reload
Start the service:
sudo systemctl start besu
Check to make sure it’s starting okay:
sudo systemctl status besu
If you see any errors, get more details by running:
sudo journalctl -f -u besu.service
You can stop the Besu service by running:
sudo systemctl stop besu
Once you feel you have things ironed out, enable the service to restart if it shuts down by running:
sudo systemctl enable besu
The Besu client still will take roughly 30 hours to sync and we’ll also have to monitor the state size over time to make sure it’s not overwhelming our machine. This will be really critical to check because it can interfere with our validation or even shut down the whole machine.
You should see something like the following when your Besu client is synced (the block number will be different, you can find the latest block number on Etherscan):
Imported #7,375,627 / 38 tx / 0 om / 5,418,618 (18.1%) gas / (0xb9790515c5a6abf3a7dd88130d618fc9c4b50b51b336e362f640aff95b757da2) in 0.828s. Peers: 29
Tweaking Teku, our Consensus Layer Client
If you’ve followed the series, you’ve already set up a systemd service for Teku at
In that file, please add the following parameters:
This is our Execution Client endpoint that we set up earlier for Besu. You have to make sure this matches the port you entered above!
This is the JWT that will allow Teku and Besu to speak to each other securely.
This is the fee-recipient option for Teku. Please fill this in with any valid Ethereum address whose private keys you have access to. This is the address that will collect block production fees. If you don’t list it, you lose access to those funds! (You must list a hexadecimal address, not an ENS name)
That’s it! We’ve set up everything that’s ready to go. Teku will start to interface with the Besu client. If Besu has not synced yet, you’ll see this message in the synced Teku logs:
Once Besu has completed syncing, you’ll see these logs on the Teku side:
A few last thoughts before we go:
- Don’t fear the downtime
Having a validator offline is not a terrible thing. During the Texas power outage in 2021, I wasn’t able to update my client before a required upgrade which knocked my validator off for a few days. This does not lead to slashing! This only deducts the rewards you would have earned if your validator was online. Therefore, you just need to run it for the number of days you were off to break even.
Rather than scrambling to fix something, please take the time to make sure you’re not running two instances of the same validator key or other truly slashable offenses.
- Optimize MEV
According to the Ethereum Foundation, maximal extractable value (MEV) is “the maximum value that can be extracted from block production in excess of the standard block reward and gas fees by including, excluding, and changing the order of transactions in a block.”
With Proof of Stake (PoS), validators will want to get the most amount of value for each block they propose. Flashbots has produced a piece of software called mev-boost to assist with this. Mev-boost will allow any validator to buy pre-built blocks optimized for maximal value. The value gained can be significant.
- Ignore FUD
There will certainly be other Ethereum Proof of Work hard forks that come up after the Merge. For those inclined, there’s very little incentive not to do one. Along with being aware of the dangers of interacting with ETHPoW coins, it’s really important that the community focus on the Merge and making PoS as successful as it can be.
Despite their software underpinnings, blockchain networks are fundamentally social constructions. Networks have value because people use them. The Merge is the equivalent of the moon landing for blockchains and not just Ethereum. To make it successful, a large, rowdy group of validators need to coordinate their tools and software accordingly. It’s been an incredible journey to behold with even more excitement around the corner.
- ConsenSys Discord for Teku and Besu Support
- Ethereum.org’s Nodes and Clients (Great overview of new terms after Merge)
- Recommended Specifications (provides hard-drive size for Execution Clients)
- Hard-Drive / CPU space for Consensus Clients (accurate-ish)
- Hyperledger Besu Merge Guide
- MEV-Boost install for Teku