Tweaking speed/latency a bit and removing jitter.

Hosting an Evernode involves setting up and maintaining a linux machine which runs the evernode software and participates in network consensus. By participating, you help maintain the network's decentralized infrastructure while potentially earning rewards. Detailed tutorials and community support are available to assist node operators.
Forum rules
Welcome to the Hosting An Evernode Forum! To ensure a positive and productive environment, please adhere to the following guidelines.

Treat all members with respect and courtesy. Post content relevant to hosting an Evernode, avoiding spamming, self-promotion, or irrelevant topics. Aim to provide clear, informative, and constructive contributions. Discussions of illegal illicit activities are strictly prohibited. Protect your privacy and that of others by not sharing personal information publicly and reporting any suspicious activity to moderators. Failure to adhere to these rules may result in warnings, post deletions, or account suspension. Let’s work together to build a supportive and knowledgeable Evernode community!
Post Reply
wroomwroom
Posts: 52
Joined: Wed Jul 31, 2024 5:59 pm

Tweaking speed/latency a bit and removing jitter.

Post by wroomwroom »

Install speedtest on proxmox PVE

Code: Select all

cd /tmp && wget -q --content-disposition "https://packagecloud.io/ookla/speedtest-cli/packages/debian/bookworm/speedtest_1.2.0.84-1.ea6b6773cf_amd64.deb/download.deb?distro_version_id=215" && apt install -y ./speedtest_*.deb && speedtest --accept-license --accept-gdpr
Above command with sudo:

Code: Select all

cd /tmp && wget -q --content-disposition "https://packagecloud.io/ookla/speedtest-cli/packages/debian/bookworm/speedtest_1.2.0.84-1.ea6b6773cf_amd64.deb/download.deb?distro_version_id=215" && sudo apt install -y ./speedtest_*.deb && speedtest --accept-license --accept-gdpr
Make an initial speedtest by writing speedtest,

to do a speedtest in the same server, use speedtest -s <id of the server, which u get in ur first speedtest.>

Enabling brr is a good initial step to do before doing the rest in this thread.

Write into PVE shell:

Code: Select all

modprobe tcp_bbr
echo tcp_bbr > /etc/modules-load.d/bbr.conf

cat > /etc/sysctl.d/99-bbr.conf <<'EOF'
net.core.default_qdisc=fq
net.ipv4.tcp_congestion_control=bbr
EOF

sysctl --system
Verify with
sysctl net.ipv4.tcp_congestion_control


:::: The Rest ::::


When you do the speedtest u get information, it might for example look like this:
ISP: Digiweb
Idle Latency: 5.53 ms (jitter: 0.15ms, low: 5.34ms, high: 5.69ms)
Download: 932.26 Mbps (data used: 745.5 MB)
22.90 ms (jitter: 0.60ms, low: 7.22ms, high: 24.89ms)
Upload: 251.09 Mbps (data used: 141.9 MB)
51.55 ms (jitter: 7.28ms, low: 7.59ms, high: 147.89ms)
Packet Loss: 0.0%


We have a bit of a high top on upload. So if we cap it slightly below 250 we might get a better ms

I cap it temporareily with

Code: Select all

tc qdisc replace dev vmbr0 root cake bandwidth 235mbit besteffort
New test looks better
ISP: Digiweb
Idle Latency: 5.48 ms (jitter: 0.04ms, low: 5.43ms, high: 5.50ms)
Download: 932.77 Mbps (data used: 697.9 MB)
23.17 ms (jitter: 0.48ms, low: 7.13ms, high: 24.77ms)
Upload: 221.21 Mbps (data used: 186.7 MB)
4.75 ms (jitter: 0.63ms, low: 4.51ms, high: 33.77ms)
Packet Loss: 0.0%



Before saving permanently I'll test this with download at 920 and upload 235.

Code: Select all

modprobe ifb
ip link add ifb0 type ifb 2>/dev/null || true
ip link set ifb0 up

tc qdisc replace dev vmbr0 handle ffff: ingress
tc filter replace dev vmbr0 parent ffff: protocol all u32 match u32 0 0 action mirred egress redirect dev ifb0

Code: Select all

tc qdisc replace dev vmbr0 root cake bandwidth 235mbit besteffort
tc qdisc replace dev ifb0 root cake bandwidth 920mbit besteffort

Code: Select all

tc qdisc show dev vmbr0
tc qdisc show dev ifb0
result showing
qdisc cake 8001: root refcnt 2 bandwidth 235Mbit besteffort triple-isolate nonat nowash no-ack-filter split-gso rtt 100ms raw overhead 0
qdisc ingress ffff: parent ffff:fff1 ----------------
qdisc cake 8002: root refcnt 2 bandwidth 920Mbit besteffort triple-isolate nonat nowash no-ack-filter split-gso rtt 100ms raw overhead 0


To save my settings permanently I'll create and save following file:
/usr/local/sbin/qos.sh

Code: Select all

#!/bin/bash

modprobe ifb

ip link add ifb0 type ifb 2>/dev/null || true
ip link set ifb0 up

tc qdisc replace dev vmbr0 handle ffff: ingress
tc filter replace dev vmbr0 parent ffff: protocol all u32 \
    match u32 0 0 \
    action mirred egress redirect dev ifb0

tc qdisc replace dev vmbr0 root cake bandwidth 235mbit besteffort
tc qdisc replace dev ifb0 root cake bandwidth 920mbit besteffort
After saving I make it executeable

Code: Select all

chmod +x /usr/local/sbin/qos.sh
Then I create following file:
/etc/systemd/system/qos.service

Code: Select all

[Unit]
Description=CAKE QoS for vmbr0
After=network-online.target
Wants=network-online.target

[Service]
Type=oneshot
ExecStart=/usr/local/sbin/qos.sh
RemainAfterExit=yes

[Install]
WantedBy=multi-user.target
After saving that one I do
systemctl daemon-reload
systemctl enable --now qos.service

I have now permanently put caps.


I verify with
systemctl status qos.service
tc qdisc show dev vmbr0
tc qdisc show dev ifb0

With result
systemctl status qos.service
tc qdisc show dev vmbr0
tc qdisc show dev ifb0
Top
Post Reply

Return to “Hosting An Evernode”