VodMeister 1.0.0

[VodMeister]

A Scaleable Video On Demand experiment utilizing Evernode Nodes.

Two docker containers:

1. The video stream server
2. The load balancer server.

Functionality:
The video stream server is delivering video in chunks. You can have a lot of these servers if you want to divide the traffic among more than just 1 server.

The load balancer server is delivering video content from a list of video stream servers at random. If one is slow or doesn't work an other will be used instead.

The load balancer server is also load balanced locally on the users webbrowser, when he visits the video page his browser is fetching the list of load balancers, and if one load balancer is down the videoplayer will try fetching from an other one.

Admin
There's an admin page ( https://domnin.tld:port/admin.html ), Default password is: mypassword

At the admin page you can view your video stream server lists, bulk add/remove them and sync them.

You can also view your load balancer list, bulk add/remove and sync them.

When you sync, you are syncing to all the load balancers in your list.

So it's smart to fish remove the example load balancers I've added and then add your load balancers. Then after that, you sync your load balancers and when that's done, you can add/remove + sync your video stream servers as you wish. The video streams can be added/removed in action, because the balancers are monitoring the video stream list. However, you obviously need to sync that list aswell to all the other load balancers.


TRY IT NOW
evdevkit acquire -i vodmeister/loadbalancer:latest rAddr -m 1
evdevkit acquire -i vodmeister/videostream:latest rAddr -m 1

You can easily try it by deploying this docker upon acquiring an instance.


Docker Container Video Stream:
Folder /

index.js

Code: Select all

const http = require('http');
const fs = require('fs');
const path = require('path');

const videoDir = path.join(__dirname, 'video_segments');

// Function to create and start a server on a specific port
function createServer(port) {
  const server = http.createServer((req, res) => {
    // Set CORS headers
    res.setHeader('Access-Control-Allow-Origin', '*');
    res.setHeader('Access-Control-Allow-Methods', 'GET,HEAD,OPTIONS');
    res.setHeader('Access-Control-Allow-Headers', 'Origin, X-Requested-With, Content-Type, Accept, Range');

    // Handle OPTIONS method (CORS preflight)
    if (req.method === 'OPTIONS') {
      res.writeHead(204);
      res.end();
      return;
    }

    const filePath = path.join(videoDir, req.url);

    fs.stat(filePath, (err, stats) => {
      if (err) {
        res.writeHead(404, { 'Content-Type': 'text/plain' });
        res.end('404 Not Found');
        return;
      }

      if (stats.isDirectory()) {
        res.writeHead(403, { 'Content-Type': 'text/plain' });
        res.end('403 Forbidden');
        return;
      }

      let contentType = 'video/mp4';
      if (filePath.endsWith('.m3u8')) {
        contentType = 'application/x-mpegURL';
      } else if (filePath.endsWith('.mp4')) {
        contentType = 'video/mp4';
      }

      res.writeHead(200, { 'Content-Type': contentType });

      const readStream = fs.createReadStream(filePath);
      readStream.on('error', (streamErr) => {
        res.writeHead(500, { 'Content-Type': 'text/plain' });
        res.end('Internal Server Error');
      });
      readStream.pipe(res);
    });
  });

  server.listen(port, () => {
    console.log(`Server running at http://localhost:${port}/`);
  });
}

// Start server on each port in the range 36525-36535
for (let port = 36525; port <= 36535; port++) {
  createServer(port);
}

package.json

Code: Select all

{
  "name": "evernode-videostream",
  "version": "1.0.0",
  "description": "Evernode VideoStream",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "author": "",
  "license": "ISC"
}

start.sh

Code: Select all

#!/bin/sh

# Run vod
/usr/bin/node /usr/local/bin/hotpocket/vod &

# Set the HotPocket binary as the entry point
# $@ is used to pass all the commandline arguments fed to this script into hpcore.
/usr/local/bin/hotpocket/hpcore $@

Dockerfile

Code: Select all

FROM evernode/sashimono:hp.latest-ubt.20.04-njs.20

RUN mkdir -p /usr/local/bin/hotpocket/vod
RUN mkdir -p /usr/local/bin/hotpocket/vod/video_segments

COPY / /usr/local/bin/hotpocket/vod
COPY /video_segments /usr/local/bin/hotpocket/vod/video_segments

COPY start.sh /usr/local/bin/hotpocket/start.sh
RUN chmod +x /usr/local/bin/hotpocket/start.sh

ENTRYPOINT ["/usr/local/bin/hotpocket/start.sh"]

EXPOSE 36525-36535

path /video_segments/
Here you will find an index.m3u8 file and index0.ts to index36.ts . These are the video chunks, index.m3u8 is the playlist that the webplayer reads.

Generated with ffmpeg
ffmpeg -i "path to videoname.mp4" -codec: copy -start_number 0 -hls_time 10 -hls_list_size 0 -f hls index.m3u8

Currently each chunk is 10 seconds.



Docker Container Video Delivery Loadbalancer:

/public/admin.html

Code: Select all

<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="UTF-8">
  <meta name="viewport" content="width=device-width, initial-scale=1.0">
  <title>Admin Page</title>
</head>
<body>
  <h1>Admin Page</h1>

  <!-- Authorization Token Input -->
  <label for="authToken">Authorization Token (default password is mypassword):</label><br>
  <input type="password" id="authToken" placeholder="Enter your token">
  <br><br>

  <!-- Bulk Add Servers -->
  <h2>Add Video Delivery Nodes</h2>
  <p style="max-width:350px;">The load balancers are picking video delivery nodes at random, so a large list is very helpful for stability.</p>
  <textarea id="serversToAdd" rows="5" cols="50" placeholder="Enter servers in format ip:port or domain:port, one per line"></textarea>
  <br>
  <button onclick="bulkAddServers()">Add Servers</button>

  <!-- Bulk Remove Servers -->
  <h2>Remove Video Delivery Nodes</h2>
  <p style="max-width:350px;">Don't forget to remove the pre-configured example nodes.</p>
  <textarea id="serversToRemove" rows="5" cols="50" placeholder="Enter servers to remove in format ip:port or domain:port, one per line"></textarea>
  <br>
  <button onclick="bulkRemoveServers()">Remove Servers</button>

  <!-- View Current Servers -->
  <h2>Current Video Delivery Nodes</h2>
  <button onclick="viewServers()">View Servers</button>
  <pre id="serverList"></pre>
  
  <br><br>

  <!-- Bulk Add Load Balancers -->
  <h2>Add Load Balancers</h2><br>
  <p style="max-width:350px;">You need at least two load balancers, because if you only have one, then the system got no alternative to try if there's a connection failure. The load balancers are being stored locally at the visitor's web browser, so add these before sending out the website. Videostreams can be added/removed whenever you want.</p>
  <textarea id="loadBalancersToAdd" rows="5" cols="50" placeholder="Enter load balancers in format ip:port or domain:port, one per line"></textarea>
  <br>
  <button onclick="bulkAddLoadBalancers()">Add Load Balancers</button>

  <!-- Bulk Remove Load Balancers -->
  <h2>Remove Load Balancers</h2><br>
  <p style="max-width:350px;">Don't forget to remove the pre-configured example balancers.</p>
  <textarea id="loadBalancersToRemove" rows="5" cols="50" placeholder="Enter load balancers to remove in format ip:port or domain:port, one per line"></textarea>
  <br>
  <button onclick="bulkRemoveLoadBalancers()">Remove Load Balancers</button>

  <!-- View Current Load Balancers -->
  <h2>Current Load Balancers</h2>
  <button onclick="viewLoadBalancers()">View Load Balancers</button>
  <pre id="loadBalancerList"></pre>

  <br><br>

  <!-- Synchronize Buttons -->
  <h2>Synchronize</h2>
  <p style="max-width:350px;">Add your load balancers and stream nodes first, and then sync afterwards. Once you click the sync buttons, they will sync with all load balancers.</p>
  <button onclick="syncLoadBalancers()">Sync Load Balancers</button>
  <button onclick="syncServers()">Sync Servers</button>

  <!-- Sync Results -->
  <h3>Sync Results</h3>
  <pre id="syncResults"></pre>

  <br><br>

  <!-- Update SSL Certificates -->
  <h2>Update SSL Certificates</h2>
  <p style="max-width:350px;">Enter the load balancers' host:port, private key, and certificate in the fields below to update the SSL certificates for multiple load balancers.</p>

  <label for="loadBalancerHosts">Load Balancer Host:Port (one per line):</label><br>
  <textarea id="loadBalancerHosts" rows="5" cols="50" placeholder="Enter load balancers host:port, one per line"></textarea>
  <br><br>

  <label for="privateKey">Private Key:</label><br>
  <textarea id="privateKey" rows="5" cols="50" placeholder="Paste your private key here"></textarea>
  <br><br>

  <label for="certificate">Certificate:</label><br>
  <textarea id="certificate" rows="5" cols="50" placeholder="Paste your certificate here"></textarea>
  <br><br>

  <button onclick="updateSSLCertForLoadBalancers()">Update SSL Certificates for Load Balancers</button>
  <br><br>

  <!-- SSL Update Results -->
  <h3>SSL Update Results</h3>
  <pre id="sslUpdateResults"></pre>

  <!-- JavaScript -->
  <script>
    // Hashing function using SHA-256
    async function hashToken(token) {
      const encoder = new TextEncoder();
      const data = encoder.encode(token);
      const hash = await crypto.subtle.digest('SHA-256', data);
      return Array.from(new Uint8Array(hash)).map(b => b.toString(16).padStart(2, '0')).join('');
    }

    async function sendRequest(url, data) {
      const authToken = document.getElementById('authToken').value;
      const hashedToken = await hashToken(authToken);
      const response = await fetch(url, {
        method: 'POST',
        headers: {
          'Content-Type': 'application/json',
          'Authorization': hashedToken  // Send the hashed token
        },
        body: JSON.stringify(data)
      });


  // Read the response body once
  const responseBody = await response.text();

  // Attempt to parse the response as JSON
  try {
    const result = JSON.parse(responseBody);
    return result;  // Return the result for further processing
  } catch (error) {
    // If parsing fails, return the text response as an error
    return { error: true, message: responseBody };  // Return the text response as an error
  }
}

    async function getRequest(url, resultElementId) {
      const authToken = document.getElementById('authToken').value;
      const hashedToken = await hashToken(authToken);
      const response = await fetch(url, {
        method: 'GET',
        headers: {
          'Authorization': hashedToken  // Send the hashed token
        }
      });

      if (response.ok) {
        const result = await response.json();
        // Format the result as host:port
        const formattedResult = result.map(item => `${item.host}:${item.port}`).join('\n');
        document.getElementById(resultElementId).textContent = formattedResult;
        return result;
      } else {
        document.getElementById(resultElementId).textContent = 'Failed to fetch data: ' + response.statusText;
        return null;
      }
    }

    function parseInput(input) {
      return input.split('\n').filter(line => line.trim()).map(line => {
        const [host, port] = line.split(':');
        return { host: host.trim(), port: parseInt(port.trim()) };
      });
    }

    function bulkAddServers() {
      const servers = parseInput(document.getElementById('serversToAdd').value);
      sendRequest('/api/servers/add', { servers });
    }

    function bulkRemoveServers() {
      const servers = parseInput(document.getElementById('serversToRemove').value);
      sendRequest('/api/servers/remove', { servers });
    }

    function viewServers() {
      getRequest('/api/servers', 'serverList');
    }

    function bulkAddLoadBalancers() {
      const loadbalancers = parseInput(document.getElementById('loadBalancersToAdd').value);
      sendRequest('/api/loadbalancers/add', { loadbalancers });
    }

    function bulkRemoveLoadBalancers() {
      const loadbalancers = parseInput(document.getElementById('loadBalancersToRemove').value);
      sendRequest('/api/loadbalancers/remove', { loadbalancers });
    }

    function viewLoadBalancers() {
      getRequest('/api/loadbalancers', 'loadBalancerList');
    }

    function delay(ms) {
      return new Promise(resolve => setTimeout(resolve, ms));
    }

    async function syncLoadBalancers() {
      const syncResultsElement = document.getElementById('syncResults');
      syncResultsElement.textContent = ''; // Clear previous results

      const loadBalancers = await getRequest('/api/loadbalancers', 'loadBalancerList');
      if (loadBalancers) {
        for (const lb of loadBalancers) {
          try {
            await sendRequest(`/api/loadbalancers/sync`, { loadbalancers: loadBalancers });
            syncResultsElement.textContent += `${lb.host}:${lb.port} - Sync Sent\n`;
          } catch (error) {
            syncResultsElement.textContent += `${lb.host}:${lb.port} - Failed: ${error.message}\n`;
          }
          await delay(1000); // 1-second delay
        }
      }
    }

    async function syncServers() {
      const syncResultsElement = document.getElementById('syncResults');
      syncResultsElement.textContent = ''; // Clear previous results

      const loadBalancers = await getRequest('/api/loadbalancers', 'loadBalancerList');
      if (loadBalancers) {
        for (const lb of loadBalancers) {
          try {
            await sendRequest(`/api/servers/sync`, { servers: loadBalancers });
            syncResultsElement.textContent += `${lb.host}:${lb.port} - Sync Sent\n`;
          } catch (error) {
            syncResultsElement.textContent += `${lb.host}:${lb.port} - Failed: ${error.message}\n`;
          }
          await delay(1000); // 1-second delay
        }
      }
    }

    async function updateSSLCertForLoadBalancers() {
      const hosts = document.getElementById('loadBalancerHosts').value.split('\n').filter(line => line.trim());
      const key = document.getElementById('privateKey').value;
      const cert = document.getElementById('certificate').value;
      const sslUpdateResultsElement = document.getElementById('sslUpdateResults');
      sslUpdateResultsElement.textContent = ''; // Clear previous results

      if (hosts.length === 0 || !key || !cert) {
        sslUpdateResultsElement.textContent = 'Host:Port, Private Key, and Certificate are all required.';
        return;
      }

      for (const hostPort of hosts) {
        const [host, port] = hostPort.split(':');
        if (!host || !port) {
          sslUpdateResultsElement.textContent += `Invalid Host:Port format for ${hostPort.trim()}.\n`;
          continue;
        }

        const data = { key, cert };
        const url = `https://${host.trim()}:${parseInt(port.trim())}/update-certificates`;

        try {
          const result = await sendRequest(url, data);

          if (result.error) {
            sslUpdateResultsElement.textContent += `${host.trim()}:${port.trim()} - ${result.message}\n`;
          } else {
            sslUpdateResultsElement.textContent += `${host.trim()}:${port.trim()} - SSL Update Sent\n`;
          }
        } catch (error) {
          sslUpdateResultsElement.textContent += `${host.trim()}:${port.trim()} - Error: ${error.message}\n`;
        }

        await delay(1000); // 1-second delay
      }
    }
  </script>
</body>
</html>

/public/evernode-logo.png
just design

/public/logo-large.svg
not used, but its there anyways

/public/web.html

Code: Select all

<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="UTF-8">
  <meta name="viewport" content="width=device-width, initial-scale=1.0">
  <title>Evernode VodMeister 1.0.0</title>
  <style>
    body {min-height:100vh;display:flex;flex-direction:column;justify-content:center;align-items:center;background:#050f2c;overflow:hidden;}
    video {
      width: 100%;
      height: 100%;
      background-color: black;
      
    }
    #playButton {
      margin-top: -20px;
      margin-left:-80px;
      height:40px;
      width:180px;
      font-size: 16px;
      position:absolute;
      left:50%;
      top:50%;
      display:flex;flex-direction:row;justify-content:center;align-items:center;
      background:#050f2c;
      color:#FFF;
      font-weight:600;
      cursor:pointer;
      border-radius:50px;
      border-style:none;
    }
    .videowrapper {width:100%;max-width:800px;position:relative;display:flex;flex-direction:column;justify-content:center;align-items:center;}
    .logo {margin-bottom:75px;display:flex;flex-direction:column;align-items:center;justify-content:center;}
     span {margin-top:16px;font-size:16px;font-weight:600;color:#FFF;text-align:center;}
    .videodiv {position:relative;width:100%;}
    .below {margin-top:25px;display:flex;flex-direction:column;justify-content:center;align-items:center;}
  </style>
</head>
<body>

 <div class="videowrapper"> 
  <div class="logo"><img src="evernode-logo.png"><span>VodMeister 1.0.0</span></div>

<div class="videodiv">  <video id="my-video" controls preload="auto" width="640" height="264">
    Your browser does not support the video tag.
  </video>
  <button id="playButton"><img src="logo-large.svg" style="width:25px;height:25px;padding-right:6px;">Start Video</button></div>
<div class="below"><span>Decentralized, Scale-As-You-Go, Globally Distributed Video On Demand Solution Built On the Evernode Network. The Go-To Platform For Decentralized Infrastructure.</span></div>

</div>
  <!-- Include HLS.js library -->
  <script src="https://cdn.jsdelivr.net/npm/hls.js@latest"></script>
  <script src="/servers.js"></script>
  <script>

    const videoPath = '/index.m3u8';
    const videoElement = document.getElementById('my-video');
    const playButton = document.getElementById('playButton');
    let lastPlaybackTime = 0;
    let hls = null;
    let currentServerIndex = 0;

    playButton.addEventListener('click', () => {
      playButton.style.display = 'none';  // Hide the play button after it's clicked
      shuffleServers();  // Shuffle servers before starting playback
      tryPlayVideo();
    });

    function shuffleServers() {
      for (let i = servers.length - 1; i > 0; i--) {
        const j = Math.floor(Math.random() * (i + 1));
        [servers[i], servers[j]] = [servers[j], servers[i]];
      }
      console.log("Shuffled server list:", servers);
    }

    function tryPlayVideo() {
      const videoUrl = servers[currentServerIndex] + videoPath;
      console.log(`Attempting to play video from: ${videoUrl}`);

      if (hls) {
        hls.destroy(); // Clean up any existing HLS instance
      }

      if (Hls.isSupported()) {
        hls = new Hls({
          manifestLoadingRetryDelay: 1000, // Delay between retries in milliseconds
          manifestLoadingMaxRetry: 1,      // Max number of retries before giving up
          manifestLoadingMaxRetryTimeout: 4000, // Max time in ms before giving up
          enableWorker: true,              // Use web workers to avoid blocking UI
          lowLatencyMode: true,            // Enable low latency mode
          backBufferLength: 90,            // Keep enough back buffer for smooth playback
        });

        hls.loadSource(videoUrl);
        hls.attachMedia(videoElement);

        hls.on(Hls.Events.MANIFEST_PARSED, () => {
          console.log(`Manifest parsed. Attempting to resume video at time: ${lastPlaybackTime}`);
          videoElement.currentTime = lastPlaybackTime;

          videoElement.play().then(() => {
            console.log(`Playback started at time: ${videoElement.currentTime}`);
          }).catch(error => {
            console.error('Error starting playback:', error);
            capturePlaybackTimeAndSwitch(); // Capture time and try the next server
          });
        });

        hls.on(Hls.Events.ERROR, (event, data) => {
          console.error(`HLS.js Error: ${data.details}`);
          if (data.fatal) {
            capturePlaybackTimeAndSwitch(); // Handle fatal errors by switching server
          } else if (data.details === Hls.ErrorDetails.FRAG_LOAD_ERROR || data.details === Hls.ErrorDetails.FRAG_LOAD_TIMEOUT) {
            // Continue playing buffered content while attempting to fetch new data
            console.warn('Fragment load error or timeout, continuing to play buffered content...');
          }
        });

      } else if (videoElement.canPlayType('application/vnd.apple.mpegurl')) {
        videoElement.src = videoUrl;
        videoElement.currentTime = lastPlaybackTime;

        videoElement.addEventListener('error', () => {
          console.error(`Playback error for: ${videoUrl}`);
          capturePlaybackTimeAndSwitch(); // Capture time and try the next server
        });

        videoElement.play().then(() => {
          console.log(`Playback started at time: ${videoElement.currentTime}`);
        }).catch(error => {
          console.error('Error starting playback:', error);
          capturePlaybackTimeAndSwitch(); // Capture time and try the next server
        });
      } else {
        alert('Your browser does not support HLS playback.');
      }
    }

    function capturePlaybackTimeAndSwitch() {
      lastPlaybackTime = videoElement.currentTime || lastPlaybackTime;
      console.log(`Captured playback time before switching: ${lastPlaybackTime}`);
      switchServer();
    }

    function switchServer() {
      currentServerIndex = (currentServerIndex + 1) % servers.length;
      console.log(`Switching to server ${currentServerIndex + 1} of ${servers.length}. Current playback time: ${lastPlaybackTime}`);
      tryPlayVideo(); // Try the next server
    }

    // Store the current playback time periodically
    videoElement.addEventListener('timeupdate', () => {
      if (videoElement.currentTime > 0) {
        lastPlaybackTime = videoElement.currentTime;
        console.log(`Updated playback time: ${lastPlaybackTime}`);
      }
    });

    // Explicitly set playback time after loading metadata
    videoElement.addEventListener('loadedmetadata', () => {
      console.log(`Metadata loaded. Setting currentTime to ${lastPlaybackTime}`);
      videoElement.currentTime = lastPlaybackTime;
    });
  </script>

</body>
</html>

/node_modules
containing the stuff u can create with npm i based on the package.json file.

/Dockerfile

Code: Select all

FROM evernode/sashimono:hp.latest-ubt.20.04-njs.20

RUN mkdir -p /usr/local/bin/hotpocket/loadbalancer
RUN mkdir -p /usr/local/bin/hotpocket/loadbalancer/public

COPY / /usr/local/bin/hotpocket/loadbalancer
COPY /public /usr/local/bin/hotpocket/loadbalancer/public

COPY start.sh /usr/local/bin/hotpocket/start.sh
RUN chmod +x /usr/local/bin/hotpocket/start.sh

ENTRYPOINT ["/usr/local/bin/hotpocket/start.sh"]
EXPOSE 36525-36535

/index.js

Code: Select all

process.env.NODE_TLS_REJECT_UNAUTHORIZED = '0';

const https = require('https');
const httpProxy = require('http-proxy');
const fs = require('fs');
const express = require('express');
const bodyParser = require('body-parser');
const path = require('path');

function loadSSLOptions() {
  return {
    key: fs.readFileSync(path.resolve(__dirname, 'server.key')),
    cert: fs.readFileSync(path.resolve(__dirname, 'server.cert')),
  };
}

let options = loadSSLOptions();

// Create a proxy server
const proxy = httpProxy.createProxyServer({});

// Authentication token (for simplicity)
const AUTH_TOKEN = '89e01536ac207279409d4de1e5253e01f4a1769e696db0d6062ca9b8f56767c8'; // default password: mypassword, sha256 hashed

// Function to read servers from servers.txt
function loadServersFromFile(filename) {
  const serverList = [];
  const filePath = path.resolve(__dirname, filename);

  if (!fs.existsSync(filePath)) {
    throw new Error(`File not found: ${filePath}`);
  }

  const lines = fs.readFileSync(filePath, 'utf-8').split('\n');
  for (const line of lines) {
    const [host, port] = line.trim().split(':');
    if (host && port) {
      serverList.push({ host, port: parseInt(port, 10) });
    }
  }

  return serverList;
}

// Load servers from servers.txt
let servers = loadServersFromFile('servers.txt');

// Function to save servers to servers.txt
function saveServersToFile(filename, servers) {
  const filePath = path.resolve(__dirname, filename);
  const fileContent = servers.map(server => `${server.host}:${server.port}`).join('\n');
  fs.writeFileSync(filePath, fileContent, 'utf-8');
}

// Function to watch the servers.txt file for changes
function watchServersFile(filename) {
  const filePath = path.resolve(__dirname, filename);
  fs.watch(filePath, (eventType) => {
    if (eventType === 'change') {
      console.log('servers.txt file has changed, reloading servers...');
      servers = loadServersFromFile(filename);
    }
  });
}

// Start watching the servers.txt file
watchServersFile('servers.txt');

// Function to get a random backend server
function getRandomServer() {
  if (servers.length === 0) {
    throw new Error('No servers available.');
  }
  const randomIndex = Math.floor(Math.random() * servers.length);
  return servers[randomIndex];
}

// Function to read load balancers from loadbalancers.txt
function loadLoadBalancersFromFile(filename) {
  const loadBalancerList = [];
  const filePath = path.resolve(__dirname, filename);

  if (!fs.existsSync(filePath)) {
    throw new Error(`File not found: ${filePath}`);
  }

  const lines = fs.readFileSync(filePath, 'utf-8').split('\n');
  for (const line of lines) {
    const [host, port] = line.trim().split(':');
    if (host && port) {
      loadBalancerList.push({ host, port: parseInt(port) });
    }
  }

  return loadBalancerList;
}

// Load load balancers from loadbalancers.txt
let loadBalancers = loadLoadBalancersFromFile('loadbalancers.txt');

// Function to save load balancers to loadbalancers.txt
function saveLoadBalancersToFile(filename, loadBalancers) {
  const filePath = path.resolve(__dirname, filename);
  const fileContent = loadBalancers.map(lb => `${lb.host}:${lb.port}`).join('\n');
  fs.writeFileSync(filePath, fileContent, 'utf-8');
}

// Watch for changes in loadbalancers.txt
function watchLoadBalancersFile(filename) {
  const filePath = path.resolve(__dirname, filename);
  fs.watch(filePath, (eventType) => {
    if (eventType === 'change') {
      console.log('loadbalancers.txt file has changed, reloading load balancers...');
      loadBalancers = loadLoadBalancersFromFile(filename);
    }
  });
}

// Start watching the loadbalancers.txt file
watchLoadBalancersFile('loadbalancers.txt');

// Initialize Express app
const app = express();
app.use(bodyParser.json());

// Serve static files from the 'public' directory
app.use(express.static(path.join(__dirname, 'public')));

// Serve web.html at the root path
app.get('/', (req, res) => {
  res.sendFile(path.join(__dirname, 'public', 'web.html'));
});

// Serve the list of load balancers as a JavaScript file
app.get('/servers.js', (req, res) => {
  const serverList = loadBalancers.map(lb => `'https://${lb.host}:${lb.port}'`);
  const jsContent = `
    const servers = [
      ${serverList.join(',\n')}
    ];
  `;
  res.setHeader('Content-Type', 'application/javascript');
  res.send(jsContent);
});

// Authentication middleware
function authenticate(req, res, next) {
  const token = req.headers['authorization'];
  if (token === `${AUTH_TOKEN}`) {
    next();
  } else {
    res.status(401).json({ error: 'Unauthorized' });
  }
}

// Store server instances
const serverInstances = [];

// Listen on multiple ports (36525-36535)
for (let port = 36525; port <= 36535; port++) {
  const server = https.createServer(options, app).listen(port, () => {
    console.log(`Node.js Load Balancer running on https://localhost:${port}`);
  });
  serverInstances.push(server);
}

// API endpoint to upload new certificates
app.post('/update-certificates', authenticate, (req, res) => {
  const { key, cert } = req.body;

  if (!key || !cert) {
    return res.status(400).send('Both key and cert are required.');
  }

  // Save new certificates
  try {
    fs.writeFileSync(path.resolve(__dirname, 'server.key'), key, 'utf8');
    fs.writeFileSync(path.resolve(__dirname, 'server.cert'), cert, 'utf8');

    // Reload SSL options
    options = loadSSLOptions();

    // Update SSL context for all server instances
    serverInstances.forEach(server => {
      server.setSecureContext(options);
    });

    console.log('SSL certificates updated and reloaded successfully.');
    res.send('SSL certificates updated and reloaded successfully.');
  } catch (error) {
    console.error('Failed to update and reload SSL certificates:', error);
    res.status(500).send('Failed to update and reload SSL certificates.');
  }
});

// API to bulk add servers
app.post('/api/servers/add', authenticate, (req, res) => {
  const newServers = req.body.servers; // Expecting an array of { host, port }

  if (!Array.isArray(newServers)) {
    return res.status(400).json({ error: 'Invalid input format. Expected an array of servers.' });
  }

  for (const server of newServers) {
    if (server && server.host && server.port) {
      servers.push({ host: server.host, port: parseInt(server.port) });
    } else {
      return res.status(400).json({ error: 'Invalid server data. Each server must have a host and port.' });
    }
  }

  saveServersToFile('servers.txt', servers);
  res.status(200).json({ message: 'Servers added successfully' });
});

// API to bulk remove servers
app.post('/api/servers/remove', authenticate, (req, res) => {
  const serversToRemove = req.body.servers; // Expecting an array of { host, port }

  if (!Array.isArray(serversToRemove)) {
    return res.status(400).json({ error: 'Invalid input format. Expected an array of servers.' });
  }

  servers = servers.filter(server => 
    !serversToRemove.some(remServer => remServer.host === server.host && parseInt(remServer.port) === server.port)
  );

  saveServersToFile('servers.txt', servers);
  res.status(200).json({ message: 'Servers removed successfully' });
});

// API to bulk add load balancers
app.post('/api/loadbalancers/add', authenticate, (req, res) => {
  const newLoadBalancers = req.body.loadbalancers; // Expecting an array of { host, port }

  if (!Array.isArray(newLoadBalancers)) {
    return res.status(400).json({ error: 'Invalid input format. Expected an array of load balancers.' });
  }

  for (const lb of newLoadBalancers) {
    if (lb && lb.host && lb.port) {
      loadBalancers.push({ host: lb.host, port: parseInt(lb.port) });
    } else {
      return res.status(400).json({ error: 'Invalid load balancer data. Each load balancer must have a host and port.' });
    }
  }

  saveLoadBalancersToFile('loadbalancers.txt', loadBalancers);
  res.status(200).json({ message: 'Load balancers added successfully' });
});

// API to bulk remove load balancers
app.post('/api/loadbalancers/remove', authenticate, (req, res) => {
  const loadBalancersToRemove = req.body.loadbalancers; // Expecting an array of { host, port }

  if (!Array.isArray(loadBalancersToRemove)) {
    return res.status(400).json({ error: 'Invalid input format. Expected an array of load balancers.' });
  }

  loadBalancers = loadBalancers.filter(lb => 
    !loadBalancersToRemove.some(remLb => remLb.host === lb.host && parseInt(remLb.port) === lb.port)
  );

  saveLoadBalancersToFile('loadbalancers.txt', loadBalancers);
  res.status(200).json({ message: 'Load balancers removed successfully' });
});

// API to view all servers
app.get('/api/servers', authenticate, (req, res) => {
  res.status(200).json(servers);
});

// API to view all load balancers
app.get('/api/loadbalancers', authenticate, (req, res) => {
  res.status(200).json(loadBalancers);
});

// Sync Load Balancers with Cleanup
app.post('/api/loadbalancers/sync', authenticate, async (req, res) => {
  try {
    for (const lb of loadBalancers) {
      const targetUrlAdd = `https://${lb.host}:${lb.port}/api/loadbalancers/add`;
      const targetUrlRemove = `https://${lb.host}:${lb.port}/api/loadbalancers/remove`;

      // Fetch the current list of load balancers from each load balancer
      const currentLoadBalancers = await fetchCurrentList(`https://${lb.host}:${lb.port}/api/loadbalancers`);

      // Ensure the currentLoadBalancers is an array and has the correct structure
      if (!Array.isArray(currentLoadBalancers)) {
        console.error(`Expected an array, but got: ${typeof currentLoadBalancers}`);
        continue;
      }

      // Determine load balancers to be added (only those not already in the list)
      const loadBalancersToAdd = loadBalancers.filter(
        newLB => !currentLoadBalancers.some(currentLB => currentLB.host === newLB.host && currentLB.port === newLB.port)
      );

      // Determine load balancers to be removed (those in currentLoadBalancers but not in loadBalancers)
      const loadBalancersToRemove = currentLoadBalancers.filter(
        currentLB => !loadBalancers.some(newLB => newLB.host === currentLB.host && newLB.port === currentLB.port)
      );

      // Remove old load balancers
      if (loadBalancersToRemove.length > 0) {
        await sendSyncRequest(targetUrlRemove, { loadbalancers: loadBalancersToRemove });
      }

      // Add new load balancers
      if (loadBalancersToAdd.length > 0) {
        await sendSyncRequest(targetUrlAdd, { loadbalancers: loadBalancersToAdd });
      }
    }
    res.status(200).json({ message: 'Load balancers synchronized and cleaned up successfully.' });
  } catch (error) {
    console.error('Error syncing load balancers:', error);
    res.status(500).json({ error: 'Failed to sync load balancers.' });
  }
});

// Sync Servers with Cleanup
app.post('/api/servers/sync', authenticate, async (req, res) => {
  try {
    for (const lb of loadBalancers) {
      const targetUrlAdd = `https://${lb.host}:${lb.port}/api/servers/add`;
      const targetUrlRemove = `https://${lb.host}:${lb.port}/api/servers/remove`;

      // Fetch the current list of servers from each load balancer
      const currentServers = await fetchCurrentList(`https://${lb.host}:${lb.port}/api/servers`);

      // Ensure the currentServers is an array and has the correct structure
      if (!Array.isArray(currentServers)) {
        console.error(`Expected an array, but got: ${typeof currentServers}`);
        continue;
      }

      // Determine servers to be added (only those not already in the list)
      const serversToAdd = servers.filter(
        newServer => !currentServers.some(currentServer => currentServer.host === newServer.host && currentServer.port === newServer.port)
      );

      // Determine servers to be removed (those in currentServers but not in servers)
      const serversToRemove = currentServers.filter(
        currentServer => !servers.some(newServer => newServer.host === currentServer.host && newServer.port === currentServer.port)
      );

      // Remove old servers
      if (serversToRemove.length > 0) {
        await sendSyncRequest(targetUrlRemove, { servers: serversToRemove });
      }

      // Add new servers
      if (serversToAdd.length > 0) {
        await sendSyncRequest(targetUrlAdd, { servers: serversToAdd });
      }
    }
    res.status(200).json({ message: 'Servers synchronized and cleaned up successfully.' });
  } catch (error) {
    console.error('Error syncing servers:', error);
    res.status(500).json({ error: 'Failed to sync servers.' });
  }
});

// Helper function to fetch the current list of servers/load balancers from a given URL
async function fetchCurrentList(url) {
  const https = require('https');

  return new Promise((resolve, reject) => {
    const requestOptions = {
      method: 'GET',
      headers: {
        'Authorization': AUTH_TOKEN // Use the same authentication token
      }
    };

    https.get(url, requestOptions, (res) => {
      let data = '';

      res.on('data', (chunk) => {
        data += chunk;
      });

      res.on('end', () => {
        if (res.statusCode >= 200 && res.statusCode < 300) {
          try {
            resolve(JSON.parse(data));
          } catch (error) {
            reject(new Error(`Failed to parse JSON: ${error.message}`));
          }
        } else {
          reject(new Error(`Request failed with status code ${res.statusCode}: ${data}`));
        }
      });
    }).on('error', (error) => {
      reject(error);
    });
  });
}

// Helper function to send synchronization requests
async function sendSyncRequest(url, data) {
  const https = require('https');

  return new Promise((resolve, reject) => {
    const requestData = JSON.stringify(data);

    const urlObject = new URL(url); // Parse the URL

    const requestOptions = {
      method: 'POST',
      hostname: urlObject.hostname, // Extract hostname from URL
      port: urlObject.port, // Extract port from URL
      path: urlObject.pathname, // Extract path from URL
      headers: {
        'Content-Type': 'application/json',
        'Content-Length': requestData.length,
        'Authorization': AUTH_TOKEN,
      },
      rejectUnauthorized: false,  // Bypass SSL certificate verification
    };

    const req = https.request(requestOptions, (res) => {
      let responseData = '';

      res.on('data', (chunk) => {
        responseData += chunk;
      });

      res.on('end', () => {
        if (res.statusCode >= 200 && res.statusCode < 300) {
          resolve(JSON.parse(responseData));
        } else {
          reject(new Error(`Request failed with status code ${res.statusCode}: ${responseData}`));
        }
      });
    });

    req.on('error', (error) => {
      reject(error);
    });

    req.write(requestData);
    req.end();
  });
}

// Handle proxy requests
app.use((req, res) => {
  // Set CORS headers
  res.setHeader('Access-Control-Allow-Origin', '*');
  res.setHeader('Access-Control-Allow-Methods', 'GET,HEAD,OPTIONS');
  res.setHeader('Access-Control-Allow-Headers', 'Origin, X-Requested-With, Content-Type, Accept, Range');

  try {
    // Select a backend server randomly
    const target = `http://${getRandomServer().host}:${getRandomServer().port}`;

    // Proxy the request to the selected backend server
    proxy.web(req, res, { target }, (error) => {
      console.error(`Proxy error: ${error}`);
      res.writeHead(502, { 'Content-Type': 'text/plain' });
      res.end('Bad Gateway');
    });
  } catch (error) {
    console.error('No servers available to proxy the request:', error);
    res.writeHead(502, { 'Content-Type': 'text/plain' });
    res.end('No servers available');
  }
});

/loadbalancers.txt

Code: Select all

example.balancer:36525
example.balancer:36527

/package.json

Code: Select all

{
  "name": "node-load-balancer",
  "version": "1.0.0",
  "description": "A Node.js-based load balancer with API support for updating active IPs.",
  "main": "index.js",
  "scripts": {
    "start": "node index.js"
  },
  "author": "VodMeister",
  "license": "MIT",
  "dependencies": {
    "http-proxy": "^1.18.1",
    "https": "^1.0.0",
    "express": "^4.18.2",
    "body-parser": "^1.20.2",
    "os": "^0.1.1"
  }
}

/server.cert
(random cert, should be replaced with the one u generate for ur domain that is pointing the A records)

Code: Select all

-----BEGIN CERTIFICATE-----
MIIDwTCCAqmgAwIBAgIJALZW4cduwiJ0MA0GCSqGSIb3DQEBBQUAMEkxCzAJBgNV
BAYTAkNBMRAwDgYDVQQIEwdPbnRhcmlvMRAwDgYDVQQHEwdUb3JvbnRvMRYwFAYD
VQQKEw1pTlRFUkZBQ0VXQVJFMB4XDTA4MTIxNjE1MjMzNFoXDTExMTIxNjE1MjMz
NFowSTELMAkGA1UEBhMCQ0ExEDAOBgNVBAgTB09udGFyaW8xEDAOBgNVBAcTB1Rv
cm9udG8xFjAUBgNVBAoTDWlOVEVSRkFDRVdBUkUwggEiMA0GCSqGSIb3DQEBAQUA
A4IBDwAwggEKAoIBAQDqnDHLq4nfdFuSuxHSo0vM4PbyCYCLBun1uV4fdQdvnQSl
raLbQQMSx44B+wJCgYP4f2FLe9OhllNZ55i4TWpfKu9rmL5qq5uNBvEEUCuNJKEE
QGtesWaj6cVmxFEjOZda3Cj5Sy7988Sv8pDjlDu4vqBvSrzssSIEXy6gjq+pK7uB
eSklYWglYnTjwdS2F/IwuCatqsK7Afp+l07R6ofEnj9y+bS0EEz7+Lb99zmu4e+w
ANImSV+W8rfJspXGMd2CGp6MEg+Te1NCbl+Du5G7tIAx0qp7xV4ik/+6NdrAXbyD
KeG1l8EO5QAIgBSDeCOPbjccCYEM17NF8eYWliVvAgMBAAGjgaswgagwHQYDVR0O
BBYEFGD7SIq57+klnVi8TF7ypr9PpDC/MHkGA1UdIwRyMHCAFGD7SIq57+klnVi8
TF7ypr9PpDC/oU2kSzBJMQswCQYDVQQGEwJDQTEQMA4GA1UECBMHT250YXJpbzEQ
MA4GA1UEBxMHVG9yb250bzEWMBQGA1UEChMNaU5URVJGQUNFV0FSRYIJALZW4cdu
wiJ0MAwGA1UdEwQFMAMBAf8wDQYJKoZIhvcNAQEFBQADggEBAOg+0M8BTZNT6CNT
7GFVxtDYGGAFiQEpmbcJlLdFsH5snxbH8OvVp5RkpaQlFesyX2LldnJbSEyKH5Tz
YxCqAUIw1awBvsl7QNW8/O3Cv9iDtCL02aBDB4VH4bUF6HD3TMjntYC7Hax8JiL0
RW8RXBezgy260A/mP/EupdWs2n+HKe5z3BSMdVXJDTc8m9R9D3bvtP0IzvQQIPpr
uHBP3u9tAigAE/BofSWi68uCLyZQnIQnHtak2seFf8N1r3cHrPu7GBgodBDdlXNw
7s2wsTAeyDGcmhbJF/nzGqdKhnvOFrsBdiWPKCcECD2oGj/ISNoXimqdmhQfjvn2
mzG7Jpk=
-----END CERTIFICATE-----

server.key
(random cert, should be replaced with the one u generate for ur domain that is pointing the A records)

Code: Select all

-----BEGIN RSA PRIVATE KEY-----
MIIEpgIBAAKCAQEA6pwxy6uJ33RbkrsR0qNLzOD28gmAiwbp9bleH3UHb50Epa2i
20EDEseOAfsCQoGD+H9hS3vToZZTWeeYuE1qXyrva5i+aqubjQbxBFArjSShBEBr
XrFmo+nFZsRRIzmXWtwo+Usu/fPEr/KQ45Q7uL6gb0q87LEiBF8uoI6vqSu7gXkp
JWFoJWJ048HUthfyMLgmrarCuwH6fpdO0eqHxJ4/cvm0tBBM+/i2/fc5ruHvsADS
JklflvK3ybKVxjHdghqejBIPk3tTQm5fg7uRu7SAMdKqe8VeIpP/ujXawF28gynh
tZfBDuUACIAUg3gjj243HAmBDNezRfHmFpYlbwIDAQABAoIBAQCP4/p6gxwNi+z6
Inf866B66OMscX2AR15JEkbTHlDQOMp33vYKaWY8J15Ggq/RIGRTjbSbujeDXJKE
ipHVP83kzo2HPWhUPioqJb6+uXjsmTGUTPpNWpqsH52tuOxWoWTeGjebJmyM3uyc
STZqDilO1sPJXlpfBQjrC4GqgbjlFLugOKX4VviGvECcvsThL+7F+SKRJ6lekea2
LXK5Cj1W5w2/Kke62+rnZTUrbGGxm9Flxuy0PVSA/S0KI3fCE1bCi7NTXRstOAAz
vZCyb7Z0gI29/0c2lyDTd6J3jxGEEpDQ37FB4pbWnUkX0ZVbMy6Y2VTeBZE0R3O4
SY6/ZIv5AoGBAP26AT9/u2XwKduIb2YtG5W6cX37FmBwzI5pditi0k4ngG21f9xo
oye/c3BUoyF9TQ3jgWObZSPYocljJ8rzEkdfHJ8QhCYlVTmdInRjVRffgKOrHDjF
9FNK1ggXGLVwGmMbUUeIKXQRw4tKfPpLG0Qcgilx3y69aYHSP6qMVfcTAoGBAOy2
Vy5xzjq6U8RL1I1FncArJgB94ae0H2PkPYgmQnFCI87vuG7pv3wZvTK8P0RtFVtB
3n+CJZFVby1eTcqHPJkcmeeED323bCftMNW7tf/a7CBaH56k3dqh4h4v6+0P4pyo
sqldHb3bqTsvc5o7KWTXDH+lx9zKlsuVDHR1rVe1AoGBAMFK3b6JSbN8BfdX9j3p
6VTkx6dJDKAF7uAjWcHts/eUQlPR7Il2Ma2LPZ966xgNRBFrm1vNu3xWgdJRNrR2
/xreS4imZXZGBKoymlf+gIoCXBbTuVlK/TojDfD1334B3ChaXE5ZXfMtwUGxSorH
gwsdiM+YD4WlCOa8zIHaDXd/AoGBAJN7B9ZoEZWFgatLk6JxPVf9ii/EPlO+ZdBW
4/9v1vW5v5WuxbpU6HjpkHeL0d9QF35EC9xlugJSuHILz2vf1mGO8FTOcthg74Hw
xfxkd4BxZazCefDdx1vwgHFOai/JNedlM+tRmLYxpb66UcxGEARD+AWPxHZLwqgU
tS3aI6YBAoGBAKrnyHc3RN/kgyLvyngJUMuqDrbr2sS7TMm0fDoezunv6e7mtG6+
GExKDefmpvxmk2vsFU7feQSqiNBicDgOaiV8G1byvA3SauGmNhtRAXpgQMzAaJDI
vT86Y0QyLeoUNcW+i8FNqEBpJiWqHnCe3FI6WmF+ISDP6MNHmjLJRG84
-----END RSA PRIVATE KEY-----

/servers.txt

Code: Select all

example.streamserver:36525
example.streamserver:36527
example.streamserver:36527

/start.sh

Code: Select all

#!/bin/sh

# Run vod
/usr/bin/node /usr/local/bin/hotpocket/loadbalancer &

# Set the HotPocket binary as the entry point.
# $@ is used to pass all the commandline arguments fed to this script into hpcore.
/usr/local/bin/hotpocket/hpcore $@

RESULT:


Address to this page: https://Your load balancer load address:port

https is a requirement!

An other important detail:
You need at a minimum 2 load balancers added, it can be the same address, but there has to be two because the system is looping through the loadbalancers, and only attempts 1 time!