compression added

README.md

@@ -12,9 +12,9 @@ Works with any game — just fill in the configuration blocks at the top of each
 ## How it works
 
 1. You put your game's files in a folder on the server (e.g. `game_client/`).
-2. You run `generate_manifest.py` on that folder. It walks the folder, hashes every file (MD5), and writes a `manifest.json` listing `relative/path: hash` for every file.
+2. You run `generate_manifest.py` on that folder. It walks the folder, hashes every file (MD5), and writes a `manifest.json` listing metadata for every file. It also gzip-compresses every file into a `<file>.gz` copy next to the original — if the compressed copy isn't actually smaller (common for already-compressed formats like `.jpg`, `.mp3`, `.zip`), the `.gz` copy is discarded and the manifest just records the original.
 3. `server.py` serves that folder over HTTP, keyed by an access key defined in `config.json`.
-4. Players run `updater.exe`. It downloads the server's `manifest.json`, compares hashes against their local files, and downloads anything missing or different.
+4. Players run `updater.exe`. It downloads the server's `manifest.json`, compares hashes against their local files, and for anything missing or different: downloads the `.gz` copy if one exists, verifies it, decompresses it locally, and writes the result — or downloads the original directly if there's no compressed copy.
 5. Once everything matches, the "Play" button launches the game executable.
 
 ## Setup
@@ -105,6 +105,19 @@ pyinstaller --onefile --noconsole --icon=icon.ico --add-data "banner.png;." --ad
 
 Ship the resulting `updater.exe` alongside the game folder.
 
+## Settings
+
+On first run, the updater creates a `settings.ini` file next to itself with the selected client version:
+
+```ini
+[updater]
+client = Main
+```
+
+Whenever the player changes the client version dropdown, it's saved immediately so the choice persists across restarts. If `settings.ini` is missing, corrupted, or contains an unrecognized client name, the updater silently falls back to `DEFAULT_CLIENT` and continues — it never crashes over a bad settings file.
+
+This is currently the only setting. To add more later, extend `load_settings()` / `save_settings()` in `updater.py` with additional keys under the same `[updater]` section.
+
 ## Access keys
 
 Client folders are addressed by a long random hex key instead of a plain name like `main`, so the update path isn't obvious or easy to stumble on by guessing.
@@ -145,9 +158,42 @@ IGNORE_EXTENSIONS = {
 
 Add any player-specific files (settings, saves, screenshots, logs) here so the updater never overwrites or deletes them.
 
+## Compression
+
+`generate_manifest.py` gzip-compresses every distributable file into a `<file>.gz` copy alongside the original, unconditionally — no file-type filtering. If the compressed copy ends up the same size or larger than the original (common for already-compressed formats), it's discarded automatically and that file is just served uncompressed; there's nothing to configure per file type.
+
+The manifest records both the original file's hash/size and, when a compressed copy exists, its hash/size too:
+
+```json
+{
+    "script.txt": {
+        "hash": "d3650c70e7d942736c3ca77daf381262",
+        "size": 60000,
+        "compressed": true,
+        "compressed_size": 177,
+        "compressed_hash": "852ab2df07a4c65cddd6ed3ec0282301"
+    },
+    "texture.bin": {
+        "hash": "1b6d041423b9863dee54e87fed1f0cdd",
+        "size": 20000,
+        "compressed": false
+    }
+}
+```
+
+`updater.py` uses this to download the smaller `.gz` copy when one is available, verifies its hash, decompresses it in memory, and verifies the decompressed result against the original hash before writing it to disk. If anything about the compressed copy fails (missing, corrupted, hash mismatch), it automatically falls back to downloading the uncompressed original — a bad `.gz` on the server never blocks an update.
+
+Two settings control this, and must be kept in sync between `generate_manifest.py` and `updater.py`:
+
+- `COMPRESSED_SUFFIX` (default `.gz`) — the suffix used for compressed copies.
+- `COMPRESSION_LEVEL` (default `6`, in `generate_manifest.py` only) — gzip level 1 (fastest, larger) to 9 (slowest, smallest).
+
+Re-run `generate_manifest.py` after any change to the client files — it regenerates both the manifest and the `.gz` copies. Compressed output is generated with a fixed mtime so re-running it without any real file changes produces byte-identical `.gz` files and stable hashes.
+
 ## Notes / limitations
 
 - File integrity is checked with MD5 — fine for detecting changed files, not intended as a cryptographic guarantee.
+- Compression roughly doubles the disk space needed on the server for each client version (original + `.gz` copies), in exchange for faster downloads for compressible files.
 - The server has no built-in rate limiting or HTTPS; put it behind a reverse proxy (nginx, Caddy) if you need either.
 - The server never serves directory listings — a bare `/`, an unknown key, or a path that tries to escape the client folder (`..`) all get a 404, including the server's own script/config files.
 - `updater.py` uses Tkinter for the UI, which ships with most standard Python installs on Windows.

generate_manifest.py

@@ -1,5 +1,7 @@
 import os
 import json
+import gzip
+import shutil
 import hashlib
 import argparse
 
@@ -9,6 +11,13 @@ import argparse
 
 MANIFEST_FILENAME = "manifest.json"
 
+# Suffix used for the precompressed copy of each file, stored alongside the
+# original (e.g. "texture.dds" -> "texture.dds.gz").
+COMPRESSED_SUFFIX = ".gz"
+
+# gzip compression level, 1 (fastest, larger) to 9 (slowest, smaller).
+COMPRESSION_LEVEL = 6
+
 # Files the updater must never touch (launcher itself, local settings, etc.)
 IGNORE_FILES = {
     MANIFEST_FILENAME,
@@ -34,8 +43,13 @@ IGNORE_EXTENSIONS = {
 # ============================================================
 
 
+def get_md5_bytes(data):
+    """Computes the MD5 hash of an in-memory bytes object."""
+    return hashlib.md5(data).hexdigest()
+
+
 def get_md5(filepath):
-    """Computes the MD5 hash of a file."""
+    """Computes the MD5 hash of a file on disk."""
     hash_md5 = hashlib.md5()
     try:
         with open(filepath, "rb") as f:
@@ -46,6 +60,41 @@ def get_md5(filepath):
         return None
 
 
+def compress_file(filepath, compressed_path):
+    """
+    Compresses filepath into compressed_path with gzip. Every file is
+    compressed unconditionally - if the result isn't actually smaller
+    (common for already-compressed formats like .jpg/.mp3/.zip), the
+    compressed copy is discarded and the manifest entry omits "compressed",
+    so the server/updater fall back to serving the original file untouched.
+
+    Uses mtime=0 and a GzipFile with no filename so the compressed output
+    is byte-identical across runs when the input hasn't changed - otherwise
+    gzip embeds the source mtime in its header and compressed_hash would
+    change on every manifest regeneration even with no real content change.
+
+    Returns (compressed_size, compressed_hash) or (None, None) if the
+    compressed copy wasn't kept.
+    """
+    original_size = os.path.getsize(filepath)
+
+    with open(filepath, "rb") as f_in:
+        data = f_in.read()
+
+    with open(compressed_path, "wb") as raw_out:
+        with gzip.GzipFile(fileobj=raw_out, mode="wb", compresslevel=COMPRESSION_LEVEL, mtime=0) as f_out:
+            f_out.write(data)
+
+    compressed_size = os.path.getsize(compressed_path)
+
+    if compressed_size >= original_size:
+        os.remove(compressed_path)
+        return None, None
+
+    compressed_hash = get_md5(compressed_path)
+    return compressed_size, compressed_hash
+
+
 def generate(client_dir):
     if not os.path.exists(client_dir) or not os.path.isdir(client_dir):
         print(f"Error: directory '{client_dir}' not found!")
@@ -56,6 +105,9 @@ def generate(client_dir):
 
     print(f"Scanning client folder: {client_dir}")
 
+    total_original = 0
+    total_saved = 0
+
     for root, dirs, files in os.walk(client_dir):
         # Filter out ignored folders (case-insensitive)
         dirs[:] = [d for d in dirs if d.lower() not in IGNORE_FOLDERS]
@@ -65,24 +117,48 @@ def generate(client_dir):
                 continue
             if any(file.lower().endswith(ext) for ext in IGNORE_EXTENSIONS):
                 continue
+            # Skip stray leftover compressed copies from a previous run so
+            # we never treat "texture.dds.gz" as a distributable file itself.
+            if file.endswith(COMPRESSED_SUFFIX):
+                continue
 
             filepath = os.path.join(root, file)
 
             # Path relative to the client folder (this is what the updater expects)
             rel_path = os.path.relpath(filepath, client_dir)
-
-            # Convert Windows-style separators to web-style
             rel_path_web = rel_path.replace("\\", "/")
 
             file_hash = get_md5(filepath)
-            if file_hash:
+            if not file_hash:
-                manifest[rel_path_web] = file_hash
+                continue
+
+            entry = {
+                "hash": file_hash,
+                "size": os.path.getsize(filepath),
+            }
+
+            compressed_path = filepath + COMPRESSED_SUFFIX
+            compressed_size, compressed_hash = compress_file(filepath, compressed_path)
+
+            if compressed_hash:
+                entry["compressed"] = True
+                entry["compressed_size"] = compressed_size
+                entry["compressed_hash"] = compressed_hash
+                total_saved += entry["size"] - compressed_size
+            else:
+                entry["compressed"] = False
+
+            total_original += entry["size"]
+            manifest[rel_path_web] = entry
 
     try:
         with open(manifest_path, "w", encoding="utf-8") as f:
             json.dump(manifest, f, indent=4)
         print(f"Done! Manifest saved to: {manifest_path}")
         print(f"Total files in manifest: {len(manifest)}")
+        if total_original:
+            pct = (total_saved / total_original) * 100
+            print(f"Compression saved {total_saved:,} bytes of {total_original:,} ({pct:.1f}%)")
     except IOError as e:
         print(f"Error saving manifest: {e}")
 

server.py

@@ -27,9 +27,16 @@ PORT = CONFIG.get("port", 3232)
 CLIENTS = CONFIG.get("clients", {})
 
 # Absolute, resolved base directories for each client key, computed once at
-# startup. Every request is checked against these to make sure the final
+# startup. Relative folder paths in config.json are resolved relative to
-# path can never land outside the intended client folder.
+# this script's own location, not the process's current working directory,
-CLIENT_BASE_DIRS = {key: os.path.realpath(folder) for key, folder in CLIENTS.items()}
+# so `python server.py` works the same regardless of where it's launched
+# from. Every request is checked against these to make sure the final path
+# can never land outside the intended client folder.
+SCRIPT_DIR = os.path.dirname(os.path.abspath(__file__))
+CLIENT_BASE_DIRS = {
+    key: os.path.realpath(os.path.join(SCRIPT_DIR, folder))
+    for key, folder in CLIENTS.items()
+}
 
 # A path that is guaranteed not to exist, used to force a 404 through the
 # normal http.server flow instead of ever falling back to serving files

updater.py

@@ -1,11 +1,13 @@
 import os
 import sys
 import json
+import gzip
 import hashlib
 import threading
 import urllib.request
 import urllib.parse
 import subprocess
+import configparser
 import tkinter as tk
 from tkinter import ttk, messagebox
 
@@ -22,7 +24,7 @@ except ImportError:
 GAME_NAME = "My Game"                  # Shown in the window title
 UPDATER_NAME = "Launcher"               # Updater, Launcher, etc.
 WINDOW_WIDTH = 450
-WINDOW_HEIGHT = 300
+WINDOW_HEIGHT = 280
 
 # Banner image is scaled to exactly this size (pixels) regardless of the
 # source file's actual dimensions, so the layout never shifts if someone
@@ -61,6 +63,13 @@ ICON_RELATIVE_PATH = "icon.ico"
 
 MANIFEST_FILENAME = "manifest.json"
 
+# Suffix used for precompressed files on the server (must match
+# COMPRESSED_SUFFIX in generate_manifest.py).
+COMPRESSED_SUFFIX = ".gz"
+
+# Local settings file (created next to the updater on first run).
+SETTINGS_FILENAME = "updater.ini"
+
 # ============================================================
 # END CONFIGURATION
 # ============================================================
@@ -82,6 +91,42 @@ BASE_PATH = get_base_path()
 BANNER_FILE = get_resource_path(BANNER_RELATIVE_PATH)
 ICON_FILE = get_resource_path(ICON_RELATIVE_PATH)
 GAME_EXE = os.path.join(BASE_PATH, GAME_EXE_RELATIVE_PATH)
+SETTINGS_FILE = os.path.join(BASE_PATH, SETTINGS_FILENAME)
+
+
+def load_settings():
+    """
+    Reads settings.ini next to the updater. If it doesn't exist yet (first
+    run) or is missing/invalid, creates it with default values and returns
+    those defaults. Unknown/corrupt values fall back to the default rather
+    than crashing the app.
+    """
+    config = configparser.ConfigParser()
+    settings = {"client": DEFAULT_CLIENT}
+
+    if os.path.exists(SETTINGS_FILE):
+        try:
+            config.read(SETTINGS_FILE, encoding="utf-8")
+            client = config.get("updater", "client", fallback=DEFAULT_CLIENT)
+            if client in CLIENT_KEYS:
+                settings["client"] = client
+        except (configparser.Error, OSError):
+            pass  # Fall back to defaults below and rewrite the file.
+    else:
+        save_settings(settings)  # First run: create the file with defaults.
+
+    return settings
+
+
+def save_settings(settings):
+    """Writes updater.ini next to the updater. Failures are non-fatal."""
+    config = configparser.ConfigParser()
+    config["updater"] = {"client": settings.get("client", DEFAULT_CLIENT)}
+    try:
+        with open(SETTINGS_FILE, "w", encoding="utf-8") as f:
+            config.write(f)
+    except OSError:
+        pass  # Not critical - the app just won't remember the choice.
 
 
 def build_server_base_url():
@@ -92,6 +137,8 @@ def build_server_base_url():
 class UpdaterApp:
     def __init__(self, root):
         self.root = root
+        self.settings = load_settings()
+
         self.root.title(f"{GAME_NAME} {UPDATER_NAME}")
         self.root.geometry(f"{WINDOW_WIDTH}x{WINDOW_HEIGHT}")
         self.root.resizable(False, False)
@@ -116,12 +163,13 @@ class UpdaterApp:
 
         tk.Label(mode_frame, text="Client version:").pack(side="left")
 
-        self.mode_var = tk.StringVar(value=DEFAULT_CLIENT)
+        self.mode_var = tk.StringVar(value=self.settings["client"])
         self.mode_combo = ttk.Combobox(
             mode_frame, textvariable=self.mode_var,
             values=list(CLIENT_KEYS.keys()), state="readonly", width=15
         )
         self.mode_combo.pack(side="left", padx=5)
+        self.mode_combo.bind("<<ComboboxSelected>>", self.on_client_changed)
 
         # Status text
         self.status_var = tk.StringVar(value="Select a version and click Check for Updates...")
@@ -181,6 +229,10 @@ class UpdaterApp:
         except Exception:
             return None
 
+    def on_client_changed(self, event=None):
+        self.settings["client"] = self.mode_var.get()
+        save_settings(self.settings)
+
     def update_ui(self, text=None, prog_val=None, prog_max=None):
         if text: self.root.after(0, lambda: self.status_var.set(text))
         if prog_max: self.root.after(0, lambda: self.progress.config(maximum=prog_max))
@@ -198,6 +250,9 @@ class UpdaterApp:
         except IOError:
             return None
 
+    def get_md5_bytes(self, data):
+        return hashlib.md5(data).hexdigest()
+
     def start_update(self):
         self.btn_update.config(state="disabled")
         self.btn_play.config(state="disabled")
@@ -222,13 +277,13 @@ class UpdaterApp:
             total_files = len(server_manifest)
 
             self.update_ui(prog_max=total_files, prog_val=0)
-            for i, (rel_path, server_hash) in enumerate(server_manifest.items()):
+            for i, (rel_path, entry) in enumerate(server_manifest.items()):
                 self.update_ui(text=f"Checking: {rel_path}", prog_val=i)
 
                 local_path = os.path.join(BASE_PATH, os.path.normpath(rel_path))
                 local_hash = self.get_md5(local_path)
 
-                if local_hash != server_hash:
+                if local_hash != entry["hash"]:
                     files_to_download.append(rel_path)
 
             if files_to_download:
@@ -238,13 +293,11 @@ class UpdaterApp:
                 for i, rel_path in enumerate(files_to_download):
                     self.update_ui(text=f"Downloading ({i + 1}/{total_dl}): {rel_path}", prog_val=i)
 
-                    safe_url_path = urllib.parse.quote(rel_path, safe='/')
+                    entry = server_manifest[rel_path]
-                    file_url = current_server_url + safe_url_path
-
                     local_path = os.path.join(BASE_PATH, os.path.normpath(rel_path))
                     os.makedirs(os.path.dirname(local_path), exist_ok=True)
 
-                    urllib.request.urlretrieve(file_url, local_path)
+                    self._download_file(current_server_url, rel_path, entry, local_path)
 
             self.update_ui(text=f"Client '{display_name}' is up to date! Ready to play.", prog_val=self.progress['maximum'])
             self.root.after(0, lambda: self.btn_play.config(state="normal"))
@@ -256,6 +309,39 @@ class UpdaterApp:
             self.root.after(0, lambda: self.btn_update.config(state="normal"))
             self.root.after(0, lambda: self.mode_combo.config(state="readonly"))
 
+    def _download_file(self, base_url, rel_path, entry, local_path):
+        """
+        Downloads one file. If the manifest marks it as compressed, fetches
+        the .gz copy, verifies its hash, and decompresses it into local_path.
+        Otherwise downloads the original file directly. Falls back to the
+        uncompressed original if the compressed copy fails verification or
+        download, so a bad/missing .gz on the server doesn't block the update.
+        """
+        if entry.get("compressed"):
+            gz_url = base_url + urllib.parse.quote(rel_path + COMPRESSED_SUFFIX, safe='/')
+            try:
+                with urllib.request.urlopen(gz_url, timeout=30) as resp:
+                    compressed_data = resp.read()
+
+                if self.get_md5_bytes(compressed_data) != entry.get("compressed_hash"):
+                    raise ValueError("compressed file hash mismatch")
+
+                data = gzip.decompress(compressed_data)
+
+                if self.get_md5_bytes(data) != entry["hash"]:
+                    raise ValueError("decompressed file hash mismatch")
+
+                with open(local_path, "wb") as f:
+                    f.write(data)
+                return
+            except Exception:
+                # Fall through to downloading the uncompressed original below.
+                pass
+
+        safe_url_path = urllib.parse.quote(rel_path, safe='/')
+        file_url = base_url + safe_url_path
+        urllib.request.urlretrieve(file_url, local_path)
+
     def play_game(self):
         if os.path.exists(GAME_EXE):
             game_dir = os.path.dirname(GAME_EXE)