Skip to content
Back to Blog

Under the Hood: Script Sandboxing

By Rawframe Team
technical architecture engine

The Problem with Shared State

In many scriptable game engines, all script packages share a single environment. One global Lua state, one set of globals, one namespace. This is simple to implement, but it creates serious problems:

  • One bad package crashes everything. An infinite loop or memory leak in one script takes down your whole game along with it.
  • Name collisions are invisible. Two packages defining a global update() function silently overwrite each other.
  • Hot-reload is all-or-nothing. You can't reload one package without tearing down the entire script state.
  • Resource tracking is impossible. You can't tell which package is using how much memory or CPU time.

This matters even more in Rawframe, where your game can include script packages from the Asset Store — code you didn't write yourself.

One VM Per Package

When the Rawframe server loads a game, it creates a dedicated lua_State* for the game's own scripts and for each installed asset package. Each VM gets:

  • Its own global table — no name collisions between packages
  • Its own sandboxed environment — file system access, os.execute, and other dangerous APIs are blocked
  • Its own memory budget — a package that allocates too much memory gets terminated, not the entire server
  • Its own error boundary — a runtime error in one package is caught and logged without affecting others

The engine's VM manager orchestrates all of this. It owns a map of package IDs to their VM instances and handles the lifecycle: creation, script loading, tick dispatch, hook routing, and teardown.

Cross-VM Communication

Isolated VMs can't share Lua values directly — a table in VM-A is meaningless to VM-B. Rawframe solves this with LuaValue serialization.

When your game calls place.call("pvp_core", "getScore", playerId), the engine:

  1. Serializes the arguments into a LuaValue tree (a C++ variant type that can represent nil, bool, number, string, and tables)
  2. Looks up the target package's exported function in its VM
  3. Deserializes the arguments into that package's Lua state
  4. Calls the function
  5. Serializes the return value back
  6. Deserializes it into the caller's state

This is not free — there's a serialization cost for every cross-package call. But in practice, these calls are infrequent (game logic runs within a single package), and the safety guarantee is worth the overhead.

The serializer handles nested tables with circular reference detection. If a table refers to itself (directly or indirectly), the serializer stops with an error rather than looping forever.

Hook Dispatch and Re-Entrance

Rawframe uses a priority-based hook system. When the engine fires a hook like "Think" or "PlayerConnect", the VM manager iterates over all package VMs and calls each registered handler for that hook.

This raises a subtle problem: re-entrance. What if package A's Think handler calls place.call() into package B, which fires another hook, which calls back into package A? Without protection, you get infinite recursion.

Rawframe prevents this with a re-entrance guard. Each VM tracks whether it's currently executing. If a hook dispatch tries to enter a VM that's already running, the call is skipped (or deferred, depending on the hook type). This is a simple boolean flag per VM, but it prevents an entire class of hard-to-debug stack overflows.

Hot-Reload Without Downtime

Per-package VMs make hot-reload straightforward:

  1. Detect that a package's files have changed (file watcher or manual trigger)
  2. Call all cleanup hooks in the package's VM (OnUnload)
  3. Destroy the package's lua_State*
  4. Create a fresh lua_State*
  5. Re-register the engine API functions
  6. Load and execute the package's scripts
  7. Fire the OnLoad hook

The entire process takes milliseconds. The rest of your game keeps running uninterrupted. Players on the server don't disconnect. This is essential for development — change a script, hit reload, see the result instantly.

Memory Limits and Profiling

Each VM is created with a custom allocator that tracks total memory usage. The engine enforces per-package memory limits (configurable by the server operator). When a package exceeds its limit, the allocator returns NULL, the Lua state throws an out-of-memory error, and the package is terminated cleanly.

The ScriptProfiler can also track per-package CPU usage by hooking into Luau's interrupt system. Server operators can see exactly which package is consuming the most resources and act accordingly.

Why This Matters

Script sandboxing isn't glamorous, but it's the foundation that makes everything else possible. It means:

  • You can install Asset Store packages without auditing every line of code
  • Developers can iterate with instant hot-reload
  • One buggy script can't ruin the experience for everyone
  • The engine can enforce fair resource usage across packages

It's the kind of infrastructure that, when it works well, you never notice. And that's exactly the point.

Back to Blog