Why Pigeon
Almost every Roblox game networks the same way: one RemoteEvent per feature. That shape causes five specific problems. This page names them, shows what Pigeon does about each, says why the design is the way it is rather than some other way, and ends with the cases where you should not use it.
What everyone does today
You add a feature. You make a RemoteEvent for it, give it a name, parent it into
ReplicatedStorage, and on the client you WaitForChild it
before you connect a handler.
-- Server
local shopEvent = Instance.new("RemoteEvent")
shopEvent.Name = "ShopEvent"
shopEvent.Parent = ReplicatedStorage
shopEvent.OnServerEvent:Connect(function(player, action, itemId)
-- one handler, one big if-else, one place to forget a check
end)
-- Client
local shopEvent = ReplicatedStorage:WaitForChild("ShopEvent")
shopEvent.OnClientEvent:Connect(onShopMessage)
Nothing here is wrong. It works. It just does not hold up as the game grows, for five reasons.
The five problems
1. The Instance count tracks the feature count
Forty features means forty RemoteEvents. Every one is an Instance you create, name, parent, find and wait for. Half the boilerplate in a networking module is just moving Instances around.
2. Startup is a race you lose
The server fires. The client has not connected its handler yet. The message is gone, and nothing tells you it happened. The usual fix is to make the client ask for a fresh copy of everything once it loads, which is a second code path doing the same job as the first.
3. Any client can fire any remote
A RemoteEvent in ReplicatedStorage is visible to every client, and
any of them can fire it with any arguments. Access checks live inside each
handler, so every new handler is a new chance to forget one. There is no way to
say "this channel is for moderators" and have it be true.
4. Keeping state in sync is hand written
You have a table on the server. You want the client to see it. So you write the diffing, the sending, the applying, and a separate "here is the whole thing" path for when a player joins. Then you write it again for the next table.
5. Turning a system off is manual
Shutting a feature down means remembering every connection it made. Miss one and it keeps firing into dead code. There is no handle that means "this whole system".
How Pigeon answers them
1. One pool of remotes, shared
Pigeon does not make a RemoteEvent per channel. It keeps a small pool and hashes each channel onto one of them. The pool is one RemoteEvent per 16 transformers, never fewer than 1 and never more than 32. Each slot has a reliable RemoteEvent and an UnreliableRemoteEvent twin, so the ceiling is 64 remotes no matter how many channels you have.
You never touch any of it. You name a channel and send on it.
local shop = Pigeon.new("Shop")
shop:Broadcast("StockChanged", getStock())
See The Ref Pool.
2. The channel opens when you say so
The client calls Init() on a channel once its handlers are in place.
Until then the server holds everything addressed to that client on that channel,
up to 64 packets, for 30 seconds. When Init lands, the backlog
arrives in the order it was held. Unreliable sends are the one exception. They
are dropped rather than held, because a packet allowed to go missing in transit
has no business turning up late.
A message sent before the client is ready
Readiness is per channel. Bringing "Shop" up does not release "Combat", so a channel's backlog never lands in a handler that does not exist yet.
3. One guard per channel
Install a handshake guard on the server and the channel is closed by default. A client has to pass the guard before that channel will carry anything for them. The check sits in one place, not in every handler.
local mods = Pigeon.new("Moderation")
mods:UseHandshake(function(player, token)
return isModerator(player, token)
end)
-- Only clients the guard approved receive this.
mods:Broadcast("Alert", "someone is spawning parts")
See Handshakes.
4. Tables that replicate themselves
A staged table is a plain table. Write to it on the server and every client holding it sees the write. There is no diffing to write and no snapshot path to maintain, because requesting the table gives you the current contents first and the updates after.
local match = Pigeon.StagedTable({ round = 1, scores = {} })
local carrier = Pigeon.new("Match")
carrier:CaptureTable("live", match)
local data = match:GetTable()
data.round = 2 -- every client holding it sees this
local carrier = Pigeon.new("Match")
carrier:Init()
local match = carrier:RequestTable("live")
print(match:GetTable().round)
See Staged Tables.
5. One call turns a group off
Everything a carrier subscribes to is tracked. carrier:Destroy()
disconnects all of it. Put several carriers on one transformer and
transformer:Destroy() drops the whole group at once.
local combat = Pigeon.Transformer("Combat")
local damage = Pigeon.new("Damage", { Transformer = combat })
local status = Pigeon.new("Status", { Transformer = combat })
local effects = Pigeon.new("Effects", { Transformer = combat })
-- One call drops all three channels and every listener on them.
combat:Destroy()
See Cleanup.
Side by side
| Concern | Plain RemoteEvents | Pigeon |
|---|---|---|
| Instance count | One per feature, and it only ever grows. | A shared pool. One remote per 16 transformers, capped at 32 pairs. |
| Startup race | Fire before the client connects and the message is gone. | Held per channel until Init. Up to 64 packets, for 30 seconds. |
| Access control | Any client can fire any remote. Checks live in each handler. | One handshake guard per channel, closing it in both directions. |
| State replication | You write the diffing, the sending and the join path. | Write to a staged table. Writes batch and replicate on their own. |
| Teardown | Disconnect every connection by hand. | carrier:Destroy(), or one transformer:Destroy() for a group. |
| Request and response | A separate RemoteFunction, and an invoke that can yield forever. | Call and CallTo on the same channel, 10 second default timeout. |
| Rooms | Keep your own player lists and prune them on leave. | CreateRoom, JoinRoom, SendToRoom. Leavers are pruned for you. |
What is actually new here
Plenty of libraries wrap RemoteEvents in nicer functions. Five things in Pigeon are not wrapping. Each one had an obvious simpler option that was rejected, so each section below says what that option was and why it does not work.
A shared hashed remote pool
Instance count stops tracking feature count. A stable hash of the transformer uuid picks the remote, and a carrier takes that uuid from its channel name. Two hundred channels ride on thirteen remotes and their thirteen unreliable twins.
| At 200 channels | One remote per feature | Pigeon |
|---|---|---|
Instances in ReplicatedStorage | 200 | 26: thirteen buckets, each with an unreliable twin |
| Instances you name and parent | 200 | None. You name channels, not Instances. |
| What the client waits on | 200, one per feature | One folder. Refs are bound as they replicate. |
| Channels sharing a remote | 1 | About 15 |
Thirteen is not a round number chosen for the example. The server counts its transformers, divides by 16, rounds up and clamps the result between 1 and 32. 200 over 16 is 12.5, so 13.
Why hash rather than hand out slots in order
The simple version is a counter: the first transformer takes ref 1, the second takes ref 2. It fails because the two machines would have to agree on the order, and they never see the same one. The server creates transformers in the order its scripts run. The client creates its own in the order its scripts run. Either side can add one on any frame and drop one later. Agreeing on a shared numbering would need a registry and a round trip before the first packet could leave.
A hash needs no agreement at all. Both sides already hold the same string: the
transformer uuid, which for a plain carrier is the channel name you passed to
Pigeon.new. FNV-1a folds that string to a number, the number is taken
modulo the pool size, and one is added so buckets start at 1.
Pigeon.new("Shop") on the server and Pigeon.new("Shop")
on the client work out the same bucket without either being told the other exists.
Picking a remote, worked out separately on each machine
It also stays put. Adding a "Combat" channel does not move "Shop", and destroying one does not either. A channel's bucket only changes when the pool size itself changes, which happens on multiples of 16. A counter would reshuffle every time a system came up in a different order.
Why the server publishes the count
The bucket is the hash modulo the pool size, so the size has to be the same number
on both machines or they land on different remotes. Neither side can work it out
alone, because the size comes from how many transformers exist and the two sides
do not hold the same set. A client with three of its own would compute a pool of 1
while a server with 100 transformers is using 7. Worse in the other direction, a
client that guessed high would sit on WaitForChild for a
PigeonRef_9 the server never made, and that wait never ends.
So the server is the one that decides. It writes the size to a
RefCount attribute on the PigeonRefs folder and clients
read it there. A client that has not received the attribute yet uses 1, which is
always safe because bucket 1 is always in the pool.
What sharing a remote costs
The tradeoff is real: at 200 channels roughly 15 of them ride the same RemoteEvent. It does not cause mixups, because channels are not kept apart by which remote they use. Every packet carries its own address. The event string on the wire is the carrier name, a zero byte, then the event name, and both sides route on that string rather than on the remote it arrived through. That is also why a pool resize is harmless. If the two machines briefly disagree about the size while the attribute replicates, a packet goes out on a different lane for a moment and still lands in the same handler.
What you lose is legibility. PigeonRef_4 tells you nothing about which
feature it serves, so you cannot read traffic per feature out of the explorer the
way you can with named remotes. The two constants are judgement calls as well, not
measurements: 16 channels per remote keeps the folder small, and the ceiling of 32
stops a game with hundreds of transformers from filling
ReplicatedStorage with remotes it barely uses.
See The Ref Pool.
Per channel readiness
Not "the client is loaded", which is one flag for the whole game. Each channel declares itself ready on its own, and only that channel's backlog is released.
Why not one flag per client
Because one flag is released by whichever system happens to come up first, and it releases everything. The shop finishes loading, the flag flips, and the combat backlog is delivered into handlers that have not been registered yet. That is the original race, moved later and made harder to see, since it only shows up when load order shifts.
Keying readiness and the backlog by channel and player means Init on
one carrier says one thing and one thing only: this channel's handlers are in
place. Nothing the client has not asked for yet is released. A client can bring
systems up one at a time, in whatever order suits it, and each one opens on its
own clock.
Two channels, two clocks
The cost is that every carrier on the client needs its own Init, and a
carrier that never calls it never receives anything. That is the deal, and it is
the most common thing to forget when you start.
Note
Replies to a Call skip the queue on purpose. A reply is addressed
to a thread that is already sitting there waiting for it, not to a handler that
may not exist yet, so it goes straight out. That is why a request works before
Init while a broadcast does not.
A handshake that closes the channel in both directions
This is the part most access control misses. A guarded carrier will not deliver
what an unapproved client sends, and it will not include that client in what the
server sends. Every inbound packet passes one door and every outbound path
filters through one approval list, so a plain
Broadcast on a moderator channel is safe to write.
Why the filter runs before the send
The tempting shortcut is to broadcast to everyone and let each client ignore what it should not have. It does not work, for a reason no amount of client code fixes: once the bytes are on the machine they are readable. Whether the client shows them is up to whoever is running the client. A moderator alert filtered out by client code has still been delivered to every player in the server.
So the filter is on the way out. Broadcast,
BroadcastTo, BroadcastExcept and SendToRoom
all resolve their recipient list against the approval list first, and if that list
comes back empty nothing is packed and nothing is sent. CallTo returns
nil for an unapproved target without a packet leaving the server. Data
a client should not see never leaves the server, so there is nothing to ignore.
Inbound is the same door in reverse. A packet from a client that has not passed the guard is dropped before middleware runs and before any handler is looked up, with the handshake itself as the only exception. That is what makes a handler on a guarded carrier safe to write plainly: if it runs at all, the player passed.
Note
The guard runs on every attempt, so a client refused now can be admitted later without rebuilding anything. A later attempt that fails also revokes an earlier grant, and a guard that throws counts as a refusal, which clears an approval the player already had. Keep the guard cheap, because a client may ask as often as it likes.
The honest costs: approval is per carrier, so a player approved on "Moderation" is not approved on "Admin" until that carrier's guard says so. And a message sent while a client is locked out is gone rather than queued, because the server has no reason to believe it will ever be allowed to send it.
See Handshakes.
On and When are two different things
Traffic is routed by the kind of packet that arrived, not by which handlers happen to be registered.
| What you send | Where it goes |
|---|---|
Emit, Broadcast, BroadcastTo, BroadcastExcept, SendToRoom |
Every On listener on that event. |
Call, CallTo |
The single When responder for that event. |
Why they are not one thing
One event name is often both a notification and a question. "Score" as something the server announces and "Score" as something a client asks for are different traffic that happen to share a word. Splitting on the kind of packet lets both live on one name, and each goes to the side that can serve it.
Route on the handler list instead and one callback has to do both jobs. Then a fire
and forget delivery has to decide what to do with whatever that callback returns,
and there is no good answer. Nobody asked, so there is nowhere to send it. It gets
dropped, quietly, and the code that wrote return true looks correct.
Pigeon makes that structural rather than surprising. An On callback is
started with task.spawn, so it is already running on its own thread by
the time it returns and there is nothing on the other end of its return value by
construction. If you want to answer, you write When.
On | When | |
|---|---|---|
| How many per event | As many as you like. All of them run. | Exactly one. Registering again replaces it. |
| Thread | Its own, spawned. Order of completion is not fixed. | The thread handling the packet. |
| Yielding | Does not hold up the other listeners. | Allowed. The caller waits. |
| Return value | Goes nowhere. | Sent back as the answer. |
| On error | A normal Roblox error with a traceback. | Warns ERR_CALLBACK_ERROR naming "responder". The caller gets no answer and waits out its timeout. |
Both may exist on one event name at once, which is the point. Pigeon's own reserved
events follow the same rule and are a fair test of it:
__pigeon_ping, __pigeon_handshake and
__pigeon_table_request are questions, so they are registered with
When. __pigeon_table_patch,
__pigeon_table_snapshot, __pigeon_table_release and
__pigeon_table_revoke are announcements, so they use On.
Careful
A request for an event that has On listeners but no
When is not answered by those listeners. It is held like any other
unhandled packet, for up to 10 seconds, and replayed to the first
When that registers. A held request only ever replays to a
When, and a held fire and forget only ever replays to an
On.
The hold is the same length as the default Call timeout, so a
responder registered a moment late still answers, and one that never arrives
costs the caller a timeout and nothing else.
Off follows the same split. Passed a callback, it removes that callback
from the On list and clears the responder if that is what it was.
Passed no callback, it clears every listener and the responder for that event. The
transport binding is only dropped once both sides are empty.
See Sending and Receiving and Requests and Replies.
Plain tables that replicate themselves
Not a state class with getters and setters. You write
data.stats.health = 90 and that write replicates, nested path and
all. A burst of writes in the same frame coalesces into one patch, so filling a
table field by field costs one send instead of one per field.
Careful
Staged tables carry data, not objects. Instances, functions, threads,
metatables and cycles are rejected when you write them, rather than arriving as
nil on the far side.
See Staged Tables.
What happens when a client leaves mid request
Worth saying plainly, because it is the failure every hand written layer hits eventually. A player can disconnect between the moment the server decides to send them something and the moment it actually sends. Firing at a Player object that has left throws, and a throw inside a request handler takes out whatever else that thread was doing.
Pigeon checks reachability before every send to a client: the target has to be an
Instance, must not be marked as departed, and must still have a parent. The send
itself is wrapped as well, so losing the race between the check and the call cannot
error the server. That covers replies to a Call, every broadcast, and
the release of a buffered backlog.
When not to use Pigeon
If you have three remotes and no plans to grow, plain RemoteEvents are simpler and
Pigeon is a dependency you do not need. Three Instances and three
WaitForChild calls are less to hold in your head than channels, a
readiness call per channel and a guard model.
| If this is you | Then |
|---|---|
| A handful of remotes, a finished game | Stay as you are. The pool saves nothing below 16 transformers, where it is one remote and its unreliable twin, plus a folder and an attribute. |
| You need the bytes on the wire minimised | This is not that. Every packet carries a kind tag, the transformer uuid, and the channel and event names as strings, and requests carry a GUID as well. A hand rolled buffer format beats it. |
| You want to send Instances or functions in your state | Staged tables reject them. Send an identifier and look the object up on the far side, or do not use staged tables for it. |
| You already have a networking layer your team knows | The migration is real work. It only pays if you recognise the five problems above as ones you actually have. |
Nothing stops the two living together. Pigeon does not take over
ReplicatedStorage or intercept your remotes, so adding it for the two
systems that grew, and leaving the rest alone, is a normal way to start.
Where to go next
The fastest way to judge any of this is to build a channel and watch it work.
- Installation to add Pigeon to a place.
- Getting Started to build your first channel.
- The Ref Pool for the transport under all of it.
- Known Issues for what is still rough.
Pigeon takes its shape from roblox-sockets, which is where the idea of a named two-sided channel over a shared transport came from. See Credits.