Skip to main content

Conduit

Conduits are a wrapper transport type that simplify forward EventSub notifications to other transports.

Conduits are intended for server-side applications, and utilize app access tokens to manage EventSub subscriptions.

A conduit is comprised of shards, which can be webhooks or websockets.

info

Conduits are typically utilized by backends where webhooks are not feasible (e.g., due to firewall restrictions). Thus, the server can receive EventSub notifications via websockets (that are associated with a conduit).

A conduit can be associated with many EventSub subscriptions (often spanning multiple users). When an EventSub notification is triggered, Twitch decides which of the conduit shards will receive the notification (as if Twitch were handling load balancing for you). If the target shard did not receive the notification, Twitch will retry with one other shard.

note

Conduits can also easily be scaled up with additional shards. Scaling down is also possible, but more tedious.

Clients are limited to 5 enabled conduits, with up to 20,000 shards per conduit.

Conduits can be more resilient than plain webhooks or websockets. If a standard websocket momentarily disconnects, all of the EventSub subscriptions must be recreated. If a standard webhook doesn't acknowledge a notification across Twitch's multiple attempts (typically 5 within a few minutes), the EventSub subscription is disabled. When the websocket or webhook is associated with a conduit, these failures only disable the shard. With conduits, all of the shards must be disabled for 72 hours before Twitch deletes the conduit.

tip

Avoid having more than one disabled websocket shard at a time, as EventSub notifications could be dropped by Twitch.

If webhook shards are used, Twitch will retry notifications for the given webhook, but will not retry across multiple shards.

You can utilize the conduit.shard.disabled topic to track when shards are disabled.

If you are using a Conduit backed by websockets on a single server, you can generally ignore this detail.

The subscription cost model is the same as normal webhooks; users simply authorize to your Client ID with the appropriate scopes, so you can create zero cost subscriptions. The default max_total_cost is still 10,000.

For more information, please consult the official documentation.

Library-managed websocket pool​

For the common use case of websocket-backed conduits, the library offers TwitchConduitSocketPool, which simplifies shard and subscription management.

Single Client​

If your application runs on a single server, you only need one TwitchConduitSocketPool instance.

Simply decide how many websocket shards you would like created (ConduitSpec#poolShards(int)) and provide client authorization (i.e., an app access token or both the client id/secret).

IEventSubConduit conduit = TwitchConduitSocketPool.create(spec -> {
    spec.clientId("your-client-id");
    spec.clientSecret("your-client-secret");
    spec.poolShards(4); // customizable pool size
});
conduit.register(SubscriptionTypes.STREAM_ONLINE, b -> b.broadcasterUserId("926829122").build());
conduit.getEventManager().onEvent(StreamOnlineEvent.class, System.out::println);

Under the hood, Twitch4J will create a new conduit, spawn each websocket, associate each websocket to the conduit, register the requested subscriptions to the conduit, and fire events as the conduit receives EventSub notifications.

You can utilize TwitchConduitSocketPool#getConduitId to obtain the conduit ID associated with the pool.

Once a pool has been created, you can utilize ConduitSpec#conduitId(String) to reuse the same conduit, which can be useful when restarting your bot (e.g., to avoid recreating every EventSub subscription).

Distributed Clients​

TwitchConduitSocketPool can also be used to subscribe to a single conduit across multiple servers.

For example, a conduit could be created with 12 total shards, and shared across 3 servers. Then, each server can create a TwitchConduitSocketPool instance with 4 pool shards each, all associated to the same conduit ID. The first server would cover shards with indices 0, 1, 2, 3, so the shard offset should be 0. The second server would cover shards 4, 5, 6, 7, so the shard offset should be 4. The third server would cover shards 8, 9, 10, 11, so the shard offset should be 8.

First server:

IEventSubConduit conduit = TwitchConduitSocketPool.create(spec -> {
    // spec.conduitId("the-conduit-id"); // specify the Conduit ID if it has already been created
    spec.clientId("your-client-id");
    spec.clientSecret("your-client-secret");
    spec.poolShards(4);
    spec.totalShardCount(12);
    spec.shardOffset(0);
});
conduit.getEventManager().onEvent(StreamOnlineEvent.class, System.out::println);

Second server:

IEventSubConduit conduit = TwitchConduitSocketPool.create(spec -> {
    spec.conduitId("the-conduit-id");
    spec.clientId("your-client-id");
    spec.clientSecret("your-client-secret");
    spec.poolShards(4);
    spec.totalShardCount(12);
    spec.shardOffset(4);
});
conduit.getEventManager().onEvent(StreamOnlineEvent.class, System.out::println);

Third server:

IEventSubConduit conduit = TwitchConduitSocketPool.create(spec -> {
    spec.conduitId("the-conduit-id");
    spec.clientId("your-client-id");
    spec.clientSecret("your-client-secret");
    spec.poolShards(4);
    spec.totalShardCount(12);
    spec.shardOffset(8);
});
conduit.getEventManager().onEvent(StreamOnlineEvent.class, System.out::println);

With the distributed approach, only one client needs to register the EventSub subscriptions, and events will be distributed across the various websockets by Twitch.

Manual conduit management​

For certain uncommon use cases, you may have to directly call the Helix API to manage the Conduit. Feel free to join our Discord (linked in the header) if you have any questions.

Adding webhook shards​

While TwitchConduitSocketPool only registers websockets, you could add webhook shards manually. Typically, this is only relevant for distributed clients where some servers cannot use webhooks while others can; most commonly people use exclusively webhooks or exclusively websockets across their shards rather than a hybrid of both.

First, you need to increase the shard count for the conduit via TwitchHelix#updateConduit. To obtain the current shard count, you can call TwitchHelix#getConduits.

Then, you can call TwitchHelix#updateConduitShards to register the new webhook shards at the specific shard indices that were just created.

note

Shard IDs are numeric strings that are zero-indexed.

Finally, you still will need to spin up a HTTPS webhook server as described here.

Decreasing the shard count on an existing conduit​

Single Client​

To reduce the number of shards used for the single TwitchConduitSocketPool use case:

  1. Specify the reduced shard count in ConduitSpec#poolShards(int) (and restart the client).
  2. Reflect the reduced shard count in the Helix API via TwitchHelix#updateConduit.

Distributed Clients​

For distributed clients, one typically only deletes shards if they are taking down a client for an extended period of time.

If the shard IDs to be deleted are the last shards (i.e., highest IDs), you can simply call TwitchHelix#updateConduit with the desired shard count.

Otherwise, you need to call TwitchHelix#updateConduitShards to shift existing shards to overwrite the shards to be deleted. Then you can call TwitchHelix#updateConduit with the reduced shard count.

For example, if a conduit has 10 shards, the IDs will be 0 through 9. If we want to remove the second and third shards, we can simply overwrite the id=1 and id=2 shards with the last two shards (i.e., id=8 and i=9) in the Helix API, which works because multiple shards can share the same transport. Then we can reduce the overall shard count of the conduit in the Helix API. Ideally you should restart the websocket clients that managed the last two shards with the updated ConduitSpec#shardOffset.