The combat rules governing ship-to-ship combat in Spelljammer: Adventures in Space are spread out across several parts of the Astral Adventurers Guide, and over into other books as well. Here is a brief digest of the rules as they are used in the DungeonSports Coliseum, as written in the Astral Adventurers Guide, pp. 17-57.

Spelljammer Combat Rules, for encounters with a hammership like this one.
Hammership art from Astral Adventurers Guide

Initiative

Initiative in ship-to-ship combat follows the “Side Initiative” variant found in the Dungeon Master’s Guide:

Side Initiative
…players roll a d20 for their initiative as a group, or side. You (e.g. the DM) also roll a d20. Neither roll receives any modifiers. Whoever rolls highest wins initiative. In case of a tie, keep rerolling until the tie is broken.

When it’s a side’s turn, the members of that side can act in any order they choose. Once everyone on the side has taken a turn, the other side goes. A round ends when both sides have completed their turns.

If more than two sides take part in a battle, each side rolls for initiative. Sides act from the highest roll to lowest. Combat continues in the initiative order until the battle is complete.

Dungeon Masters Guide p. ??

DungeonSports Rule: Combat continues in this initiative order until and unless crew members are separated from their vessel. This may occur either intentionally (e.g. as part of a boarding), or as a result of hostile action. All creatures that board another vessel, along with all creatures on the boarded vessel immediately roll initiative.

The DungeonSports Coliseum uses standard initiative rules for Spelljammer encounters. The vessel moves on the Spelljammer’s turn. Individual PCs/NPCs may apply actions to prepare ships weapons to fire.

Attunement to a Spelljammer Vessel

Any creature with spellcasting ability can attune to the spelljammer helm of a vessel following normal attunement rules. For the purposes of the DungeonSports Coliseum, this means that the Spelljammer for a given combat encounter must be selected prior to combat. This attunement may be transferred to another spellcaster through physical contact. This consumes one action.

Maneuvering a Vessel

While seated in a Spelljammer helm that it is attuned to, a spellcaster may move the vessel and ram without using any of their actions.

Operating Shipboard Weapons

Shipboard weapons list a number of crew necessary to operate. These weapons are inoperable without the specified number of crew.

Boarding

Creatures may move from one vessel to another when they pass within 5 feet of each other. This may be done while the vessel is moving, as a held action by the creature.

Damage Threshold

Vessels have a Damage Threshold. Damage that does not exceed the damage threshold is not taken by the vessel.

Repairing Your Vessel

It’s real slow, and you mostly won’t need to worry about it in combat.

Spelljammer Combat Rules May Need Some Tightening

We’ll be tightening up our understanding of these rules, and developing new DungeonSports rules as we play more! For now, we’ll strike through any rules we edit so that you can see how changes have progressed. In the meantime, you can find the rest of the DungeonSports Coliseums rules here.