This is actually a very common scenario, it may be less recognizable because you have custom hardware. The more commonly recognized scenario is a wireless input controller, like an XBox controller. An XBox controller sends and receives data. It tells an application that a button is being pressed, and an application tells it to vibrate.
Your custom hardware talks to your application via some defined data standard, like serialized JSON data. This enables them to send data back and forth, respond to commands, provide input, etc.
The medium for that data is dependent on your platform, it could be WiFi or Bluetooth or USB. The medium you use will dictate the connection process and how transparent that process is to the user.
Since this is your application, with your custom hardware, the decisions about what will work best are up to you as the developer. Just know that you'll need to decide on a data format and a medium (or you can support multiple mediums if you're feeling ambitious).
As an example. I worked on a project for the HoloLens that communicated with a Raspberry Pi. The Pi application hosted a web server, which exposed various endpoints for collecting data and receiving commands. It also broadcast itself as a host on the WiFi network it was connected to. The HoloLens would then connect to the same WiFi network, and the HoloLens application would listen for host broadcasts, and connect. Then the HoloLens app would show a Mixed Reality interface to read data from and control various bits of hardware that were attached to the Pi. The HoloLens application also had an alignment phase, allowing you to align to the Pi. This enabled the holograms to be appropriately placed over the components they were getting data from or sending commands to.