Some notes on Bitfocus Companion and the StreamDeck

[ajay_m]

While I posted on this topic in connection with adding functionality to my DM3 through a StreamDeck, I thought it might be more useful to post what I know now about the product as a more general topic.

The art of documentation is largely a lost one, so these wonderful open source projects suffer terribly from unstructured, out of date and often fairly incoherent documentation. This led to a bit of head-scratching getting to understand what you can do with the StreamDeck and Bitfocus, which is a shame because the possibilities are huge.

What is Bitfocus Companion?

Bitfocus Companion is a free open source product that directly connects to the StreamDeck series of products. It augments, rather than replaces, the Elgato software. The product is written in Javascript using node.js and uses your browser for the user interface. It can run on Windows, Mac and Linux, including the Raspberry Pi.

It connects to devices which can be remote from the computer on which it is running, using standard TCP/IP connections. You install one or more 'connectors', which expose functionality specific to a device. For example, there is a connector for the Yamaha RCP protocol (which is just OSC as far as I can see), enabling you to control almost anything on a wide range of Yamaha mixers.
There is also a connector for Reaper. This allows you to control pretty much anything in Reaper from the Streamdeck.

But wait, I here you say. Why would I bother. I can create keystroke sequences using the standard Elgato software to control stuff.

Well there are two reasons. Firstly you might not want to lug a laptop around for a gig. With Companion you could have it running on a little Raspberry Pi instead, and have everything connect at boot time; just plug the Streamdeck in and off you go. Secondly, Elgato keystroke binding doesn't give you full duplex control. If you want Reaper transport controls where the record button lights up red when Reaper is recording, a play button that lights up green when playing, etc, and the buttons change if you also control Reaper directly, then you can't do that with keystroke binding.

You *can* use the MIDI plugin with Elgato for quite extensive full duplex control, mind you, and the latest version of this has a Mackie Control button that can send a range of Mackie commands, as well.

Connecting to a StreamDeck

There are two modes by which you can connect. Either you natively connect to one or more Streamdecks from Companion, in which case you cannot also run the Elgato software, or you use a companion button plugin inside the Elgato software to map one or more buttons to Companion. I think the latter is much more flexible because you can 'mix and match' buttons that are controlled either via Companion or via Elgato, on the same page.

How you set up the plugin isn't immediately obvious. You put a Companion plugin button on a StreamDeck page and then you fill in the row and column that maps it to the button you already defined in Companion. It will then inherit the icon and behaviour of the Companion button. You must also disable direct connection in the Companion settings, as it will rely on Elgato to connect instead.

Full duplex operation

When you create a button in Companion you define the press action (and it's possible to define release actions as well for quick and slow releases). This maps through whatever connector you choose to some kind of action e.g Reaper Play. Then you can optionally add one or more 'feedbacks'. For example you could change the background colour of the button if Reaper was in play mode. So then whether you press play on the StreamDeck or on Reaper, the button will turn green when Reaper is playing.

It wasn't entirely obvious how to have an icon where the symbol itself changed colour. I tried to create transparent background icons but without much luck. Then I discovered that for most symbols, you can just directly use a Windows symbol character. (or characters; for instance to get a pause symbol you can add two vertical bars from the symbol font).

To do this press the Windows key and the full stop key then choose from a huge range of Unicode symbols. Because these are fonts, your feedback action can now change the foreground colour of the font character. This gives you e.g a play or stop symbol that changes colour when the transport mode changes.

Note: for buttons where you want this, don't define an icon in the Elgato software because it always covers the button face and won't change colour. You define button text in Companion and connect it up to feedback instead.

VU meters

For the Yamaha RCP plugin, there is a VU meter that you can add to any button. This is a stereo meter and you bind the L and R values to 'variables' which reflect live data back from the desk.
After adding the VU meter as a feedback entity, you add more feedback entities which populate the variables.
For example, I have monitor buttons which change colour when monitoring is active for a bus, plus they have VU meters for the bus.
This works by assigning the press value of the button to e.g

yamaha-rcp: Monitor/St/SourceCh/Mix

and assigning the bus number 1 (I have busses 1 and 2 stereo linked).

Then when I press the button this toggles monitoring for those busses. However to get a VU meter and also feedback I then:-

(a) add a VU meter in the feedbacks page
(b) bind the left channel to the variable $(yamaha-rcp:V_Meter_Mix_1_PreEQ) and the right channel to the variable $(yamaha-rcp:V_Meter_Mix_2_PreEQ)
(c) Add a feedback entity bound to yamaha-rcp: Meter/Mix and assign to bus 1. Then check the 'create variable automatically' checkbox
(d) Add a feedback entity bound to the same thing but this time bus 2 and repeat the checkbox. Now those two variables that the VU meter are bound to are updated in real time from the desk metering.
(e) Add a feedback entity for the source yamaha-rcp: Monitor/St/SourceCh/Mix. Then assign this to change the background colour of the button. Now when I toggle monitoring, the button background changes, in full duplex, so if I go to the desk and control it, the StreamDeck will correctly mimic the desk settings. Plus I have a live VU meter on the button as well for that bus.

It ought to be possible to do this for lots of other audio gear if it supports OSC metering or if there's a way to get back data from the device. You can build your own connections but I haven't looked into the details at this stage. There are connectors for A&H desks amongst other stuff, too, but I haven't looked at them.

As far as I can see, the VU meter ought to work with any piece of kit, not just Yamaha desks, if you can populate its variables. It just relies on something to fill them in, so although there are then additional feedback entities that do OSC GETs from the desk, you could have a feedback entity that got data from some other piece of equipment, e.g Reaper's metering I would imagine, though I haven't yet tried this.

Clock

You can add a handy real-time clock by adding a button and then binding its text to the inbuilt variable $(internal:time_hms)

When you do this, the button displays the current time. There are dozens of built in variables and you can make one button affect other buttons and all kinds of brain-meltingly complex stuff if you want.

Sending keystrokes from Companion

Because Companion is a client-server product and expects to control something remotely via TCP/IP, it doesn't directly support keystrokes like the Elgato software does. Assuming you don't want to use the StreamDeck in mixed mode where you put Companion buttons on an Elgato profile (perhaps because you want to run Companion on a stand-alone device like a Raspberry Pi), then the Vicreo connector lets you send keystrokes to a remote device. You install the Vicreo listener on that device and then Companion can send keystrokes to it.

Resilience

Companion seems to be resilient to stuff going offline. The buttons indicate a little red dot in the top corner for my Yamaha RCP stuff until I turn the desk on, and it doesn't seem to matter if I do this after Companion is running, so it seems fairly resilient to actual real world abuse. Since it's widely used nowadays in the video industry that's probably not surprising. I doubt Elgato would have produced their rack mounted StreamDeck if Companion didn't exist.

StreamDeck +

The SD + has rotary encoders and Companion does support these. I don't (yet) have an SD+ but I imagine you can map these full duplex to things you can twiddle e.g EQ or whatever. I suspect I will succumb and get an SD+ in the not too distant future and try it out.

Why not just use an iPad as a remote?

I think the main reason is that these are physical buttons. Unlike a touchscreen you can rest your finger lightly on the button and then press it, which gives you precise control and the interaction with a physical control surface. The SD series are rugged devices as well.

Reaper and OSC

You set up Reaper using its control surface functionality to add an OSC controller. You choose Configure device IP+local port as the mode, then I set device port = 9001 device IP = 127.0.0.1, local listen port 9002, local IP 192.168.178.63 (which is the IP address of my laptop)

Then in the Companion Reaper connector I set target IP 127.0.0.1, target port 9002, listen port 9001

[The Elf]

Wow! A lot of useful information in there.

I'll just add a perfunctory note that, having used Companion with my Stream Deck (to control a mapping projector), I found it all a bit complex to tie together, and the results were not reliable IME.