Advanced A/V Voice Control with Josh.ai

Image for post
Image for post

Here at Josh.ai, we spend a lot of time and energy figuring out how to implement straight-forward solutions to complicated problems. In the custom electronics world, the A/V client experience is usually one of the most important parts of a job. Unfortunately, it can also be one of the most complicated.

Below you’ll find a step-by-step guide to setting up an A/V system with Josh. Whether Josh is layered on top of an existing control system or as a stand alone system, you’ll find the process to be intuitive and simple, yet you’ll still be able to implement fairly complicated and impressive solutions for your client. This is aimed at home technology professionals.

An example of Josh.ai voice controls for Netflix

Recommended Devices

To provide the best experience possible, including allowing deep-linking (so that a client can simply say, “Watch House of Cards”), we recommend the following devices:

  • TV — Sony XBR models. These TVs can be reliably controlled via IP, and are also the most integrator friendly brand currently available. Note: we can also control most other models of TV via IR, using a Global Caché IP2IR / WIFI2IR device.
  • Media Player — Roku. We have a robust integration with most models of Roku. While we love Apple TV, Roku makes it easy for us to offer deep-linking functionality from many popular streaming services, such as Netflix, Hulu, Amazon, and YouTube.
  • Matrix Switchers — If you plan on setting up a distributed video solution, we recommend using the matrix switchers that are native to each control system, i.e. Crestron DigitalMedia (DM) Server and Control4 Matrix Switchers.

Choose Your Setup

To start, navigate to Josh.ai/portal and in the top right dropdown list navigate to “Edit AV Suites”. Create a new AV Suite in the bottom left corner, such as “Living Room TV”, and then choose how you’ll be controlling things:

1. Directly Controlled — If you would like Josh to connect to the devices directly, choose this option.

or

2. Controlled by (Control4 / Crestron) — If you have previously installed a Control4 / Crestron system that is already controlling the A/V components, choose one of these options.

Image for post
Image for post
This is what the UI looks like on the Josh Portal.

Directly Controlled

To directly control the TV and other devices, you’ll want to select if this is a Network controllable TV, IR controllable TV, or Manually controlled TV.

If you’re working with a networked TV, we suggest a Sony XBR, which you’ll first have to authorize by following the auth steps in the Josh Portal.

If this is an IR controllable TV, you’ll want to set up a Global Cache IP2IR/WIFI2IR device.

Image for post
Image for post
  • Note: we do support other Networked TVs, but find they aren’t as reliable and therefore we strongly recommend Sony XBR.
  • For IR TVs, select the make and model of the TV as well as the port on the IR blaster that’s attached to the TV.

Next, you’ll want to define what inputs are connected:

Again, Roku makes deep-linking content from a variety of streaming services possible, so it’s a special case.

1. If there is a Roku device connected:

  • Select the TV input that it is connected to.
  • Select Set Top Box from the drop down menu and click Update.
  • Note: If you don’t see Roku in the dropdown list, your Roku is not on the network. Make sure it has power and a network connection.

By setting it up this way, when you give a voice command such as, “Watch Stranger Things”, the TV will power on, the input will switch to Roku, and Roku will launch Netflix and play the content.

2. If the device is not a Roku:

  • Select the TV input that the device is connected to.
  • Select Custom.
  • Type in the device’s name and click Update.

This will enable voice commands to switch input. For example, if you have an Apple TV plugged into HDMI 3 and set this up, you’ll be able to say “watch Apple TV” and the TV will turn on and switch to the right input.

If you have a receiver doing the input switching, skip the TV Inputs section and instead go to Receiver.

  • Here you’ll select if you have a network receiver (Integra, Onkyo, or Sony) or if it is IR controllable. If it is IR controllable, select the make and model and the correct IR blaster port that is connected to the receiver.
  • Do not forget to choose the correct input on the TV that the receiver is connected to.

Next, configure the inputs that are connected to the receiver.

If there is a Roku device connected:

  • Select the input it is connected to on the receiver.
  • Select Set Top Box from the drop down menu and click Update.

If the device is not a Roku:

  • Select the input the device is connected to and then select Custom.
  • Type in the input name and click Update.
Image for post
Image for post
Photo: Source

Controlled by Control4

When configuring Josh to use a Control4 A/V setup, you’ll need to have Composer open and running at the same time. Before you set up an A/V suite with Control4, you’ll want to pre-program a few Control4 event IDs (for info on how to create Control4 event IDs in Composer, check out this article). You will need to create one event ID for each of the following:

1. Turn the TV on

2. Turn the TV off

3. Turn on the TV and switch the input source to Roku (repeat this for any other input you’d like to add)

4. Increase the volume

5. Decrease the volume

  • To start, select the Controlled by Control4 tile and click Save and Continue.
  • Next, select Add Macro and you’ll be prompted with the screen below:
Image for post
Image for post
  • In Composer, you’ll want to select an event ID (a number between 1–100) that Josh will trigger. Once you’ve selected the ID click Save and Continue.
  • Next, you’ll want to decide if the ID turns the TV on or off. Once you make that selection, click Continue.
  • On the next screen, you’ll choose if the ID switches inputs, changes volume, or neither. If the ID simply turns the TV on or off then select Neither Change continue.
  • If the ID switches inputs you’ll want to select that option and proceed to the screen pictured below:
Image for post
Image for post
  • If you have a network A/V device that we integrate with, such as a Roku, then you’ll see it listed as an input to choose from. If not, then you’ll need to add the custom input. You can do that by typing in the name of the input and then clicking on the blue plus button.
  • You can also add aliases for each of these inputs by clicking EDIT underneath the new input you created.
  • Now, select the input from the options you have available and click Save and Continue.

You’ve now successfully created your first Macro through Control4 that can be called by Josh. For a large complex home you’ll want to set up a Macro for each TV and input source in order to enable commands such as, “Watch ESPN in the living room” or “Turn on the Blu-ray player in the office.”

Image for post
Image for post
Photo: Source

Controlled by Crestron

Before you set up an A/V suite with Crestron, you’ll want to pre-program a few Crestron scenes. You will need to create one scene for each of the following:

1. Turn the TV on

2. Turn the TV off

3. Turn on the TV and switch the input source to Roku

4. Increase the volume

5. Decrease the volume

Once you’ve done that, you can go ahead and create a new A/V suite in Josh Portal.

  • From the options select Controlled by Crestron and then click Save and continue.
  • Then, select Add Macro.
  • Next, from the drop-down menu select the correct scene. This will show a dropdown of scenes added to the Josh Module. If you don’t see the scene you’re looking for, make sure it’s loaded into the module properly.
Image for post
Image for post

You will need to ADD MACRO for each of the scenes you created above, see example below.

Image for post
Image for post

In the case above, the user created a scene that switches to Apple TV and also a scene that switches to cable.

After this is done, you’ll be ready to voice control your A/V set-up using Josh. If you have one of the Crestron scenes turn on the TV and switch to Roku, that will trigger when you give general TV deep-linking commands such as “Watch Planet Earth.”

Becoming a Josh.ai Dealer

If you’re not already a Josh.ai dealer, but would like some more information please go to our dealer page or send us email at installer@josh.ai.

Advanced TV control with Josh.ai

The A/V suite is pretty powerful, and we’ve just explored the basics above. We encourage you to explore and try new things. If you have questions or feature requests, reach out and let us know.

Josh.ai is an artificial intelligence agent for your home. If you’re interested in learning more, visit us at https://josh.ai.

Like us on Facebook, follow us on Twitter.

Written by

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store