Tips & Tricks for Playing Video with APL and Alexa

I am currently in the process of building an Alexa skill that contains all of the knowledge of the Star Wars Universe.  This includes characters, droids, weapons, vehicles, planets, creatures, and even different species and organizations.  It also includes the ability to request the opening crawl videos from each of the movies in the Star Wars saga, and the trailers for the movies, television shows, and video games.

It’s the videos that have brought me here to share what I have learned.

Alexa is available in a wide variety of devices.  Some small, some big, some with screens, others without.  For those devices with screens, I want to be able to provide my users with a simple workflow.

  1. Ask for a specific video.
  2. View the requested video.
  3. Continue the conversation when the video ends.

For the first two steps, this was surprisingly easy to implement using Alexa Presentation Language (APL.) . For the third step, it required some research and trial and error, but I have it working successfully now.

Identifying the Video a User Requested

While there is nothing complicated about identifying a user’s request, I’ll show you how I am handling this so that if you want to build your own version of this, you have everything you need.

In my Interaction Model, I have an intent called “CrawlIntent.”  This is there to handle all of the ways a user might ask to see the opening crawl of a specific film.  It looks like this:

{
  "name": "CrawlIntent",
  "slots": [
  {
    "name": "media",
    "type": "Media"
  }
  ],
  "samples": [
    "show me the {media} crawl",
    "{media} crawl",
    "can I see the {media} crawl",
    "show the crawl for {media}",
    "for the {media} crawl",
    "to show the crawl for {media}",
    "show me the {media} opening crawl",
    "{media} opening crawl",
    "can I see the {media} opening crawl",
    "show the opening crawl for {media}",
    "for the {media} opening crawl",
    "to show the opening crawl for {media}",
    "play the {media} opening crawl",
    "play the {media} crawl"
  ]
}

When a user says something to my skill like one of the utterances above, I can be confident they are looking for the opening crawl video for a specific film.  I also have a slot, called media that contains a list of all of the films and shows that I want my skill to be aware of.

{
  "values": [
    {"name": { "value": "Battlefront 2","synonyms": ["battlefront 2", "battlefront"]}},
    {"name": { "value": "Clone Wars","synonyms": ["the clone wars"]}},
    {"name": { "value": "Episode 1","synonyms": ["the phantom menace"]}},
    {"name": { "value": "Episode 2","synonyms": ["attack of the clones"]}},
    {"name": { "value": "Episode 3","synonyms": ["revenge of the sith"]}},
    {"name": { "value": "Episode 4","synonyms": ["a new hope", "new hope"]}},
    {"name": { "value": "Episode 5","synonyms": ["empire", "the empire strikes back", "empire strikes back"]}},
    {"name": { "value": "Episode 6","synonyms": ["return of the jedi", "jedi"]}},
    {"name": { "value": "Episode 7","synonyms": ["the force awakens", "force awakens"]}},
    {"name": { "value": "Episode 8","synonyms": ["the last jedi", "last jedi"]}},
    {"name": { "value": "Episode 9","synonyms": ["rise of skywalker", "the rise of skywalker"]}},
    {, "name": { "value": "Rebels","synonyms": ["star wars rebels"]}},
    {"name": { "value": "Resistance","synonyms": ["star wars resistance"]}},
    {"name": { "value": "Rogue One","synonyms": ["rogue one a star wars story"]}},
    {"name": { "value": "Solo","synonyms": ["han solo movie", "solo a star wars story"]}},
    {"name": { "value": "The Mandalorian","synonyms": ["the mandalorian"]}}
],
"name": "Media"
}
This slot allows me to match the user’s request against the list of items my skill can handle, using Entity Resolution.  This allows me to be certain that I’m choosing the right video for their request.

 

Playing A Video Using APL

For the code of my skill, I am using the Alexa Skill Kit SDK.  This makes parsing through the JSON that Alexa provides far easier, and gives me greater control over building responses for my users.

To add APL to my skill’s response, I do something like this:

var apl = require("apl/videoplayer.json");
apl.document.mainTemplate.items[0].items[0].source = media.fields.Crawl;
handlerInput.responseBuilder.addDirective({
  type: 'Alexa.Presentation.APL.RenderDocument',
  token: '[SkillProvidedToken]',
  version: '1.0',
  document: apl.document,
  datasources: apl.datasources
})
handlerInput.responseBuilder.getResponse();

Line #1 refers to the location of my APL document.  This document is the markup that tells the screen what to show.  Line #2 is dynamically updating the source of the video file to be played, so that we can play the appropriate video for the appropriate request.

As you’ll see in the APL document below, we define a Video element, and include a source property that indicates a specific URL for our video.

The important lesson I learned when building this is that I don’t want to include any speech or reprompts to my user in this response.  I can send this APL document to the user’s device, which immediately starts playing the video.  This is completely counter-intuitive to everything I’ve ever considered when building an Alexa skill, but it makes sense.  I’m sending them a video to watch…not trying to continue our conversation.

Adding an Event to the Video When It Is Finished

Finally, I had to do some exploration to figure out how to not only identify when the video has concluded, but also prompt my skill to speak to the user in order to continue the conversation.  This is done using the onEnd event on the Video element that we created earlier.  Here is the entire APL document.

{
  "document": {
    "type": "APL",
    "version": "1.1",
    "settings": {},
    "theme": "dark",
    "import": [],
    "resources": [],
    "styles": {},
    "onMount": [],
    "graphics": {},
    "commands": {},
    "layouts": {},
    "mainTemplate": {
      "parameters": [
        "payload"
      ],
      "items": [
      {
        "type": "Container",
        "items": [
          {
            "type": "Video",
            "width": "100%",
            "height": "100%",
            "autoplay": true,
            "source": "https://starwarsdatabank.s3.amazonaws.com/openingcrawl/Star+Wars+Episode+I+The+Phantom+Menace+Opening+Crawl++StarWars.com.mp4",
            "scale": "best-fit",
            "onEnd": [
            {
              "type": "SendEvent",
              "arguments": [
                "VIDEOENDED"
              ],
              "components": [
                "idForTheTextComponent"
              ]
            }
            ]
          }
          ],
          "height": "100%",
          "width": "100%"
        }
        ]
      }
    },
    "datasources": {}
}
This is the second lesson that I learned when building this.  By adding this onEnd event, when the video finishes playing, it will send a new kind of request type to your skill: Alexa.Presentation.APL.UserEvent. You will need to handle this new event type, and prompt the user to say something in order to continue the conversation. I included the argument “VIDEOENDED” so that I’d be confident I was handling the appropriate UserEvent. Here is my example code for handling this:
const VideoEndedIntent = {
  canHandle(handlerInput) {
    return Alexa.getRequestType(handlerInput.requestEnvelope) === 'Alexa.Presentation.APL.UserEvent'
    && handlerInput.requestEnvelope.request.arguments[0] === 'VIDEOENDED';
  },
  handle(handlerInput) {
    const actionQuery = "What would you like to know about next?";
    return handlerInput.responseBuilder
      .speak(actionQuery)
      .reprompt(actionQuery)
      .getResponse();
  }
};
With these few additions to my Alexa skill, I was able to play videos for my users, but bring them back to the conversation once the video concludes.
Have you built anything using APL?  Have you published an Alexa skill?  I’d love to hear about it.  Share your creations in the comments!

Introducing the Smart Deck

We recently renovated our 15-year old wooden deck, and I wanted to share with you how we created a smart deck.  Here’s what it looked like before we started this project (we had already cut the benches up before I took the photo):

OldDeck

This video illustrates how it has changed pretty well.

The technology behind everything is actually pretty simple.  For the floodlight, it’s a standard floodlight hooked up to a WeMo Light Switch.  I’ve had this switch installed for about three years now, and it’s still the perfect solution.  We have 6 more of these throughout our house.

For the colored lights in the deck itself, I took a chance on a set of LED lights that I found on Amazon.com that were listed in the “Works with Alexa” category.  They’re made by a company called FVTLED.  Could not be happier with how they turned out.  Each 10-light kit costs about $100, but has a wi-fi module, a remote control, and an outdoor power supply as part of the kit.

You couldn’t see them in the dark (and I didn’t want to turn them on and wake the neighbors), but there are two speakers connected to a Bluetooth receiver mounted above the deck as well. This allows me to pair an Alexa device, or my phone, to the receiver, and play music through the speakers.

The Grace Digital receiver is small.  Maybe 6 inches wide, and 10 inches deep.

Grace Digital GDI-BTAR513 100 Watt Digital Integrated Stereo Amplifier with Built-In AptX Bluetooth Wireless Stereo Receiver

The Yamaha speakers are pretty standard outdoor speakers.  I had to run speaker wire to them, and they don’t require any additional power to run them.

Yamaha NS-AW150WH 2-Way Indoor/Outdoor Speakers (Pair, White)

 

Overall, I’m pretty happy with how this turned out.  I don’t actually expect that I’ll be running techno dance parties with flashing colored lights, but I love that I have the option.  Most of the time, I expect to be running standard white (or off white) colors.

Have you done anything cool to improve your outdoor living space?  I’d love to see it!

Making An Alexa Raspberry Pi

Last week, I ordered all of the bits and pieces I needed to get a Raspberry Pi configured to become an Alexa device.  It was incredibly easy, the tutorial was very straightforward, and I ended up with something that can do this:

What You Need

If you want to try this, here’s what you’ll need (links and prices from Amazon):

Optionally, you might want to protect your Raspberry Pi if you plan to take it anywhere.  They make a very nice, inexpensive case for it:

Finally, there are a few things you’ll need to get it running, but these are things I assume you probably have.  If you don’t, I’ve recommended some with the links.

  • USB keyboard & mouse (Logitech MK270 Wireless USB keyboard and mouse – $19.95)

    keyboard
    I like this one because it’s small, compact, and easy to travel with.  Most travel keyboards are garbage, so I tend to lean towards smaller, full-function keyboards instead.  (My primary keyboard is a Das Keyboard, much bigger and clickier.)

  • HDMI monitor (there are way too many options here, any monitor will do. I’m hunting for a tiny one I can travel with.  Like 5″ or smaller.  But in a secure case, since it will likely see the bottom of my backpack occasionally.)
  • Micro-USB Charging Cable (you literally have 100 of these in a drawer. Any of them.)
  • 3.5mm audio cable
  • Literally ANY speaker that can take a 3.5mm audio cable as input (I used the Nokia MD-12 for mine, but you can certainly find cheaper speakers if you need one.)

    nokiamd12

The How-To

I would normally give you a run-down of the steps I took, and the issues I faced, but there simply isn’t much point in that.  I followed the provided tutorial on GitHub, and it was one of the smoothest experiences I’ve ever had setting something like this up.

https://github.com/alexa/alexa-avs-sample-app/wiki/Raspberry-Pi

My Takeaways

I’m working on a few things to enhance the experience, but here’s my takeaways:

  1. If you ONLY want an Alexa device, this is probably not the project for you.  The Echo Dot is $49.99, and doesn’t require any setup to work.  This project, at a minimum cost, is about $53.15.  That being said, having an Alexa device that can also run some other services is really compelling.  Adding a touchscreen to it would allow you to see the “cards” that Alexa skills produce at http://alexa.amazon.com, for example.
  2. Each time you power up the Raspberry Pi, you have to manually start all of the services again.  I’m hoping that with some creative effort, this might not always be true, but there’s some authentication that happens that requires your monitor, mouse, and keyboard every time you power it up.  (This is why I’m looking for travel keyboards and monitors.)
  3. This was one of my first experiences in the Raspberry Pi ecosystem, and I’m very excited by what I found.  There are tons of accessories to enhance and protect your device, and I’m looking forward to seeing where I can take this project forward.

 

 

 

Blitzball Strike Zone

A couple of years ago, I played a few rounds of whiffleball with my friends Ryan and Travis Lowdermilk.  Travis went so far as to build a strike zone out of PVC and sheet metal, and it was an amazing amount of fun.

Since that time, I’ve seen all sorts of awesome things about the Blitzball, including the guys over at Dude Perfect.

This past weekend, I decided I needed to make one for myself.  This is the final product:

20160818_154118234_iOS

The rest of this post is about how I built it, so if that sort of thing isn’t for you, here’s something fun to watch instead.

The Supplies

It’s a pretty affordable setup, made entirely of PVC, sheet metal, and zip ties.

Total cost?  $41.45 with 7.5% sales tax.

The Frame

I knew that I didn’t want to have to cut the sheet metal.  I’d do a poor job, and it was pretty close to the standard strike zone size, give or take an inch or two.  This meant that I needed to make a PVC frame that would hold a 24″ x 36″ piece of metal, with a little space to give.  I ended up setting my dimensions of the inside of the frame to be an inch larger in both directions, 25″ x 37″.

This meant cutting out these pieces:

  • 2 x 37″ (sides)
  • 1 x 25″ (top)
  • 2 x 4.25″ (bottom)
  • 1 x 12″ (bottom)

Because we’re using the Tee joints to connect to the base, I had to break the bottom up into three separate pieces, that’s why you see those cuts in the list.  The entire frame should be able to come from ONE of the 10′ PVC pipes.

20160816_211334534_iOS

When it’s assembled, it looks like this:

20160816_213909777_iOS

The Base

I know, I know, you’re all about that base.  I saw some photos of a strike zone that another person had built, and it had angled legs that reminded me of how a catcher sits behind the plate.  I decided to emulate that look, because it looks cooler.

The cuts you need for the base are here:

  • 1 x 23″ (front)
  • 2 x 16″ (sides)
  • 2 x 12″ (legs)
  • 2 x 7″ (connect to frame)

20160816_213942133_iOS

When you assemble the base, it should look like this, where the legs tilt in towards the center a bit:

20160816_224718568_iOS

Glue?

I would generally recommend using some PVC primer and cement at this point.  It will make your strike zone virtually indestructible.  That being said, I simply used a rubber mallet to make sure all of my pieces were completely connected, and that seemed to make it strong enough.

Put Them Together!

Once you’ve gotten the two sections cut and assembled, put them together!  Your strike zone should resemble the one below.  (See how much cooler the angled legs look?)

20160816_225715575_iOS

Paint It!

At this point, you could say you’re done with the frame, but I highly recommend painting it.  Pick your favorite team colors, or just go with black.  Either way, a coat of paint will do wonders.

20160818_151015271_iOS

Prepping the Sheet Metal

I also painted my sheet metal, but you certainly don’t have to.  The galvanized steel finish looks good on its own, but some color really makes it look finished.  Unfortunately, I didn’t take any pictures of my sheet metal when I was painting it, but you can see what it looked like from the first photo at the top of this post.

Drill Your Holes

I’ve seen some folks that just hammer a nail through their sheet metal to make the holes for the zip ties, but I think that would result in some bends to the metal that I wanted to avoid.  I just used a simple drill bit that was slightly larger in diameter than the zip ties.

On the top and bottom (24″ edges), I punched holes at 3″, 12″, and 21″ across.  Especially on the bottom, these measurements are important, because you don’t want your drill holes to be over the Tees that connect to the base.

On the sides (36″ edges), I drilled holes at 6″, 12″, 18″ 24″, and 30″.  You can space these out however you’d like, but make sure they’re evenly spaced on both sides, or your sheet metal might hang slightly crooked.

Attach the Sheet Metal

Run the zip ties through each of the holes, and tighten.  I don’t recommend over-tightening them, because your PVC will just pull towards the center.  Hand-tightening should be plenty.

20160818_154118234_iOS

Take Advantage of the Power of Magnets

You’ve got a magnetic surface, so why not add some flair?  I had a large Cleveland Indians logo hanging on a refrigerator in my basement that added the perfect finishing element to my PVC strike zone.  You can get one for just about any team on Amazon.

So, that’s it!  Time to get some pitching practice in!  Let me know if you build one…I’d love to see it!