How to Build a Google Assistant App?

Apps for the Google Assistant are the gateway for your users to engage with your services through Google Home, Android phones, iPhones, and in the future, through every experience where the Google Assistant is available.

In this video, I’m showing how to leverage the capabilities of API.AI (around NLP) and build an assistant app.

If you prefer to read, here is a tutorial that cover the same content. Want to try it with code? Check this code-lab.

More videos on how to build Google Assistant apps, will be published on this Action on Google youtube list.

The Transcript

Apps for the Google Assistant are the gateway for your users to engage with your services through Google Home, Android phones, iPhones, and in the future, through every experience where the Google Assistant is available. It’s a new, efficient way to interact with users and make them happy!

Google has been investing heavily in the areas of:

  • Machine learning —Teaching computers how to see patterns in data and act on it.
  • Speech recognition and Language understanding — Being able to understand you when you are talking with all the little differences and nuance.

Google Assistant brings it all together.
It allows users to have conversation with Google and be more productive. With Actions on Google you can use our investments in this space and your users will enjoy these new capabilities.

The user needs to invoke your app with a phrase like “Ok Google, let’s talk to Animal joker” Now Google knows the name of the app the user choose to use.

The user is now talking to your app.

Your app generates dialog output, which is then spoken to the user.

The user then makes requests, your app processes it, and replies back again.

The user has a two way dialog until the conversation is finished.

For our example we are going to use API.AI

API.AI uses machine learning to understand what the user is trying to tell your app. Believe me, they are saving us A LOT of work!

You can then respond to the user’s request from within API.AI or send the information on the user’s intent to your own server to perform any required actions for fulfillment and respond to the user.
After the login you can create your first app for Google assistant. You will need to:
Give your app a name.

In our case, it will be “AnimalJoker”. Please note that the app name can not contain any spaces between the words.

Click on ‘Save’.
It’s the button in the top-right corner of the screen.

What are entities?

Entities are the values we are trying to capture from the user phrases. Kind of like filling out a form, requesting details from the user. API.AI looks to extract these out, and will do follow up prompts until completed . This is how an entity looks in API.AI

We will create an Animal entity.

First step is to click on the ‘Create Entity’ button (it’s at the top-right corner).

Next you should start typing animal names.
The final results should look similar to the image on the right.

Things to remember: You should ‘help’ API.AI machine learning algorithm train itself by providing synonyms.

For example, dog could also be puppy. In our case, you can give it only 2–3 animals.

That will be fine for our prototype.
In the real world, try to give many examples so it will cover more cases.

What is an intent?

An Intent is triggered by a series of “user says” phrases. This could be something like “please tell me a tiger’s joke”.

You need to specify enough sentences to train API.AI’s machine learning algorithm. Then even if the user doesn’t say exactly the words you typed here, API.AI can still understand them!

You should create separate intents for different types of actions though. Don’t try to combine all of them together.

In our example, we will create only two intents:
* Tell_Joke intent — This intent will handle the jokes.
* Quit intent — This intent will handle the part when the user wish to finish the action.

Build the “Tell_Joke” intent

After we have our new $Animal entity.

If you notice the $ before the word — It’s not a mistake. This is the way we will refer to our new entity from now. Think of it as a special sign to show us that we are referring to our entity and not just another animal.

It’s time to create the intent that will tell us the jokes.
First, click on the ‘Create Intent’ button.

Second, start typing a few sentences that you will want to use to get a joke.

For example, “please tell me a joke on dogs”.
Type a few sentences so API.AI could start training its algorithms. You can see that while you type, API.AI automatically recognizes that the phrase includes one of the entities, and highlights it.
See how it should look like.

In the ‘Action’ section we need to make sure that our @Animal entity is required and in the “promote” input line, we should type “Please tell me which animal you like”.
This will cover the cases where the user didn’t name an animal, it will be clear to her that we need this entity.

Finally, in the ‘Text Response’ section we are filling our most amazing jokes. You can take few ideas from the image below.
Please note that we are using the $Animal value in our response in order to create a joke that is based on the animal that the user asked.

After you fill all your amazing jokes, don’t forget to click on the ‘save’ button on the top-right corner of the screen.

Build the “Quit” intent

A good design principle is to allow our user to end the conversation. you will need to have it anyway in order to pass the review process.

You should click again on ‘Create Intent’ button. Than, start typing few sentences that will end the conversation.

For example, “bye bye” or “bye animal joker”.
On the bottom you can see how this intent should look like.

Last, but not least, you need to check the ‘end conversation’ checkbox so that it will know to really end the conversation at this point.


We are almost done. It’s testing time!

Click on ‘Integrations’ in the right side menu. This will open the API.AI agent page with all the options to integrate it with other services (e.g. chat apps, twitter etc’).

Click on “Actions on Google” box under “One-click integration”.
This will enable you to test your work as it will be running on the Google Assistant – on Google Home or phones but inside our web simulator.

Once you click on ‘Actions on Google’ you will see this dialog:

That was it. Congrats!

Happy Friday!




Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s