JavaScript, webdev

YouTube Analytics Dashboard With Apps Script

player_apisThere are cases where you wish to collect statistics on your youtube videos or channel. There are few options to do it with YouTube API. As the YouTube’s API supports many languages you can choose the one that will work for your environment. In many of these options, you will need to develop a server side that will fetch the data and a front-end to present it and give the users option to query it. If you wish to dive deeper (e.g. specific metric on channel performance and videos statistics), you will need to work with YouTube Analytics API.

In this post, we will see a simple example to create a dashboard that will be updated on a daily basis. Since we wish to save ourselves from building (and maintaining!) a server side and a web app to access it, we will use the power of Google Apps Script (GAS) and Google sheets.

The Dashboard

Here you can find the “Dashboard example sheet”.
Feel free to copy it and work on your own version that is not in ‘read-only’ mode. Continue reading

Advertisement
Standard
cloud

Upload Data To BigQuery With A Smile

Big QueryWhat is BigQuery?

In our world of ‘Big Data’ it can be time consuming and expensive to query massive datasets without the right infrastructure. Google BigQuery solves this problem by enabling super-fast, SQL-like queries against append-only tables, using the processing power of Google’s infrastructure.

What You need to do?

  1. Move your data into BigQuery – This is what we will do in this post.
  2. Let Google BigQuery handle the hard work.
  3. Query your big data with a smile in this cost/effective way.

How to upload data to Big Query?

There are two main approaches: stream you data or upload it directly from Google cloud storage. Let’s have a look at the steps to leverage Google cloud storage in order to upload data into BigQuery.

The main steps you need to follow:

  1. You will need to prepare your data. In this stage, you need to analyze and think what will be the best format (both JSON and CSV are supported).
  2. In our example, we will show you how to work with CSV files and even better, we will upload them to Google Cloud Storage and later with a BigQuery job we will make sure our data is being pulled automatically into BigQuery.
  3. Run a ‘sanity’ check to see that the new data is in good shape (optional step).

Tips

  • Upload your the data to a project with a good name (The default project names are not too clear in most cases).
  • Consider breaking your data (e.g monthly tables instead of a unique big one) because it will make life easier in the future to update, query and maintain the data source.
  • Have an example dataset with data that reflect the popular cases. This could be great to give developer an option to ‘play’ with the data and see its value.
  • Think on some good and bold example. A few sample queries are crucial to get people started on a dataset.

Continue reading

Standard
cloud

Google I/O 2014 – What’s interesting in the cloud?

google-io-logo 2014-06-24 14.30.42This year there are many interesting sessions on Google Cloud. Here are some that I think you might enjoy watching the live stream (or jump into the room if you are at Moscone center). There are many more great sessions, so the best advice is to check the schedule and plan in advance which one you wish to follow live. You can always catch the other sessions later on developers.google.com channel. Continue reading

Standard
cloud, webdev

What’s New On Google Cloud Platform

This year at #DevConTLV the main theme was around cloud and databases. In my talk I did my best to emphasis, that in the past 15 years, Google has been building out the world’s fastest, most powerful, highest quality cloud infrastructure on the planet and it is opening it so external developers could enjoy it. There are many services like: App Engine, BigQuery and VMs on Compute engine that all come with the same idea. You (=the developer) should focus on what you good at and not by reinventing the wheel again (and again) by trying to find the secret in memcache optimizations. It’s my 4th year in this event and I can say that it is (without doubt) one of the best developer conferences in Tel Aviv. So if you are around next time, please try to join us. It’s great fun to talk with so many talented developers and the talks that I’ve been to, where very good and informative. Continue reading

Standard
cloud

Spark Cluster on Google Compute Engine

gce+sparkWhat is Spark and Why?

Apache Spark is an open source cluster computing system that aims to make data analytics fast — both fast to run and fast to write. To run programs faster, Spark offers a general execution model that can optimize arbitrary operator graphs, and supports in-memory computing, which lets it query data faster than disk-based engines like Hadoop. In the past, I’ve wrote an intro on how to install Spark on GCE and since then, I wanted to do a follow up on the topic but with more real world example of installing a cluster. Luckily to me, a reader of the blog did the work! So after I got his approval, I wanted to share with you his script. Continue reading

Standard
Chrome, JavaScript

Monitor Your Site With Apps Script

Network alertIn this short tutorial we will see how to built an application that can be used as a (simple) Nagios but, without the need to set up a server and maintain a lot of code. It’s not a replacement to a ‘full’ monitoring systems that give you rich set of abilities but more ‘an idea’ to show you the power of Google cloud with Apps script. The main three components of our system will be:

  1. Google Spreadsheet – We will use two sheets. One to allow us to set configuration data like: urls to monitor, email to get notifications etc’. The other will be our ‘log’. We will write to it all the alerts so we could analyze them later.
  2. Google Form – This will be a quick and easy way to acknowledge that we are aware for the situation and we wish to mute the emails.
  3. Apps Script – All the logic for the checks, alerts and everything in between will be done with a short script with one trigger.

For the first two components, we will use a regular google sheet that looks similar to this: Continue reading

Standard
cloud

How To Set A Server To Server VPN on Google Compute Engine

There are many cases where you wish to migrate data from one location to another. In most cases, you will wish to do it over a secure channel. In this tutorial we will see what are the main steps in order to set a VPN (in this case: StrongSwan) on Google Compute Engine so you will have a server to server VPN solution between your on own server(s) located in the basement and Compute Engine.
In this tutorial we will setup a VPN connection from your own datacenter to Compute Engine. First, let’s look at the big picture. We wish to have one gateway in our datacenter and another on Compute Engine. These gateways, will be responsible to connect and secure our channel so we could transfer our data over it in a secure way.

High Level Network diagram

VPN on GCE network macro view

Main steps

Continue reading

Standard
Chrome, cloud, webdev

What’s New In Google Developers World

Here are my slides from a talk I gave last week on what’s new and available for you as a developer in Google’s world. This talk gives a high level overview on the main APIs. If you wish to dive deeper on some of them, you will find many links in the slides.

Continue reading

Standard