Tales of a Developer Advocate

Bootstraping Your Own Mobile Web Testing Lab for Android Part 1

What do you do if you want to quickly test your sites on mutliple versions of Android at the same time?

There are couple of solutions, such as Adobe Edge Inspect, but what if you want to create and manage your own in your own workflow? I haven’t seen single solution yet.

Here is a simple shell script that I created that starts to get you on your way to your own testing lab. It connects to every single Android device that is attached to your machine by using adb (Android Debugging tool).

The process is as follows (and please suggest better ways):

  • Kill any existing adb servers
  • Get a list of attached devices
  • For each device set up port forwarding (required for Chrome Dev tools)
  • Fire Android intent to open the browser (with a url if one is on the command line)

It’s not a complete solution, but it is a nice way to start. My ultimate goal is to have a Raspberry PI hosting adb and all the devices that I can attach to it with an attached proxy that will route all my devtools requests to the correct device.

How do you test across mutliple devices?

Traffic-lights With CSS

Just before Christmas I wanted to build a site that helped developers understand the impact that using Web Platform features would have on there potential reach. For example, if I used WebGL what is my target reach and what additional features can I use without impacting my potential audience figures.

I launched iwanttouse.com. Sweet. Anyway, that is not the point of this post. One of the features of this site is.

One of the core features I wanted to implement was a traffic light system that graded the features like a traffic light. Green = Good, Red = Bad, Amber = Be Warned.

My original implementation was just using some simple CSS classes.

.good {
  color: green;
}

.ok {
  color: amber;
}

.bad {
  color: red;
}

I knew this was bad, but using RGB I couldn’t work out sanely how to grade between 255,0,0, 255,126,0 and 0,255,0 without having logic in my code that looked like:

if (support < 25%) then .bad
if (support >25% and support < 75%) then .ok
if (support > 75%) then .good

added to that, I wanted the color to be more red than amber if it was at 35%.

Anyway, after a little chat with the awesome Paul Lewis, he mentioned that HSL (Hue, Saturation, Light) color scheme would be good for this problem because naturally the Hue value (0-359) rotates from Red, through Amber to Green (0 = Red, 45 = Amber, 90 = Green)

Red = color: hsla(0, 50%, 50%, 1) Amber = color: hsla(45, 50%, 50%, 1) Green = color: hsla(90, 50%, 50%, 1)

It is then simple given a value for percentage support to map that to 0-90 range and produce your traffic light colors.

element.styles.color = "hsla(" + ((percentage / 100) * 90) + ", 50%, 50%)";

Dear AppCache We Need to Talk.

It’s not me. It’s you!

Ok. thats a little harsh. Without AppCache we wouldn’t even be able to think about building Offline enabled web apps. But it seriously does have some issues, one of which I am blogging about today has been not discussed in depth before (as far as I can tell).

Imagine you building an offline enabled app that integrates with registerProtocolHandler or registerContentHandler. These two API’s use query parameter substitution to pass data from the client page to the service page (http://mailfoo.com/createmail.php?t=paul.kinlan@gmail.com&s=Test Message for example opens up your mail client at createmail.php when you click on a mailto link). This works well for purely online apps, but as we will see when used with apps built with AppCache you won’t get the experience you are expecting, at worst you could DoS your site.

Your createmail.php will likely render a manifest=”cache.manifest” attribute in your html element, which when visited is cached offline and works as expected. The problem lies in the fact that AppCache treats the entire URI (including query strings) as an identifier to the page, therefore if you have a unique query string each of these pages are downloaded and added to the AppCache group.

Ok, cool, but what does this mean? Well it means that http://mailfoo.com/createmail.php?t=paul.kinlan@gmail.com is cached separeately to http://mailfoo.com/createmail.php?t=paul.kinlan@hotmail.com, all the resources will be quickly fetched as the manifest has already got hold of the data and they are already cached in AppCache.

Whilst you are online, this is all gravy. It works beautifully. However now go offline and create an email to paul.kinlan@gmail.com (it works because it is cached), but try instead creating an email to paulkinlan@google.com(via a mailto link)… hmm, where did my app go? It is not there because it has not yet been cached for offline use. Big problem.

You thought that was bad? Now for the kicker.

You happily make and send 100’s of emails over a couple of weeksr. Whilst you are online, all works well. The developer of the app decides to upload a new version of their code so updates their app cache. What happens? Every single unique url (so every email you might have sent via a mailto) is requested and downloaded again (even if it is not in the manifest, it is in the app cache group). If I have sent 100 unique emails via mailto, then every single page with that query sting is fetched quickly by the browser… Seems like a good way to add a lot of load to your machine.

It is pretty hard to change a specifictation in flight so we need to start documenting patterns for building offline apps than we do today because AppCache is not easy to use, and there are lots of gotchas like this that make developers not even bother using the API.

The New iPad Is the Perfect Name

I was thinking about this the other day. I remember being in the Apple store in Liverpool about 2 months ago and I distinctly remember hearing the following: "Do you have any of the new iPads in?" several times over the space of about 30 minutes (I was playing with MacBook Airs).

Come to think of it there were several other products people asked for as "the new X".

It is pretty cool. No product name obsolescence. You go in to a store you ask for the new product, they give it to you.

You either have the new one or the old one.

I predict the iPad series will only ever now be known as iPad and The New iPad. I also believe they will do it for more products the next being The New iPhone.

I Am Willing to Pay Someone to Move My Blog From Posterous.

Posterous has been great for me, but now it is time to move it. I don’t have the time to do this so I am very willing to pay someone a modest fee for managing the move of my blog to a platform that meets the following requirements: Mandatory Requirements: - not a hosted solution like Blogger or Tumblr - static output (no database) - can write posts in markdown, text or html - all content can be managed through git. - URL structure is kept exactly the same as now - will keep all the posts that I imported into posterous from blogger. - full control over templates Nice to haves: - preferably not a ruby based system Not bothered about: - any authoring or management front end. - auto posting networks - existing comments or future commenting systems

We Need to Kill Off the localStorage API

It is a failure of the web, browser vendors and developers that we are in this situation, but we need to stop advocating for and building examples that use the LocalStorage API's, it is simply not a scalable API and the more we build for it the harder it will be for us to ween ourselves off it.  

LocalStorage has poor querying capabilities, terrible performance, small storage in many browsers, crazily inconsistent eventing and a nasty habit of locking.  It's saving grace is simple semantics and "browser support".

Client-side and offline web-apps are not a reality with localStorage, and if we keep pushing it, we are never going to be in a situation where we have a compelling offline or client-side story.

We need to bite the bullet, move on and start building compelling apps, examples and demos build around IndexedDB, that is our only future (I am still aggrieved that the web dropped WebSQL, it was simple and familiar) and we need to do this by stopping to support localStorage. Period.

To this effect, I have taken the BackboneJS example in TodoMVC (by Addy Osmani) and added IndexedDB support using the Backbone JS adapter by Julien Genestoux. I quickly removed the LocalStorage interface, and in a couple of lines of config change I had IndexedDB support in.

On Vendor Prefixes

There is a lot of chatter about Mozilla considering implementing some webkit specific prefixes - I encourage everyone to read ”Platform/Layout/CSS compatibility

My personal take on this is “great” and I am glad they have the guts to start the conversation, but in my eyes it should only happen IFF the vendor is going to choose to stop supporting their existing prefix and start supporting the other parties prefix as what they agree to be the way they want the standard to move. 

I know a lot of people who targeted mobile saw that the audience they needed to target were using almost exclusively a WebKit based browser and thus chose that as the platform to target.  If the mobile device you want to target or your customers want you to target is pretty much a single choice then web developers naturally gravitate to the browser that is most popular and for a long time in the case of iPhone etc, WebKit was (up until recently) your only web platform you had to develop for.

Remy Sharp’s blog post is a good summary of the situation, but I don’t agree with most of the proposed browser solutions.  

To Quote:
Browsers need to:
  • Fucking drop experimental prefixes. It’s unacceptable and a disservice to the developers working with your browser. You need to give timelines to dropping these things.
  • Non-production ready browsers should support experimental prefixes, production ready releases should not. If it’s Chrome 16 - the stable version - experimental support should not be baked in. The properties should be full available without the prefix.
On dropping vendor prefixes, yes but only if they commit to move to another one.  The whole gradient syntax winded me up no end - if at this point WebKit had chosen to name it consistently with the -moz implementation and prefixed it so then we could have started to completely drop the old syntax that IMO would have been better.  At the same time though we should be educating developers about the tooling available to make the vendor prefixing become a non-story (in most cases).

I had a little chuckle about “production ready browsers” - I don’t think this is sensible or feasible given the update cycles of browser and that developers want what they have chosen to implement to look ace in the browsers that they have chosen to make it look awesome in.  If you have platform features that no other browser is going to support either now in the near future then we need to have a way to not have them in the base “namespace” - putting them in there is going to cause more problems.

Anyway, these are my thoughts and can be and sometimes are hogwash - my motto is “don’t hate, advocate” - and as Remy says “As developers we need to better educate.” to which I whole heartedly agree.

Getting Your App to Support Web Intents on Chrome

Chrome just got Web Intents support in Dev and Canary builds (18 onwards).  This is a huge milestone and I am very excited by this first step along the path of building a more connected web of apps.

A lot of developers have asked me how to get started as it seems some of the demos on http://demos.webintents.org don’t register correctly.  I have a good answer for that - in short: Chrome doesn’t yet detect the intent tag, instead applications currently can only register their support for an action such as ”share” via the Chrome apps manifest.

The longer version is a little more complex:
  1. Consensus over the introduction of a new tag in to the spec has not yet been reached.
  2. Working with members of the DAP in the intents task force, it is clear that discovery of applications and services shouldn’t only take place by detecting a tag on a web page.  What happens if the service you want to “Share” a video too is a TV connected to your local network? Or an external native application wants to be able to support a “Save” action.  To enable this important use case the User Agent should be able to determine the services it presents to users, and this is why this is allowed in the specification (3rd paragraph).
Bringing this closer to home, because the discovery and presentation of an app’s capabilities can be managed by the User Agent, and Chrome has the concept of extensions and installed apps we can quickly enable the intents feature by letting developers declare their support for actions in the manifest.

So what does the declaration in the Chrome apps/extension system look like?  It is pretty easy, it is an entry into the manifest called “intents”.  It looks like:

{
  “name”: “Share to Gmail™”,
  “version”: “0.0.0.2”,
  “icons” : {
    “16” : “favicon.ico”
  },
  “intents” : {
    ”http://webintents.org/share” : {
       “type” : [“text/uri-list”],
       “title” : “Share to Gmail”,
       “path” : “/launch.html”
    }
  }
}

It is that simple.  The intent section includes a dictionary of supported action (http://webintents.org/share) and in each action object there is an array of data types that the application or extension can handle, the friendly name to appear in the picker and a path to what should be opened when the user selects your app.  The client-side code remains exactly the same as it would in a normal web app.

In the long term we want applications to be able to declare their capabilities and services directly through their html and this will be done with the Intent tag.  However whilst the standardisation work continues we want to make sure that developers today can start building apps that can take advantage of the Web Intent system.

A lot more examples can be found on the Web Intents Github repository.

Expect a lot more posts about how to build applications that love each other with Web Intents.

Two Years and Counting at Google

As of February 1st I have been at Google for two years! Yay!  It has been an amazing time and I am truly honored to work for such a company.

A year ago (+1 week) I wrote about my first year experiences, it was pretty crazy in my first year, not only did the scale of the work hit me overwhelm me, but so did the sheer number of excellent engineers and colleagues.  My overall feeling was one of awe.

My second year has taken that to the next level again!

One of the highlights of the year was that I got to speak at Google IO - Mobile Web App development: Zero to Hero, where myself and Mike Mahemoff (with the help of Eric Bidelman and Boris Smus) built an app that worked across 5 different form factors using one code base - the awesome thing about this is that we built LeviRoutes (a url routing framework) and FormfactorJS (a form factor detection library).

I got to travel a lot too - I am trying to count all the countries that I have been too; it is a lot - USA, France, Spain, Portugal, Romania, Belgium, Germany, Poland, Netherlands, Czech Republic.  Whilst I was traveling I did a lot of presentations which is a really good way to meet people that I wouldn't normally get the chance to meet. I also managed to get completely misquoted on TechCrunch too - I say misquoted, I didn't even say the words that they attributed to me.

The biggest thing for me was that Web Intents became a real living and breathing project and first native implementations have landed in Chrome, it has changed a lot since my first announcement late in 2010.  This is a huge change for users of the web and I am immensely proud to be working on the project and with some amazingly talented engineers (James Hawkins, Greg Billock, Rachel Blum, Ben Smith and many others) and helping to define the specifications.  We have big goals for Web Intents so stay tuned.

One of my resolutions last year was to be in Liverpool a little more and I managed to do that, it turns out that I unwittingly pre-announced another project (that I honestly had no idea was about to launch) I am glad it happened though, there are a huge swath of developers in the UK that are outside London and I don't think anyone reaches them properly and this gives me a chance to work with them.

So.  What is happening in the year to come?  It is hard to say, there are a lot of cool things happening, but I know two things for sure: we will keep working on making sure users' experience of the web is improved through technologies such as Web Intents and we will keep working with developers to build awesome sites and applications on the web and prove that the web is our future.

On to my third year! 

Web Intents: A Fresh Look

We have a huge problem on the web today. If I built an image gallery application and I wanted to let users edit an image so that they can remove red-eye from a photo I either have to build an application that edits the images, or integrate with a 3rd party solution. Doing this is hard and stops you from building an awesome image gallery; and what happens if the user has a favorite service that they already use to remove red-eye? Simple, you have a frustrated user.

We have a solution!

In December 2010 I announced a project called Web Intents whose goal was to allow developers to build applications and services that could work with each other, but not need to explicitly know about each other – the concept has heavily inspired by the Intent system in Android, although the API bore no resemblance. It would allow you to build applications using just the functionality your cared about, and then delegate the other functionality to the users preferred choice of service.

After some conversations, I moved to try and support Web Introducer as a specification over Web Intents. For one reason or another this didn’t quite work out so I decided to plug away at revising the WebIntents work that I started back in November.

It turns out there is a lot of interest internally with the idea of Web Intents and how it can work in modern browsers. We set up a small crack team and after a flurry of work, speccing and prototyping how we think it might look we have put a prototype API on to Github. Have a play, it is really easy to get started.

So what changed?

A lot as it happens. It is not the same as the initial project that I experimented with, although the goals are the same. We have an objective to make the developer experience of the API so painless that most developers can start integrating with applications in 5 minute of reading the spec – in fact we want it so that most developers can just copy and past examples and it will work with their service. We have tried to drastically reduce the API surface and make it so there is literally only one or two lines of code you need to start an activity.

Service registration has been made even easier that my initial project through the use of a new tag, for example:

<intent
  action="http://webintents.org/share"
  type="text/uri-list"
  href="share.html"
/>

This small tag, that is included in the head of your application will signal to the browser the intention to handle a “share” action for a selection of URI’s (think “share this page”), and will register it in the system so that the user can choose it when a client application wants to provide “share” functionality in their app.

When the service is chosen by the user, and the service is loaded the intent data is passed to the open application and is available on the window.intent object.

For clients to initiate an Activity it is easy too. Simply declare an intent and start the Activity as follows:

var intent = new Intent();
intent.action = "http://webintents.org/share";
intent.type = "text/uri-list";
intent.data = "http://paul.kinlan.me";

window.navigator.startActivity(intent);

The system will take care of the service resolution for the action and compatible data formats and give the user the choice of using their favorite application to handle the “share” intent.

I have only just touched the surface of what you can do with the API. There are a lot of things that you can do with the API over and above what I have described in this 5 minute overview. A selection of examples can be found at http://examples.webintents.org/ where we show you how to build applications that solve some common use-cases. I particularly like the cloud kitten service provided by the “pick” example.

We are working with Mozilla to define a common approach to solving the challenges that web integrators face today. We are interested in hearing your thoughts and we are still thrashing out the API so bits of it might change but the intent is still the same.

My closing thoughts are: “This project will fundamentally change and improve the way we build applications on the web today for our users.”