Hello. I am Paul Kinlan.

I lead the Chrome and the Open Web Developer Relations team at Google. Exploring the intersection of modern web design and future-facing technologies.

2 min read

Barcode detection in a Web Worker using Comlink

I'm a big fan of QRCodes, they are very simple and neat way to exchange data between the real world and the digital world. For a few years now I've had a little side project called QRSnapper — well it's had a few names, but this is the one I've settled on — that uses the getUserMedia API to take live data from the user's camera so that it can scan for QR Codes in near real time.

The goal of the app was to maintain 60fps in the UI and near instant detection of the QR Code, this meant that I had to put the detection code in to a Web Worker (pretty standard stuff). In this post I just wanted to quickly share how I used comlink to massively simplify the logic in the Worker.

qrclient.js

import * as Comlink from './comlink.js';

const proxy = Comlink.proxy(new Worker('/scripts/qrworker.js')); 

export const decode = async function (context) {
  try {
    let canvas = context.canvas;
    let width = canvas.width;
    let height = canvas.height;
    let imageData = context.getImageData(0, 0, width, height);
    return await proxy.detectUrl(width, height, imageData);
  } catch (err) {
    console.log(err);
  }
};

qrworker.js (web worker)

import * as Comlink from './comlink.js';
import {qrcode} from './qrcode.js';

// Use the native API's
let nativeDetector = async (width, height, imageData) => {
  try {
    let barcodeDetector = new BarcodeDetector();
    let barcodes = await barcodeDetector.detect(imageData);
    // return the first barcode.
    if (barcodes.length > 0) {
      return barcodes[0].rawValue;
    }
  } catch(err) {
    detector = workerDetector;
  }
};

// Use the polyfil
let workerDetector = async (width, height, imageData) => {
  try {
    return qrcode.decode(width, height, imageData);
  } catch (err) {
    // the library throws an excpetion when there are no qrcodes.
    return;
  }
}

let detectUrl = async (width, height, imageData) => {
  return detector(width, height, imageData);
};

let detector = ('BarcodeDetector' in self) ? nativeDetector : workerDetector;
// Expose the API to the client pages.
Comlink.expose({detectUrl}, self);

I really love Comlink, I think it is a game changer of a library especially when it comes to creating idiomatic JavaScript that works across threads. Finally a neat thing here, is that the native Barcode detection API can be run inside a worker so all the logic is encapsulated away from the UI.

Read full post.

Stay in the loop.

I'm trialing a newsletter. Join for monthly insights into web dev, Chrome, and the open web.

alternate_email

Get in touch

Open to chat about Chrome or Web development.

Book a consultation
2 min read

Running FFMPEG with WASM in a Web Worker

I love FFMPEG.js, it's a neat tool that is compiled with asm.js`and it let's me build JS web apps that can quickly edit videos. FFMPEG.js also works with web workers so that you can encode videos without blocking the main thread.

I also love Comlink. Comlink let's me easily interact with web workers by exposing functions and classes without having to deal with a complex postMessage state machine.

I recently got to combine the two together. I was experimenting getting FFMPEG exported to Web Assembly (it works - yay) and I wanted to clean up all of the postMessage work in the current FFMPEG.js project. Below is what the code now looks like - I think it's pretty neat. We have one worker that imports ffmpeg.js and comlink and it simply exposes the ffmpeg interface, and then we have the webpage that loads the worker and then uses comlink to create a proxy to the ffmpeg API.

Neat.

worker.js

importScripts('https://cdn.jsdelivr.net/npm/comlinkjs@3.0.2/umd/comlink.js');
importScripts('../ffmpeg-webm.js'); 
Comlink.expose(ffmpegjs, self);

client.html

let ffmpegjs = await Comlink.proxy(worker);
let result = await ffmpegjs({
   arguments: ['-y','-i', file.name, 'output.webm'],
   MEMFS: [{name: file.name, data: data}],
   stdin: Comlink.proxyValue(() => {}),
   onfilesready: Comlink.proxyValue((e) => {
     let data = e.MEMFS[0].data;
     output.src = URL.createObjectURL(new Blob([data]))
     console.log('ready', e)
   }),
   print: Comlink.proxyValue(function(data) { console.log(data); stdout += data + "\n"; }),
   printErr: Comlink.proxyValue(function(data) { console.log('error', data); stderr += data + "\n"; }),
   postRun: Comlink.proxyValue(function(result) { console.log('DONE', result); }),
   onExit: Comlink.proxyValue(function(code) {
     console.log("Process exited with code " + code);
     console.log(stdout);
   }),
});

I really like how Comlink, Workers and WASM compiled modules can play together. I get idiomatic JavaScript that interacts with the WASM module directly and it runs off the main thread.

Read full post.

4 min read

Translating a blog using Google Cloud Translate and Hugo

I recently returned from a trip to India to attend the Google4India event (report soon) and to meet with a lot of businesses and developers. One of the most interesting changes discussed was the push for more content in the language of the users in the country, and it was particularly apparent across all of Google's products which ranged from making it easier to search in the users language, to find content, and also to read it back to users in either text or voice form.

The entire trip got me thinking. My blog is built with Hugo. Hugo now supports content in written in multiple languages. Hugo is entirely static, so creating new content is matter of just making a new file and letting the build system do it's magic. So maybe I can build something that will make my content more available to more people by running my static content through a translation tool because human translation of content is very expensive.

A couple of hours before my flight back to the UK I created a little script that will take my markdown files and run them through the Google Cloud Translate to create a quick translation of the page that I can then quickly host. The entire solution is presented below. It's a relatively basic processor, it ignores the Hugo preamble it ignores 'code' and it ignores pull quotes - my assumption was to that these are always meant to be left as the way they were written.

Note: It looks like our learning software for translations uses so it's important to mark up your page so the learning tools don't use Google Translated content as input to it's algorithms.

// Imports the Google Cloud client library
const Translate = require('@google-cloud/translate');
const program = require('commander');
const fs = require('fs');
const path = require('path');

program
  .version('0.1.0')
  .option('-s, --source [path]', 'Add in the source file.')
  .option('-t, --target [lang]', 'Add target language.')
  .parse(process.argv);

// Creates a client
const translate = new Translate({
  projectId: 'html5rocks-hrd'
});

const options = {
  to:  program.target,
};

async function translateLines(text) {
  if(text === ' ') return ' ';
  const output = [];
  let results = await translate.translate(text, options);

  let translations = results[0];
  translations = Array.isArray(translations)
    ? translations
    : [translations];

  translations.forEach((translation, i) => {
    output.push(translation)
  });

  return output.join('\n');
};

// Translates the text into the target language. "text" can be a string for
// translating a single piece of text, or an array of strings for translating
// multiple texts.
(async function (filePath, target) {

  const text = fs.readFileSync(filePath, 'utf8');

  const lines = text.split('\n');
  let translateBlock = [];
  const output = [];

  let inHeader = false;
  let inCode = false;
  let inQuote = false;
  for (const line of lines) {
    // Don't translate preampble
    if (line.startsWith('---') && inHeader) { inHeader = false; output.push(line); continue; }
    if (line.startsWith('---')) { inHeader = true; output.push(line); continue; }
    if (inHeader) { output.push(line); continue; }

    // Don't translate code
    if (line.startsWith('```') && inCode) { inCode = false; output.push(line); continue; }
    if (line.startsWith('```')) { inCode = true; output.push(await translateLines(translateBlock.join(' '))); translateBlock = []; output.push(line); continue; }
    if (inCode) { output.push(line); continue; }

    // Dont translate quotes
    if (inQuote && line.startsWith('>') === false) { inQuote = false; }
    if (line.startsWith('>')) { inQuote = true; output.push(await translateLines(translateBlock.join(' '))); translateBlock = []; output.push(line); }
    if (inQuote) { output.push(line); continue; }

    if (line.charAt(0) === '\n' || line.length === 0) { output.push(await translateLines(translateBlock.join(' '))); output.push(line); translateBlock = []; continue;} 

    translateBlock.push(line);
  }

  if(translateBlock.length > 0) output.push(await translateLines(translateBlock.join(' ')))

  const result = output.join('\n');
  const newFileName = path.parse(filePath);
  fs.writeFileSync(`content/${newFileName.name}.${target}${newFileName.ext}`, result);

})(program.source, program.target);

Overall, I am very happy with the process. I understand that the machine translation is not perfect but my thinking is that I can increase the reach of my content to people who might be searching in their own languages and not in English I can increase the discovery surface area of my content and hopefully help more people.

It will take a while to see if this actually helps people, so I will report back when I have more data.... Now to run my script across more of my site :)

1 min read

Apple - Web apps - All Categories

Remember when Web Apps were a recommended way to use apps on the iPhone?

What are web apps? Learn what they are and how to use them.

Read full post.

In about 2013 Apple started to redirect the /webapps/ top-level directory to /iphone/

The thing is, the directory was actually pretty good, a lot of the apps in there still work today. However looking at the AppStore it solved a lot more problems that developers had: Better discovery and search specifically because the AppStore was directly on the device. The AppStore was also starting to introduce that removed friction from users and developers specifically with regards to payments.

1 min read

Gears API

I'm writing up a blog post about the early Mobile Web API's and Alex Russell reminded me of Google Gears

Gears modules include:

  • LocalServer Cache and serve application resources (HTML, JavaScript, images, etc.) locally
  • Database Store data locally in a fully-searchable relational database
  • WorkerPool Make your web applications more responsive by performing resource-intensive operations asynchronously

Read full post.

I think it is interesting to see that AppCache and WebSQL, Geolocation and WebWorkers came out of the ideas in Google Gears and it's only the latter two that really survived. WebSQL was never broadly supported, and was replaced by IndexedDB; and AppCache replaced by ServiceWorker

2 min read

RSS Feed to Google Chat Webhook using Cloud Functions for Firebase and Superfeedr

We use Google Chat internally a lot to communicate across our team - it's kinda like our slack; We also create a lot of content that is accessible via RSS feeds, we even have a team feed that you can all view. It wasn't until recently that I found out that it was pretty easy to create a simple post-only bot via WebHooks and that gave me the idea, I can create a simple service that polls RSS feeds and then sends them to our webhook that can post directly in to our team chat.

It was pretty simple in the end, and I've included all the code below. I used Firebase functions - I suspect that this is just as easy on other Function-as-a-service sites - and Superfeedr. Superfeedr is a service that can listen to Pubsubhubbub pings (now WebSub) and it will also poll RSS feeds that don't have Pubsub set up. Then when it finds a feed it will ping a configured URL (in my case my Cloud Function in Firebase) with an XML or JSON representation of the newly found feed data - all you have to do is parse the data and do something with it.

const functions = require('firebase-functions');
const express = require('express');
const cors = require('cors');
const fetch = require('node-fetch');
const app = express();

// Automatically allow cross-origin requests
app.use(cors({ origin: true }));

app.post('/', (req, res) => {
  const { webhook_url } = req.query;
  const { body } = req;
  if (body.items === undefined || body.items.length === 0) {
    res.send('');
    return;
  }

  const item = body.items[0];
  const actor = (item.actor && item.actor.displayName) ? item.actor.displayName : body.title;

  fetch(webhook_url, {
    method: 'POST',
    headers: {
      "Content-Type": "application/json; charset=utf-8",
    },
    body: JSON.stringify({
      "text": `*${actor}* published <${item.permalinkUrl}|${item.title}>. Please consider <https://twitter.com/intent/tweet?url=${encodeURIComponent(body.items[0].permalinkUrl)}&text=${encodeURIComponent(body.items[0].title)}|Sharing it>.`
    })  
  }).then(() => {
    return res.send('ok');
  }).catch(() => {
    return res.send('error')
  });
})
// Expose Express API as a single Cloud Function:
exports.publish = functions.https.onRequest(app);

Read full post.

I was surprised and delighted about how easy it was to set up.

3 min read

'Moving to a Chromebook' by Rumyra's Blog

Ruth John moved to Chrome OS (temporarily):

The first thing, and possibly the thing with the least amount of up to date information out there, was enabling Crostini. This runs Linux in a container on the Chromebook, something you pretty much want straight away after spending 15 minutes on it.

I have the most recent Pixel, the 256GB version. Here's what you do.

  • Go to settings.
  • Click on the hamburger menu (top left) - right at the bottom it says 'About Chrome OS'
  • Open this and there's an option to put your machine into dev mode
  • It'll restart and you'll be in dev mode - this is much like running Canary over Chrome and possibly turning on a couple of flags. It may crash, but what the hell you'll have Linux capabilities ��
  • Now you can go back into Settings and in regular settings there's a 'Linux apps' option. Turn this on. It'll install Linux. Once this is complete you'll have a terminal open for you. Perfect

Read full post.

Ruth has a great write-up of moving to Chrome OS because her main machine broke.

I moved to Chrome OS full-time 4 months ago (before Google I/O) and only moved to the Mac because I broke my PixelBook (now fixed).

For me it's one of the best web development machines out there today. It's the only device that I can test 'true mobile' on - you can install Chrome on Mobile on it, Firefox Mobile, Samsung Browser, Brave etc via the ARC platform. Crostini is also a game changer for Chrome OS as it brings a lot of the Linux App ecosystem to Chrome OS and it really starts to fill a huge app-gap for me on Chrome OS; I've got Firefox, vim, git, VS Code, Node, npm, all my build tools, GIMP and Inkscape... That's not to say it has been perfect, Crostini could be faster, it's not GPU accelerated yet and it could be more integrated with Filemanager etc, and finally the PixelBook really needs more physical ports - I can attach two 4k screens to it, but I can't charge at the same time.

I think Ruth's wrap up is also quite accurate, the PixelBook is an expensive machine, but I am very very excited to see this coming to more and more devices (especially those at vastly lower price points.)

Would I pay full price for it? I'm not sure I would pay full price for anything on the market right now. Point me in the direction of a system that will run my graphics software and makes a good dev machine (with minimal setup) and lasts more than 18 months, point me in the direction of a worthy investment and I will pay the money.

Yup.

8 min read

PWA: Progressive Web All-the-things

This blog post discusses the evolution of Progressive Web Apps (PWAs) since their inception in 2015. While PWAs offer numerous benefits like offline functionality, push notifications, and installability, the author observes that adoption hasn't been universal. Many developers and businesses misunderstand PWAs, sometimes treating them as separate products or focusing on single features like push notifications. The post argues that the focus should shift from "apps" to user experience. It proposes a set of principles for modern web experiences: discoverable, safe, fast, smooth, reliable, and meaningful. These principles aim to guide developers towards building better web experiences that naturally embody the core values of PWAs, benefiting both users and businesses.