Hello.

I am Paul Kinlan.

A Developer Advocate for Chrome and the Open Web at Google.

Building a video editor on the web. Part 0.1 - Screencast

Paul Kinlan

You should be able to create and edit videos using just the web in the browser. It should be possible to provide a user-interface akin to Screenflow that lets you create an output video that combines multiple videos, images, and audio into one video that can be uploaded to services like YouTube. Following on from the my previous post that briefly describes the requirements of the video editor, in this post I just wanted to quickly show in a screencast how I built the web cam recorder, and also how how to build a screencast recorder :)

Read More

Paul Kinlan

Trying to make the web and developers better.

RSS Github Medium

894556 - Multiple video tracks in a MediaStream are not reflected on the videoTracks object on the video element

Paul Kinlan

The first issue I have found trying to build a video editor on the web.

I have multiple video streams (desktop and web cam) and I wanted to be able to toggle between the video streams on one video element so that I can quickly switch between the web cam and the desktop and not break the MediaRecorder.

It looks like you should be able to do it via toggling the selected property on the videoTracks object on the <video> element, but you can’t, the array of tracks contains only 1 element (the first video track on the MediaStream).

What steps will reproduce the problem? (1) Get two MediaStreams with video tracks (2) Add them to a new MediaStream and attach as srcObject on a videoElement (3) Check the videoElement.videoTracks object and see there is only one track

Demo at https://multiple-tracks-bug.glitch.me/

What is the expected result? I would expect videoElement.videoTracks to have two elements.

What happens instead? It only has the first videoTrack that was added to the MediaStream.

Read full post.

Repro case.

window.onload = () => {
  if('getDisplayMedia' in navigator) warning.style.display = 'none';

  let blobs;
  let blob;
  let rec;
  let stream;
  let webcamStream;
  let desktopStream;

  captureBtn.onclick = async () => {

       
    desktopStream = await navigator.getDisplayMedia({video:true});
    webcamStream = await navigator.mediaDevices.getUserMedia({video: { height: 1080, width: 1920 }, audio: true});
    
    // Always 
    let tracks = [...desktopStream.getTracks(), ... webcamStream.getTracks()]
    console.log('Tracks to add to stream', tracks);
    stream = new MediaStream(tracks);
    
    console.log('Tracks on stream', stream.getTracks());
    
    videoElement.srcObject = stream;
    
    console.log('Tracks on video element that has stream', videoElement.videoTracks)
    
    // I would expect the length to be 2 and not 1
  };

};

Building a video editor on the web. Part 0.

Paul Kinlan

You should be able to create and edit videos using just the web in the browser. It should be possible to provide a user-interface akin to Screenflow that lets you create an output video that combines multiple videos, images, and audio into one video that can be uploaded to services like YouTube. This post is really just a statement of intent. I am going to start the long process of working out what is and isn’t available on the platform and seeing how far we can get today.

Read More

Barcode detection in a Web Worker using Comlink

Paul Kinlan

I’m a big fan of QRCodes, they are very simple and neat way to exchange data between the real world and the digital world. For a few years now I’ve had a little side project called QRSnapper — well it’s had a few names, but this is the one I’ve settled on — that uses the getUserMedia API to take live data from the user’s camera so that it can scan for QR Codes in near real time.

The goal of the app was to maintain 60fps in the UI and near instant detection of the QR Code, this meant that I had to put the detection code in to a Web Worker (pretty standard stuff). In this post I just wanted to quickly share how I used comlink to massively simplify the logic in the Worker.

qrclient.js

import * as Comlink from './comlink.js';

const proxy = Comlink.proxy(new Worker('/scripts/qrworker.js')); 

export const decode = async function (context) {
  try {
    let canvas = context.canvas;
    let width = canvas.width;
    let height = canvas.height;
    let imageData = context.getImageData(0, 0, width, height);
    return await proxy.detectUrl(width, height, imageData);
  } catch (err) {
    console.log(err);
  }
};

qrworker.js (web worker)

import * as Comlink from './comlink.js';
import {qrcode} from './qrcode.js';

// Use the native API's
let nativeDetector = async (width, height, imageData) => {
  try {
    let barcodeDetector = new BarcodeDetector();
    let barcodes = await barcodeDetector.detect(imageData);
    // return the first barcode.
    if (barcodes.length > 0) {
      return barcodes[0].rawValue;
    }
  } catch(err) {
    detector = workerDetector;
  }
};

// Use the polyfil
let workerDetector = async (width, height, imageData) => {
  try {
    return qrcode.decode(width, height, imageData);
  } catch (err) {
    // the library throws an excpetion when there are no qrcodes.
    return;
  }
}

let detectUrl = async (width, height, imageData) => {
  return detector(width, height, imageData);
};

let detector = ('BarcodeDetector' in self) ? nativeDetector : workerDetector;
// Expose the API to the client pages.
Comlink.expose({detectUrl}, self);

I really love Comlink, I think it is a game changer of a library especially when it comes to creating idiomatic JavaScript that works across threads. Finally a neat thing here, is that the native Barcode detection API can be run inside a worker so all the logic is encapsulated away from the UI.

Read full post.

Running FFMPEG with WASM in a Web Worker

Paul Kinlan

I love FFMPEG.js, it’s a neat tool that is compiled with asm.js`and it let’s me build JS web apps that can quickly edit videos. FFMPEG.js also works with web workers so that you can encode videos without blocking the main thread.

I also love Comlink. Comlink let’s me easily interact with web workers by exposing functions and classes without having to deal with a complex postMessage state machine.

I recently got to combine the two together. I was experimenting getting FFMPEG exported to Web Assembly (it works - yay) and I wanted to clean up all of the postMessage work in the current FFMPEG.js project. Below is what the code now looks like - I think it’s pretty neat. We have one worker that imports ffmpeg.js and comlink and it simply exposes the ffmpeg interface, and then we have the webpage that loads the worker and then uses comlink to create a proxy to the ffmpeg API.

Neat.

worker.js

importScripts('https://cdn.jsdelivr.net/npm/comlinkjs@3.0.2/umd/comlink.js');
importScripts('../ffmpeg-webm.js'); 
Comlink.expose(ffmpegjs, self);

client.html

let ffmpegjs = await Comlink.proxy(worker);
let result = await ffmpegjs({
   arguments: ['-y','-i', file.name, 'output.webm'],
   MEMFS: [{name: file.name, data: data}],
   stdin: Comlink.proxyValue(() => {}),
   onfilesready: Comlink.proxyValue((e) => {
     let data = e.MEMFS[0].data;
     output.src = URL.createObjectURL(new Blob([data]))
     console.log('ready', e)
   }),
   print: Comlink.proxyValue(function(data) { console.log(data); stdout += data + "\n"; }),
   printErr: Comlink.proxyValue(function(data) { console.log('error', data); stderr += data + "\n"; }),
   postRun: Comlink.proxyValue(function(result) { console.log('DONE', result); }),
   onExit: Comlink.proxyValue(function(code) {
     console.log("Process exited with code " + code);
     console.log(stdout);
   }),
});

I really like how Comlink, Workers and WASM compiled modules can play together. I get idiomatic JavaScript that interacts with the WASM module directly and it runs off the main thread.

Read full post.

Translating a blog using Google Cloud Translate and Hugo

Paul Kinlan

I recently returned from a trip to India to attend the Google4India event (report soon) and to meet with a lot of businesses and developers. One of the most interesting changes discussed was the push for more content in the language of the users in the country, and it was particularly apparent across all of Google’s products which ranged from making it easier to search in the users language, to find content, and also to read it back to users in either text or voice form.

The entire trip got me thinking. My blog is built with Hugo. Hugo now supports content in written in multiple languages. Hugo is entirely static, so creating new content is matter of just making a new file and letting the build system do it’s magic. So maybe I can build something that will make my content more available to more people by running my static content through a translation tool because human translation of content is very expensive.

A couple of hours before my flight back to the UK I created a little script that will take my markdown files and run them through the Google Cloud Translate to create a quick translation of the page that I can then quickly host. The entire solution is presented below. It’s a relatively basic processor, it ignores the Hugo preamble it ignores ‘code’ and it ignores pull quotes - my assumption was to that these are always meant to be left as the way they were written.

Note: It looks like our learning software for translations uses so it’s important to mark up your page so the learning tools don’t use Google Translated content as input to it’s algorithms.

// Imports the Google Cloud client library
const Translate = require('@google-cloud/translate');
const program = require('commander');
const fs = require('fs');
const path = require('path');

program
  .version('0.1.0')
  .option('-s, --source [path]', 'Add in the source file.')
  .option('-t, --target [lang]', 'Add target language.')
  .parse(process.argv);

// Creates a client
const translate = new Translate({
  projectId: 'html5rocks-hrd'
});

const options = {
  to:  program.target,
};

async function translateLines(text) {
  if(text === ' ') return ' ';
  const output = [];
  let results = await translate.translate(text, options);

  let translations = results[0];
  translations = Array.isArray(translations)
    ? translations
    : [translations];

  translations.forEach((translation, i) => {
    output.push(translation)
  });

  return output.join('\n');
};

// Translates the text into the target language. "text" can be a string for
// translating a single piece of text, or an array of strings for translating
// multiple texts.
(async function (filePath, target) {

  const text = fs.readFileSync(filePath, 'utf8');

  const lines = text.split('\n');
  let translateBlock = [];
  const output = [];

  let inHeader = false;
  let inCode = false;
  let inQuote = false;
  for (const line of lines) {
    // Don't translate preampble
    if (line.startsWith('---') && inHeader) { inHeader = false; output.push(line); continue; }
    if (line.startsWith('---')) { inHeader = true; output.push(line); continue; }
    if (inHeader) { output.push(line); continue; }

    // Don't translate code
    if (line.startsWith('```') && inCode) { inCode = false; output.push(line); continue; }
    if (line.startsWith('```')) { inCode = true; output.push(await translateLines(translateBlock.join(' '))); translateBlock = []; output.push(line); continue; }
    if (inCode) { output.push(line); continue; }

    // Dont translate quotes
    if (inQuote && line.startsWith('>') === false) { inQuote = false; }
    if (line.startsWith('>')) { inQuote = true; output.push(await translateLines(translateBlock.join(' '))); translateBlock = []; output.push(line); }
    if (inQuote) { output.push(line); continue; }

    if (line.charAt(0) === '\n' || line.length === 0) { output.push(await translateLines(translateBlock.join(' '))); output.push(line); translateBlock = []; continue;} 

    translateBlock.push(line);
  }

  if(translateBlock.length > 0) output.push(await translateLines(translateBlock.join(' ')))

  const result = output.join('\n');
  const newFileName = path.parse(filePath);
  fs.writeFileSync(`content/${newFileName.name}.${target}${newFileName.ext}`, result);

})(program.source, program.target);

Overall, I am very happy with the process. I understand that the machine translation is not perfect but my thinking is that I can increase the reach of my content to people who might be searching in their own languages and not in English I can increase the discovery surface area of my content and hopefully help more people.

It will take a while to see if this actually helps people, so I will report back when I have more data…. Now to run my script across more of my site :)

Apple - Web apps - All Categories

Paul Kinlan

Remember when Web Apps were a recommended way to use apps on the iPhone?

What are web apps? Learn what they are and how to use them.

Read full post.

In about 2013 Apple started to redirect the /webapps/ top-level directory to /iphone/

The thing is, the directory was actually pretty good, a lot of the apps in there still work today. However looking at the AppStore it solved a lot more problems that developers had: Better discovery and search specifically because the AppStore was directly on the device. The AppStore was also starting to introduce that removed friction from users and developers specifically with regards to payments.

Gears API

Paul Kinlan

I’m writing up a blog post about the early Mobile Web API’s and Alex Russell reminded me of Google Gears

Gears modules include:

  • LocalServer Cache and serve application resources (HTML, JavaScript, images, etc.) locally
  • Database Store data locally in a fully-searchable relational database
  • WorkerPool Make your web applications more responsive by performing resource-intensive operations asynchronously

Read full post.

I think it is interesting to see that AppCache and WebSQL, Geolocation and WebWorkers came out of the ideas in Google Gears and it’s only the latter two that really survived. WebSQL was never broadly supported, and was replaced by IndexedDB; and AppCache replaced by ServiceWorker

RSS Feed to Google Chat Webhook using Cloud Functions for Firebase and Superfeedr

Paul Kinlan

We use Google Chat internally a lot to communicate across our team - it’s kinda like our slack; We also create a lot of content that is accessible via RSS feeds, we even have a team feed that you can all view. It wasn’t until recently that I found out that it was pretty easy to create a simple post-only bot via WebHooks and that gave me the idea, I can create a simple service that polls RSS feeds and then sends them to our webhook that can post directly in to our team chat.

It was pretty simple in the end, and I’ve included all the code below. I used Firebase functions - I suspect that this is just as easy on other Function-as-a-service sites - and Superfeedr. Superfeedr is a service that can listen to Pubsubhubbub pings (now WebSub) and it will also poll RSS feeds that don’t have Pubsub set up. Then when it finds a feed it will ping a configured URL (in my case my Cloud Function in Firebase) with an XML or JSON representation of the newly found feed data - all you have to do is parse the data and do something with it.

const functions = require('firebase-functions');
const express = require('express');
const cors = require('cors');
const fetch = require('node-fetch');
const app = express();

// Automatically allow cross-origin requests
app.use(cors({ origin: true }));

app.post('/', (req, res) => {
  const { webhook_url } = req.query;
  const { body } = req;
  if (body.items === undefined || body.items.length === 0) {
    res.send('');
    return;
  }

  const item = body.items[0];
  const actor = (item.actor && item.actor.displayName) ? item.actor.displayName : body.title;

  fetch(webhook_url, {
    method: 'POST',
    headers: {
      "Content-Type": "application/json; charset=utf-8",
    },
    body: JSON.stringify({
      "text": `*${actor}* published <${item.permalinkUrl}|${item.title}>. Please consider <https://twitter.com/intent/tweet?url=${encodeURIComponent(body.items[0].permalinkUrl)}&text=${encodeURIComponent(body.items[0].title)}|Sharing it>.`
    })  
  }).then(() => {
    return res.send('ok');
  }).catch(() => {
    return res.send('error')
  });
})
// Expose Express API as a single Cloud Function:
exports.publish = functions.https.onRequest(app);

Read full post.

I was surprised and delighted about how easy it was to set up.

Using HTTPArchive and Chrome UX report to get Lighthouse score for top visited sites in India.

Paul Kinlan

A quick dive in to how to use Lighthouse,HTTPArchive and Chrome UX report to try and understand how users in a country might experience the web.

Read More

Getting Lighthouse scores from HTTPArchive for sites in India.

Paul Kinlan

A quick dive in to how to use Lighthouse to try and understand how users in a country might experience the web.

Read More

'Moving to a Chromebook' by Rumyra's Blog

Paul Kinlan

Ruth John moved to Chrome OS (temporarily):

The first thing, and possibly the thing with the least amount of up to date information out there, was enabling Crostini. This runs Linux in a container on the Chromebook, something you pretty much want straight away after spending 15 minutes on it.

I have the most recent Pixel, the 256GB version. Here’s what you do.

  • Go to settings.
  • Click on the hamburger menu (top left) - right at the bottom it says ‘About Chrome OS’
  • Open this and there’s an option to put your machine into dev mode
  • It’ll restart and you’ll be in dev mode - this is much like running Canary over Chrome and possibly turning on a couple of flags. It may crash, but what the hell you’ll have Linux capabilities ��
  • Now you can go back into Settings and in regular settings there’s a ‘Linux apps’ option. Turn this on. It’ll install Linux. Once this is complete you’ll have a terminal open for you. Perfect

Read full post.

Ruth has a great write-up of moving to Chrome OS because her main machine broke.

I moved to Chrome OS full-time 4 months ago (before Google I/O) and only moved to the Mac because I broke my PixelBook (now fixed).

For me it’s one of the best web development machines out there today. It’s the only device that I can test ‘true mobile’ on - you can install Chrome on Mobile on it, Firefox Mobile, Samsung Browser, Brave etc via the ARC platform. Crostini is also a game changer for Chrome OS as it brings a lot of the Linux App ecosystem to Chrome OS and it really starts to fill a huge app-gap for me on Chrome OS; I’ve got Firefox, vim, git, VS Code, Node, npm, all my build tools, GIMP and Inkscape… That’s not to say it has been perfect, Crostini could be faster, it’s not GPU accelerated yet and it could be more integrated with Filemanager etc, and finally the PixelBook really needs more physical ports - I can attach two 4k screens to it, but I can’t charge at the same time.

I think Ruth’s wrap up is also quite accurate, the PixelBook is an expensive machine, but I am very very excited to see this coming to more and more devices (especially those at vastly lower price points.)

Would I pay full price for it? I’m not sure I would pay full price for anything on the market right now. Point me in the direction of a system that will run my graphics software and makes a good dev machine (with minimal setup) and lasts more than 18 months, point me in the direction of a worthy investment and I will pay the money.

Yup.

PWA: Progressive Web All-the-things

Paul Kinlan

PWA. Progressive Web Apps. Frances Berriman and Alex Russell coined the term “progressive web apps” in 2015 with what I think is a seminal post “Progressive Web Apps: Escaping Tabs Without Losing Our Soul”. 3 years later, we’ve come a long way. From a loose collection of technologies - Service Worker, Manifest, Add to Homescreen, Web Push - that were originally only implemented in one browser engine, to a brand that has started to stick across the industry with businesses and developers, and all of the major browser vendors implementing the majority of the ‘PWA’ stack.

Read More

What are the pain points for web designers? - Mustafa Kurtuldu

Paul Kinlan

Mustafa writes:

Tooling is complicated, we are a tooling focused industry, and they change so much. I have used maybe rough eight different tools, from Photoshop to Sketch. That’s before we add prototyping tools to the mix. This may be something we just have to accept. After all, type standards only really started to settle in the 90s, and typography is a 500-year-old discipline.

Designers are still finding it difficult to prove the importance of the process. I think this is something that we have to take on board: to learn how to educate and not just expect everyone to trust us by default. That takes time — perhaps using scenario-based design or design workshops like a design sprint would help. Getting non-designers to observe users while using a prototype they created is one of the best experiences I have seen in this field.

Cross-browser support is lacking crucial features. Designers need to understand developer tooling, to better scope out what is possible. I think using paired programming or the design process described above can help.

Responsive design is still challenging. I think this is in part due to the tools we use; I would love Chrome Design Tools that would help turn the browser into a creative tool. This space is where I think the next evolutionary step for site and web app creation is at. Mozilla is doing some fantastic work in this space, with their layout and shapes tooling.

All in all the challenges that we face seem to be all the age-old ones. Process, tools, and respect.

Read full post.

I found this a very interesting post that is also a complement to a post I wrote about the challenges for web developers. It’s not surprising that browser compat is an issue, but what is still a concern is that building for IE11 is still something that is holding the industry back. Likewise, Mustafa points out that there is still an issue with the tooling around Responsive Design and the emphasis on a single responsive solution always leads to the following (that is in Mustafa’s post):

Designing once and using everywhere is still hard to reach ambition.

This is a problem that I think we all still wrestle with. On one hand we want everyone to build a responsive solution that can serve everyone on every device form-factor, on the other hand user context is important and often the user will only be willing to perform certain actions at certain times; we see this a lot in the retail and commerce industry: people will browse on mobile, and complete on desktop, and the question then becomes do you cater for this multi-modal model more or build a consistent experience across all devices… I suspect the answer is ‘it depends’, but either way it’s a hard problem for everyone from product teams to engineering teams.

Page Lifecycle API - Philip Walton

Paul Kinlan

Philip Walton has an awesome deep dive into a new API the Chrome team has been working on to give you (the developer) control over how to respond when the browser unloads your tabs.

Application lifecycle is a key way that modern operating systems manage resources. On Android, iOS, and recent Windows versions, apps can be started and stopped at any time by the OS. This allows these platforms to streamline and reallocate resources where they best benefit the user.

On the web, there has historically been no such lifecycle, and apps can be kept alive indefinitely. With large numbers of web pages running, critical system resources such as memory, CPU, battery, and network can be oversubscribed, leading to a bad end-user experience.

While the web platform has long had events that related to lifecycle states — like load, unload, and visibilitychange — these events only allow developers to respond to user-initiated lifecycle state changes. For the web to work reliably on low-powered devices (and be more resource conscious in general on all platforms) browsers need a way to proactively reclaim and re-allocate system resources.

In fact, browsers today already do take active measures to conserve resources for pages in background tabs, and many browsers (especially Chrome) would like to do a lot more of this — to lessen their overall resource footprint.

The problem is developers currently have no way to prepare for these types of system-initiated interventions or even know that they’re happening. This means browsers need to be conservative or risk breaking web pages.

The Page Lifecycle API attempts to solve this problem by:

  • Introducing and standardizing the concept of lifecycle states on the web.
  • Defining new, system-initiated states that allow browsers to limit the resources that can be consumed by hidden or inactive tabs.
  • Creating new APIs and events that allow web developers to respond to transitions to and from these new system-initiated states.
  • This solution provides the predictability web developers need to build applications resilient to system interventions, and it allows browsers to more aggressively optimize system resources, ultimately benefiting all web users.

The rest of this post will introduce the new Page Lifecycle features shipping in Chrome 68 and explore how they relate to all the existing web platform states and events. It will also give recommendations and best-practices for the types of work developers should (and should not) be doing in each state.

Read full post.

My first comment is that you should read Philips post. It’s incredible.

On mobile, Chrome can be pretty aggressive at backgrounding (freezing or discarding) the page to conserve resources when the user is not using it (for example, when you swap tabs or move from the Chrome app on Android), when the browser backgrounds your page as a developer you traditionally have no knowledge of when this happens so you can’t easily persist state or even close down open resources and just as importantly when you’re app is back re-hydrate the state cleanly. When developers have control they can make more informed choices, which also means that the browser can be more aggressive in conserving resources in the future without severely impacting user or developer experience.

Finally, the below diagram explains it all pretty darn well.

Page Lifecycle API

Add to homescreen changes in Chrome 68 - Pete LePage

Paul Kinlan

Pete LePage writes about important changes to Add to Homescreen in Chrome

Add to Home Screen changes

If your site meets the add to home screen criteria, Chrome will no longer show the add to home screen banner. Instead, you’re in control over when and how to prompt the user.

To prompt the user, listen for the beforeinstallprompt event, then, save the event and add a button or other UI element to your app to indicate it can be installed.

Read full post.

I had mixed feelings about this originally because so many people don’t handle the beforeinstallprompt event it meant that all of a sudden the number of installs of Web APK’s would drop quite significantly, but I think it’s actually the right thing to do.

The goal is to reduce the number of annoying prompts happening on the web, and the last thing that we need in the industry is for a relatively large prompt to appear when we think the user might want to install a PWA, instead you now need to think about where and when you want to prompt for an install and you have to do it in response to a user-gesture.

The neat thing is that we (Chrome) are introducing more ambient ways of letting the user know that an experience is able to be installed, right now it’s the small bottom bar that appears on the first load, and hopefully in the future we can explore more subtle ways of letting the user know they can take action.

A one year PWA retrospective - Pinterest Engineering

Paul Kinlan

A great overview of Pinterest’s PWA

The verdict

Now for the part you’ve all been waiting for: the numbers. Weekly active users on mobile web have increased 103 percent year-over-year overall, with a 156 percent increase in Brazil and 312 percent increase in India. On the engagement side, session length increased by 296 percent, the number of Pins seen increased by 401 percent and people were 295 percent more likely to save a Pin to a board. Those are amazing in and of themselves, but the growth front is where things really shined. Logins increased by 370 percent and new signups increased by 843 percent year-over-year. Since we shipped the new experience, mobile web has become the top platform for new signups. And for fun, in less than 6 months since fully shipping, we already have 800 thousand weekly users using our PWA like a native app (from their homescreen).

Looking back over one full year since we started rebuilding our mobile web, we’re so proud of the experience we’ve created for our users. Not only is it significantly faster, it’s also our first platform to support right-to-left languages and “night mode.” Investing in a full-featured PWA has exceeded our expectations. And we’re just getting started.

Read full post.

This is a really great post show the benefits of building high-quality, fast and engaging sites. It’s great to also see that the ‘App’ part of PWA, specifically ‘Add to Homescreen’ installability is being used by lots of users. Reading the broader post it’s good to see that they were focusing on a great website experience, that is focus on fast loading sites that have repeatable and predictable performance via code-splitting to reduce the initial load and also good architecture that they can share across the team. Then once you have the base experience you can layer on features like push notifications for the users that want them.

Configuring hugo server to serve 'mjs' ES modules

Paul Kinlan

By default Hugo doesn’t serve .mjs files with the correct content type. In fact it wasn’t until recently that hugo could serve more than one file extension per mime-type. It looks like with v0.43 this has been fixed.

[mediaTypes] [mediaTypes.“text/javascript”] suffixes = [“js”, “mjs”]

Read full post.

The above code lets me serve mjs files for ES Modules with the correct mime-type (note modules need to be served with ‘text/javascript’). This is only needed for local testing, hosting is another issue :)

Thoughts on importing npm modules to the web as JavaScript modules

Paul Kinlan

I’ve got thoughts on the post I did yesterday about ES Modules

I needed a quick way import a simple module get-urls into my project. The module is well tested and it does what I needed … ignore the fact that it’s pretty easy to implement in a couple of lines of JavaScript. The problem I had is that my project is built in ES6, uses modules and I didn’t want to have to bundle up using CommonJS (require).

I couldn’t find a lot of guidance on what to do here, so I went to experiement and this solution is the solution I came across:

  1. Create a file that imports the npm module I needed. module.exports = require(‘get-urls’); This module will be what’s converted to ES6 style.
  2. Create a rollup config that
    1. Imports the node globals, and builtins.
    2. Resolves all npm modules required for my usage of this module.
    3. Pass the results through the commonjs plugin so that it’s now in JavaScript module format.
    4. Compress the output, because it’s huge :
  3. Include the bundled file in your project and rejoice.

Read full post.

One of the things that I wanted to try and articulate in the original article but I decided to pull out is that there is a huge amount of code in the Node ecosystem that is not really that specific to Node per se but has been tightly coupled with Node via Common JS and other very specific Node API’s (Buffer, old URL etc etc) that it’s going to take a lot of effort to pull ourselves up and thus the change be required to make ES Modules ubiquitous will be potentially quite painful, and until the ecosystem changes we are going to need to use a lot of conversion tools and bundlers to be able to cleanly share code across multiple platforms (web/server).

We are where we are, there wasn’t an importing story on the web, we didn’t have a heap of the primitives that Node introduced and are now what many would now consider de-facto platform requirements, so I hope that this is more of an acknowledgement of the situation than a criticism.

There is also a move to use ‘.mjs’ as a file extension that is standard across both node and the web. I feel totally comfortable with this, however .msj is not a file that any infrastructure yet recognises as ‘text/javascript’ and I’m looking forward to this just being sorted so that it’s automatically inferred by every web server on the planet, so I don’t have to deploy yet more configuration changes to my serving infrastructure.

Lots of fun times ahead, I for one am looking forward to being able to bring a lot more functionality to the web.

Importing npm modules to the web as JavaScript modules

Paul Kinlan

I’ve been working on a way to make it easier to push content into my static site and it’s been a fun little exercise that I will share more in another post. In this post I want to share the rollup config that I used to import nearly any npm module in to a frontend project using JavaScript modules. I needed a quick way import a simple module get-urls into my project.

Read More