Journal May 22nd, 2022

  • Created No Face - It's a simple Web App that quickly detects and hides people's faces. It's inspired by Casey Liss' Maskerade iOS App. I wanted to see how easy it was to build... We've got a Web API for FaceDetection, right?
    • It's built using SvelteKit and is based on an earlier project GitHub - jecfish/mosaic-it that Jecelyn Yeen, Eiji Kitamura and myself made.
    • FaceDetection
      • I thought the FaceDetection API was fully launched. It's not it seems. It's a bit of a mess of a situation. We designed this API at least 3 years ago (maybe 4) and it's still not available or ever likely to be if you ask me. Details here: The Shape Detection API: a picture is worth a thousand words, faces, and barcodes
        • The situation with half shipped Fugu API's is mightly frustrating.
        • TODO Speak to the team to see what we can do about finishing what's been started, or can we even stop?
      • I went with TensorFlow JS Face detection API tfjs-models/face-detection at master · tensorflow/tfjs-models · GitHub
        • It uses MediaPipe (a google Vision project) under the hood and is a convenience wrapper over it. I wasted 3 hours trying to get MediaPipe to build on the web thinking it would be more efficient. I could not get it working and there is 0 help or support. It might be reasonable to use in production if you can get it to work, but I personally wouldn't rely on it.
        • I reverted back to Tensorflow JS and it was set up to get MediaPipe running it had it's own problems. There is 0 guidance about how to get this running in a production environment. For both projects it's 'codepen' examples that pulling from jsdeliver (or another CDN) and there is no thoughts about how to get this running in a build environment. It turns out that esbuild was stripping out code it thought was dead and I had to build the dependency with custom parameters.
    • Build tools
      • SvelteKit uses vite under the hood, and at a basic level the documentation and project is sound. But I kept hitting edges that stalled any progress. The dead code elimination above was one such example. The Tensorflow JS library just doesn't seem to work well with bundlers and minifiers and I can't really change any of it so I have to work out to force it in.
        • TODO speak to the Tensorflow JS team to see if they can test this more with production systems and build tools that developers use.
      • I found it hard to find any guides about working with SvelteKit and vite. There's certainly things out there, and the Github Issue tracker is very active, but I was hoping to find something that would guide me along a little more.
    • Lingering thoughts
      • There was a thread on Twitter about building a "Translation API" in to the browser, and it mentions "ML" and other stuff but the discussion was centred around the fact that it would likely be impossible to standarise and it reminded my of WebSQL - the JS interface might be simple, but the model and the engine underneath would have to be created and maintained by each engine, developers would expect consistent results and given that ML is still so nascent, it's a folly to chase. It left thinking more and more that the browser should be offering fewer higher-level API's and the ecosystem should provide the solutions. User-space solutions allow for rapid iteration and competition especially for use-cases where they're unlikely to be used everywhere. FaceDetection there are libraries that should be competing on developer experience and utility, if there are no libraries and no competition maybe the demand isn't there and in that case there certainly shouldn't be a 'platform API' for it.

About Me: Paul Kinlan

I lead the Chrome Developer Relations team at Google.

We want people to have the best experience possible on the web without having to install a native app or produce content in a walled garden.

Our team tries to make it easier for developers to build on the web by supporting every Chrome release, creating great content to support developers on web.dev, contributing to MDN, helping to improve browser compatibility, and some of the best developer tools like Lighthouse, Workbox, Squoosh to name just a few.