I integrated the Squoosh CLI into my web app to optimize images. Although Squoosh offers a great CLI, I needed its functionality within my app. Leveraging my experience with FFMPEG in web apps, I adapted the Squoosh CLI code, replacing Node.js dependencies with web APIs. Now, I can call Squoosh's 'run' method directly in my app to resize and compress images. This unofficial solution works for now, but a dedicated browser API would be ideal for broader integration in CMS platforms, performance analysis tools, and other web applications.
I've been exploring Chrome's Shape Detection API and its potential, even creating a QR code detector with a JS polyfill. While I appreciate the API, I'm more impressed by libraries like Jeeliz, which offer faster face detection and broader cross-platform compatibility via WebGL and WASM. This tech allows for augmented reality experiences without large app downloads and data harvesting. There are numerous practical applications for this technology, from text selection and translation to product search and accessibility features. I believe the web is ideal for such experiences, and although I appreciate the native Shape Detection API, its inconsistencies across platforms raise questions about the need for native implementations when web-based solutions using WebGL and WASM can offer better consistency and reach.
I'm embarking on a project to build a web-based video editor! The goal is to create a tool that simplifies video creation and editing entirely within the browser. Think Screenflow, but accessible to everyone directly on the web. This project is driven by my own needs for creating device demos, screencasts, and other videos. I've already made some progress (check out the demo!), but there's a lot more to do. I'll be exploring existing web technologies to record audio/video, manipulate content (watermarks, filters, overlays), and output in various formats. This isn't about building a massive commercial product, but rather about understanding what's possible and empowering others to create great videos using the open web.
I combined FFMPEG.js, a tool compiled with asm.js for video editing in web apps, with Comlink, a library that simplifies web worker interactions. This integration, along with my experiment of exporting FFMPEG to Web Assembly, allows for cleaner video encoding off the main thread. The provided code snippets demonstrate the simplicity of using Comlink to expose the ffmpeg interface within a web worker and then access it from the main thread as a proxy, offering a neat solution for asynchronous video processing.
Emscripten is a great tool for compiling to WebAssembly, but it can introduce unnecessary overhead. It's important to minimize the runtime size for any language compiled to WebAssembly. This post explores how to use Emscripten with a minimal runtime, avoiding excessive magic and focusing on efficiency.
Following on from my list of the things that I am [excited about on the web in 2017](/2017-exciting-times/) here are three predictions of things that I think will happen in 2017 based off reading tea-leaves and random musings of things that I have 0 direct knowledge of but I might do if I was 'them'.
This blog post explores the potential of WebAssembly (Wasm) for full-stack development, allowing code sharing between client and server. I discuss how Wasm could enable progressive enhancement for web APIs like the Shape Detection API. Using this API as an example, I illustrate how a C-binding library like OpenCV, compiled to Wasm, could be used on both client and server to provide consistent functionality regardless of native browser support. This approach involves creating a wrapper around OpenCV and the target web API to bridge the gap between them. I express my excitement about Wasm's potential to simplify deployment and maintenance by enabling the use of a single binary across different environments.
Building ffmpeg.js, a tool I used in my project Device Frame, can be tricky if you need custom filters and encoders. This post details the steps I took to successfully build it on Ubuntu, including installing dependencies, cloning the repo, setting up Emscripten, and running the build. I also included solutions to some common errors encountered during the process.
