I built a Progressive Web App (PWA) that extracts text from images shared to it. It uses the Share Target API to receive images, the Shape Detection API's TextDetector to analyze them, and EXIF-Js to handle image rotation issues. While it's a handy tool, it currently suffers from cross-browser and cross-version compatibility problems due to the lumpy nature of the web platform. The code snippets highlight key implementation details, like manifest setup, service worker handling, text extraction, and image rotation.
I've been exploring Chrome's Shape Detection API and its potential, even creating a QR code detector with a JS polyfill. While I appreciate the API, I'm more impressed by libraries like Jeeliz, which offer faster face detection and broader cross-platform compatibility via WebGL and WASM. This tech allows for augmented reality experiences without large app downloads and data harvesting. There are numerous practical applications for this technology, from text selection and translation to product search and accessibility features. I believe the web is ideal for such experiences, and although I appreciate the native Shape Detection API, its inconsistencies across platforms raise questions about the need for native implementations when web-based solutions using WebGL and WASM can offer better consistency and reach.
This blog post explores the potential of WebAssembly (Wasm) for full-stack development, allowing code sharing between client and server. I discuss how Wasm could enable progressive enhancement for web APIs like the Shape Detection API. Using this API as an example, I illustrate how a C-binding library like OpenCV, compiled to Wasm, could be used on both client and server to provide consistent functionality regardless of native browser support. This approach involves creating a wrapper around OpenCV and the target web API to bridge the gap between them. I express my excitement about Wasm's potential to simplify deployment and maintenance by enabling the use of a single binary across different environments.
I'm excited to share that barcode detection is now available in Chrome Canary via the Shape Detection API! This feature, along with QR code detection, offers a standardized way to access device hardware and bridge the physical and digital worlds. It's especially useful for mobile, eliminating the need for special apps just to scan barcodes. The API is simple: use BarcodeDetector.detect() with an image input to get a promise resolving to a list of barcodes. I've already integrated this into my QR Code Scanner App, and the ability to use it in worker threads is a huge bonus. It's a very promising addition to the web platform and I'm looking forward to seeing what people build with it!
I'm excited about the new experimental Shape Detection API in Chrome Canary! It provides a simple JavaScript API for face and barcode detection, leveraging underlying hardware for performance. This opens up new possibilities for web apps, from faster face detection and profile picture cropping to real-time tagging and optimized facial recognition. While currently only available in Chrome for Android (desktop support coming soon), I've shared a demo on JSBin. I also discuss strategies for progressive enhancement to ensure broader compatibility, including server-side detection, client-side JavaScript libraries, and the potential of Web Assembly. This API has the potential to revolutionize object detection performance on the web, and I'm particularly keen to see its impact on barcode scanning apps like my own QR Snapper.
