I needed higher resolution screenshots for an ML model to classify elements on a webpage, but the default Lighthouse screenshot was too compressed. So, I created a custom Lighthouse Gatherer using Puppeteer. This gatherer captures a full-page, high-resolution screenshot encoded as base64 and returns it along with the device pixel ratio. This was a fun little project, and the code is surprisingly concise. However, future Lighthouse versions may include higher-resolution screenshots, making this gatherer redundant.

I built a web app using Deno, Fresh, and TensorFlowJS to classify images as links or buttons. The app uses a pre-trained ML model and allows users to drag and drop multiple images for classification. I encountered challenges with server-side rendering and islands, specifically with integrating a file-drop web component. I also documented the process of integrating the TensorFlowJS model, including model loading and prediction handling. The code is available on GitHub.

I trained a machine learning model to differentiate between buttons and links on web pages. Using a dataset of ~3000 button images and ~4000 link images, I trained a convolutional neural network (CNN) with added noise for better generalization. Preprocessing included grayscale conversion, dataset diversification with multilingual sites, and image compression. The model performed well in initial tests, correctly classifying button-like and link-like elements. Next, I'll build a web app for easier testing and a Lighthouse audit for website analysis.

In this project, I'm working on an accessibility tool to detect links styled as buttons, a common issue that can confuse users. My approach involves scraping websites to gather images of buttons and links, and then training a machine learning model to distinguish between them. This post focuses on the scraping process using Puppeteer. I encountered challenges like occluded elements and smooth scrolling, which I addressed by checking for occlusion and disabling smooth scrolling. The next step is training the ML image classifier.

I added ActivityPub support to my static Hugo blog hosted on Vercel. It now automatically announces new posts to followers on the Fediverse. Key challenges included implementing the ActivityPub protocol for a static site, handling WebFinger discovery, managing Follow/Unfollow requests, and sending signed HTTP requests. I used Vercel Serverless Functions for dynamic request handling and Firebase Firestore for storing follower data. Check out the code and follow me @paul@paul.kinlan.me to see it in action!

I needed to find a way to send webhooks after a successful deployment on Vercel, which wasn't a built-in feature. Since Vercel integrations can listen for deployment events, I created one to solve this. It's a simple tool hosted on GitHub that lets you set up custom webhooks for your Vercel projects. It's not on the Vercel Marketplace, and it's more of a workaround until Vercel natively supports deployment webhooks. Check out the GitHub repo for instructions on setting it up with Firebase Firestore.

Mastodon's cross-instance following is clunky. I propose a solution using the web+follow URL scheme, like mailto, to streamline following users on ActivityPub platforms. A user's home instance registers as a handler for web+follow. Clicking a web+follow link opens the home instance, redirecting to a /follow request on the target instance. This page uses a meta refresh tag to redirect to /resolve-app, which redirects back to web+follow: with the user's ID. This triggers the registered handler (if available), creating a seamless follow experience within the user's preferred app. If no handler is installed, the user remains on the target instance's follow page, preserving existing functionality.

This blog post explores how machine learning (ML) can enhance the developer experience. Inspired by Corridor Crew's use of ML in VFX, I initially brainstormed ways ML could automate tedious developer tasks, like accessibility improvements and performance optimization. I also considered ML's potential for generating layouts and images. The emergence of tools like GitHub Copilot and DALL-E-2 significantly impacted my thinking, especially regarding the future of software development and my role as a DevRel lead. Ultimately, the transformative power of GPT-Chat, demonstrated through its ability to generate webpage layouts and populate them with images based on simple prompts, left me questioning the future of my profession and considering the role I might play in training the next generation of AI tools.

Custom URL schemes can enhance web app functionality by handling specific URLs, but detecting scheme support is tricky. Several methods exist, including click handlers, navigation handlers (Blink), and server-side redirects with meta refresh. While the server-side approach offers the most robust solution, it introduces complexity. A key challenge is the limited user understanding of custom schemes, leading to a preference for standard HTTPS URLs. This post explores a common pattern for custom scheme usage, involving detecting navigation failures and presenting alternative UI. The pattern addresses the issue of handling custom schemes like web+follow for Mastodon, aiming to improve user experience. While custom schemes are valuable developer tools, user preference for HTTPS URLs persists. Despite this, custom schemes empower developers to guide users to preferred apps or sites while gracefully handling cases where no suitable option exists. This approach also opens possibilities for other applications, like rebuilding web intents.

I'm heartbroken about the situation at Twitter. So many of my friends have been impacted by the changes, and I feel helpless beyond offering support and sharing job opportunities. Twitter played a huge role in my career journey, offering a place to connect with others and find my voice. It's how I made many friends and career connections, and I'm grateful for the community it fostered. I dislike the rapid changes happening now, and like many, I'm concerned about its future and how it will survive. While I haven't paid for the service, I feel a sense of debt to the platform and community for all it's given me.