I explored using LLMs for checking web API browser compatibility. Existing LLMs struggle with outdated data, so I experimented with MDN's Browser Compat Data (BCD). Initial trials using raw BCD JSON with GPT-4 had limitations. To improve this, I converted the BCD into English descriptions of API support and loaded it into a Polymath instance. This allows natural language queries about API compatibility across browsers, like "Is CSS Grid supported in Safari, Firefox, and Chrome?" or "When was CSS acos available in Chrome?". The results are promising, but further refinement is needed to ensure accuracy and reliability.
Web compatibility is a major developer concern. While projects like Compat 2021 aim to address these issues, data-driven analysis is crucial for understanding the web's evolving compatibility landscape. This post highlights Browser Compat Data (BCD), a valuable resource from Mozilla that offers detailed compatibility information for web APIs. BCD bridges the gap between raw Web Platform Tests data and user-friendly tools like caniuse.com. I've created a demo app, "The Web Of...", utilizing BCD to visualize API availability across different browsers at specific points in time. This data empowers developers to make informed decisions about API usage, assess compatibility across browser engines, and track the overall progress of web compatibility. The availability of such data opens up possibilities for new metrics like a "CompatIndex" to quantify web compatibility. Contributions to the BCD project are encouraged to further enhance this valuable resource.
This post explores the concept of web compatibility and whether it can be quantified. It draws parallels to the Receiver Operating Characteristic (ROC) curve, questioning if a similar metric could represent the trade-off between compatibility and feature availability. The post also examines how browser vendors' incentives influence feature adoption and proposes leveraging compatibility data sources like caniuse and web platform tests (WPT) to prioritize compatibility improvements. Potential tools, such as a compatibility bot and automated blog updates, are suggested to highlight these improvements.
