RE: Some things about XLinq
This post responds to Mike Champion's comment on my previous XLinq blog post. I clarify the XML file used (Wikipedia XML Abstract) and explain why I chose an XMLReader for its speed, especially when combined with custom data structures for a cyclic graph representation. XLinq's syntax and lambda expressions felt less intuitive for my task of converting XML into SQL statements. The project involves relating "title" elements with "sublink" entities, resulting in a complex graph structure not easily handled by XLinq without excessive data duplication and memory consumption. While XStreamingElement offers some improvement by avoiding redundant data scans, I desire deferred data loading for processing only necessary slices of the XML. This approach could handle selects, wheres, and counts efficiently in a single pass, and even joins with clever indexing. Defining a schema during XML iteration seems redundant when XLinq expressions already specify data requirements. Pre-loading entire XML documents into memory feels inefficient when only a small portion is used. I propose deferring data loading until needed, despite potential issues with repeated XDocument inspections. Ideally, XLinq should scale without forcing users to revert to less efficient methods due to data size limitations. I inquire about potential hard limits and scaling formulas related to XML document size in XLinq.