So much of my experience of browsing the web is fighting sites over how I want them displayed, that I'm beginning to think that I don't want a web browser that renders web pages, and instead I want a web browser that scrapes content from web pages and displays it according to my preferences.
I've started working on this. The web has gotten sufficiently annoying, even with ad-blocking, that it feels worth it.
Still working on this. Got as far as the body tag and into the header of the page I'm working on. But, surprise, the HTML is malformed. They failed to close an element. So I had to add a rule so that, if an ancestor element is closed, it treats that as implicitly closing the current element.
Browsers don't care if your HTML is well-formed or not so this isn't surprising.
@madewokherd i wonder if a better architecture wouldn't be building this around headless chrome
@tbodt That would be a very different project.
@madewokherd the train of thought is, first it's just html, but someday you're probably going to find some website that needs to fetch the content with javascript, so maybe it does make sense to reuse all the browser crap but wrap it in a highly controlled layer. but that does make it a very different kind of project
@tbodt When that happens, I will write my own logic to fetch the content.
@madewokherd obligatory embroidery troubleshooting page (header tag left open in each level) https://web.archive.org/web/20140310190221/http://www.sewingandembroiderywarehouse.com/embtrb.htm
@mavica_again I'll make sure I test with this, thanks.
I reached a point where I can parse most of an HTML header and now I'm exhausted.