I spent 3 days scraping a site until I tried LLMs for data extraction

The author spent three days struggling with traditional web scraping methods due to constantly changing HTML classes and field order. They eventually found a solution by using Language Models (LLMs) to treat the entire page as a blob of text for data extraction, shifting from pattern-finding to understanding the meaning.