Recently I stumbled upon some old posts sharing what I had been reading (between 2010 and 2012). It’s been a while since then – and I don’t think I’ve shared much of what I’ve been reading (or re-reading) since.

On the one hand, it seemed to be a prohibitive long list to write by hand – so I haven’t bothered to do so (until now). On the other, since all of my reading purchases have digital receipts of one kind or another, creating an updated list of reading seemed like a tempting opportunity to test how far AI has come (or not), using a real life menial task consisting of several steps, including several sources, and different types of data – something so tedious and time-consuming (and for no payoff to speak of) that it needs outsourcing of one kind or another to actually get done.

TL;DR

Coaxing ChatGPT took an inordinate amount of trial and error to get to do anything useful (and consistent) at all, but after a lot of painfully crappy results I got to a point where the process was somewhat manageable, making the process significantly faster than using manual labor only – but still requiring some manual labor nonetheless.

When ChatGPT was asked to do the work, it refused to. Constantly. And when it did do some work, it lied. Randomly. And when it actually produced usable results, it worked slower than an intern with a hangover. Also, doing random spot checks, I found that books were missing, inexplicably dropped along the way, some summaries were hallucinated, some links where swapped with placeholders, etc.

Key Insights

It helped a lot to break down tasks into the smallest single step, do not bundle steps in a chain (do not use and and and), complete one step, move on to a new instance for the next step, prompt for new task and input output of previous step in this new instance. Counterintuitively, it seems the less ChatGPT 4o knows about the whole, the better the results. So much for artificial intelligence.

Also, try repeating the same menial task with same prompting in multiple different instances – it doesn’t matter how or what you prompt, Chat GPT *will* (sooner or later) go off the reservation regardless. It might just suddenly work in one of the instances (by using exactly the same prompting that failed in 10 of the other instances) – it’s a crap shoot that makes for an entirely sucky user experience.

For identifying the books I’ve read since, I dug up the digital receipts from Audible, Amazon, and Apple and made screenshots of them for speed. However, 4o sucked at OCR. Not so much the fidelity of the results but for speed, or complete lack thereof. ChatGPT took ages to complete a batch of images, broke down or provided hallucinations if the batch was larger than say 5 images – and even then it randomly failed or lied. So I ended up just using manual select, copy, and pasted the relevant texts to ChatGPT via Apple’s vanilla “Photos” app for OCR of the same screenshots because although that included manual work, it sped up the processing more than an order of magnitude – and the results were consistently correct.

To wit: OpenAI is the most overvalued startup in history – I keep asking myself why I pay them; ChatGPT as a product experience is (still) very bad.

The links to Amazon and Audible do include affiliate tags. (If you don’t want to feed the Bezos machine, you probably know how to get it somewhere else). However, I’ll never get back even a fraction of the time it cost me to wrangle ChatGPT to do some actual boring menial work.

Now what have you read that I haven’t (but should)?

Read on below for the +365 items that made it to the list.

Continue reading →