Recently I stumbled upon some old posts sharing what I had been reading (between 2010 and 2012). It’s been a while since then – and I don’t think I’ve shared much of what I’ve been reading (or re-reading) since.
On the one hand, it seemed to be a prohibitive long list to write by hand – so I haven’t bothered to do so (until now). On the other, since all of my reading purchases have digital receipts of one kind or another, creating an updated list of reading seemed like a tempting opportunity to test how far AI has come (or not), using a real life menial task consisting of several steps, including several sources, and different types of data – something so tedious and time-consuming (and for little to no payoff) that it needs outsourcing of one kind or another to actually get done.
TL;DR
Coaxing ChatGPT took an inordinate amount of trial and error to get to do anything useful (and consistent) at all, but after a lot of painfully crappy results I got to a point where the process was somewhat manageable, making the process significantly faster than using manual labor only – but still requiring some manual labor nonetheless.
When ChatGPT was asked to do the work, it refused to. Constantly. And when it did do some work, it lied. Randomly. And when it actually produced usable results, it worked slower than an intern with a hangover. Also, doing random spot checks, I found that books were missing, inexplicably dropped along the way, some summaries were hallucinated, some links where swapped with placeholders, etc.
Key insights: Break down tasks into the smallest single step, do not bundle steps in a chain (do not use and and and), complete one step, move on to a new instance for the next step, prompt for new task and input output of previous step in this new instance. Counterintuitively, it seems the less ChatGPT knows about the whole, the better the results. Also, try repeating same task with same prompting in multiple different instances – it doesn’t matter how or what you prompt, Chat GPT *will* (sooner or later) go off the reservation regardless – it might just suddenly work in one of the instances (by using exactly the same prompting that failed in 10 of the other instances).
To wit: OpenAI is the most overvalued startup in history – I keep asking myself why I pay them; ChatGPT as a product experience is (still) very bad.
The links to Amazon and Audible do include affiliate tags. (If you don’t want to feed the Bezos machine, you know how to get it somewhere else already). However, I’ll never get back even a fraction of the time it cost me to wrangle ChatGPT to do some actual work.
Now what have you read that I haven’t (but should)? Read on below for the +365 items that made it to the list.
Continue reading