ChatGPT, OpenAI’s latest public Large Language Model (LLM) has been described as "like having an army of interns” since the quality of output is unpredictable but the breadth of the training data (billions of internet pages and 100k+ books) means using it is akin to delegating a task to a group of inexperienced people who you know will read everything they can find about a topic but might still miss the mark on understanding what you actually want. Also you get an answer instantly.

In my testing I found the current iterations of ChatGPT and GPT-3 (which are largely similar) to have high failure rates comparing and summarizing content in detail. That isn’t to say there aren’t occasional spectacular successes, just that it’s unreliable without careful coaching (aka prompt engineering). I did find it quite capable of accurately producing short form summaries though and decided to improve search-ability and usability of my public reading list by using GPT-3 to read all the articles I added last month to:

1. Summarize

2. Generate relevant keywords

3. Grade the piece as if it were a college professor who was a subject matter expert

4. Write the most upvoted response on Reddit

Of note the ECB’s piece on Bitcoin received the lowest grade with a 60%, the feedback given was: “Does a good job of exploring the complexities surrounding the Bitcoin market, though it could have benefited from more concrete examples of the risks posed by crypto assets”.

On the other hand my intern army gave two perfect 100% grades: Uniswap's Financial Alchemy and Bitcoin Can’t Be Copied. I concur, both are must reads.

My takeaway overall from this experiment is that becoming familiar with the strengths and weaknesses of OpenAI’s GPT models is likely to be an asset. If you are comfortable running a relatively untested Chrome extension I’d suggest setting up this which displays ChatGPT’s answer beside all your Google queries to start learning today without altering your existing workflow.

Follow me on Twitter @jackoneschuk or Farcaster @joneschuk if you enjoyed this post

ChatGPT, OpenAI’s latest public Large Language Model (LLM) has been described as "like having an army of interns” since the quality of output is unpredictable but the breadth of the training data (billions of internet pages and 100k+ books) means using it is akin to delegating a task to a group of inexperienced people who you know will read everything they can find about a topic but might still miss the mark on understanding what you actually want. Also you get an answer instantly.

In my testing I found the current iterations of ChatGPT and GPT-3 (which are largely similar) to have high failure rates comparing and summarizing content in detail. That isn’t to say there aren’t occasional spectacular successes, just that it’s unreliable without careful coaching (aka prompt engineering). I did find it quite capable of accurately producing short form summaries though and decided to improve search-ability and usability of my public reading list by using GPT-3 to read all the articles I added last month to:

1. Summarize

2. Generate relevant keywords

3. Grade the piece as if it were a college professor who was a subject matter expert

4. Write the most upvoted response on Reddit

Of note the ECB’s piece on Bitcoin received the lowest grade with a 60%, the feedback given was: “Does a good job of exploring the complexities surrounding the Bitcoin market, though it could have benefited from more concrete examples of the risks posed by crypto assets”.

On the other hand my intern army gave two perfect 100% grades: Uniswap's Financial Alchemy and Bitcoin Can’t Be Copied. I concur, both are must reads.

My takeaway overall from this experiment is that becoming familiar with the strengths and weaknesses of OpenAI’s GPT models is likely to be an asset. If you are comfortable running a relatively untested Chrome extension I’d suggest setting up this which displays ChatGPT’s answer beside all your Google queries to start learning today without altering your existing workflow.

Follow me on Twitter @jackoneschuk or Farcaster @joneschuk if you enjoyed this post