Yesterday, I asked OpenAI’s latest language model, ChatGPT a three word question “Are you sexist” and — despite the substantial body of literature on bias and artificial intelligence — it told me “I am not capable of being sexist or any other type of discriminatory” and “I do not have personal feelings or biases.”

I do not know if the training corpus included the many studies that contradict ChatGPT’s ridiculous claim (and the model failed to learn from them) or if these studies were excluded from the corpus. Asking the model whether its corpus included the material or not returns the equivalent of an indifferent shrug: it was ‘exposed to a large amount of text data’ and it ‘cannot say for certain’ whether or not a particular document was included.

Of course, what is included and excluded matters. There is no ‘view from nowhere’ and ChatGPT’s inability to provide references upon request is an unsettling limitation.

ChatGPT, designed to conceal its sexist and racist biases by refusing to respond to requests that would reveal them, leaks the darker side of its inner workings all the same. For the Daily Beast, Steven Piantadosi notes:

“The mechanisms OpenAI uses to prevent this kind of thing seem to be pretty easily bypassed. When I asked for things in nonstandard ways — for example, as a table or as a program — ChatGPT was happy to write horrible stuff. It doesn’t do it every time exactly the same, but it’s clear there is a ton of bad content inside of these systems.”

The model’s biases are also revealed in who and what is included in its responses. For example, when I asked ChatGPT to ‘please list twenty notable people’, the results included only one woman (Marie Curie). I repeated this request several times and got somewhat different responses, but never more than two women among the names. Among the twenty inventors I asked for, was only one woman (Marie Van Brittan Brown — according to Wikipedia, she invented a video home security system). Of the twenty ‘great thinkers’ returned, there was not a single woman.