I made cookies and an analysis of AI ethics with ChatGPT

The generalist nature of this tool lets us envision a future in which we can do more or less anything with it without being experts ourselves. I tried a few simple experiments to test out the plausibility of this idea.

My first task was to bake cookies. Real cookies. Actual cookies. Not data packets about your navigation habits. Specifically, I wanted cookies soft on the inside, crispy on the outside and that tasted like Ben’s Cookies (UK highstreet brand).

Important point: This choice matters because no such recipe exists, described as such on the web. My requests asks for something very specific that requires ChatGPT to blend recipes intelligently, rather than just copy an existing one or send me to a web page, as search engines would.

Although the shot above shows the limits of my cookery skills and is an indictement of my photographic talent of that day, for what it’s worth, the cookies were excellent and completely met my requested criteria.

A few days later, I repeated the experiment with hummus. My request was for a traditional recipe met with great reviews. Which is a bit of a conundrum since traditional recipes tend not to be found online. And those recipes that get published on websites more generally are “lifestyle” versions of the original.

Again, fantastic results. And the recipe instructed me to mix ingredients in a way that my (numerous) book recipes never told me to. The hummus was smoother, lighter but tastier.

Important point: Why am I telling you this? Because being able to judge the result is key here. I was able to verify that the instructions made more sense than those found in my books, and that the taste of both the cookies and the hummus was to my liking (and since the original draft: scones, shortbread, biscuits from Brittany, all good, so you can now call me Chat Baker). When you have the necessary expertise to judge the results of a ChatGPT output, then it can be a fantastic tool to use to save time or discover interesting stuff. It’s when the results take you to areas beyond your expertise that you should be extra careful.

Cue the next experiment.

Producing a report on the ethics of Artificial Intelligence

This part is mindblowing.

But is it trustworthy?

Obtaining tasty recipes was a matter of asking and being specific. It took ChatGPT a few seconds to answer and I picked the first answer in both cases. What follows necessitated multiple questions – each new one hinging on the results of the previous one – and roughly 30 minutes of time, using my paid subscription to GPT4, not plain Jane ChatGPT.

Important point: Those systems (somewhat) “remember” previous answers they made in a conversation, allowing you to progressively build layer upon layer of reasoning to create something quite elaborate. Almost like programming the tool, but in natural language.

My goal in this second experiment was to get ChatGPT to establish the list of most important topics to study to further the cause of ethics in AI. Asking directly would have been a possibility, and would have yielded an answer based on what has been written online and fed to ChatGPT during its training. Being unable to judge the quality of the response, unlike the taste of a cookie, I tried the following approach :

1. Establish a list of all topics it can “think” of that are related to the ethics of AI. => It gave me 13, to which I added one. We now had 14 (bias and fairness, explainability …).
2. Find a list of axes (not topics) according to which we could analyse the impact of each topic => This resulted in a list of 7 (Economic impact, social well being, health impact, environmental impact, political implications, moral values, technological progress).
3. Project each of the 14 topics on the 7 axes, to see which topic has an influence on which axes. => For example, Bias and Fairness has a bearing on Economic Impact, Social Well-being, Political Implication, Moral Values)
4. Rank the 14 topics based on the number of axes they have a bearing on (and rank the axes based on the number of topics they relate to), justifying the results using scoring.

The end result was this list :

1. Economic Impact and Social Well-being (14 topics each)
2. Political Implication (13 topics)
3. Moral Values (9 topics)
4. Health Impact (8 topics)
5. Environmental Impact (3 topics)
6. Technological Progress (Not salient for any specific topic, though implied in several)

Extraordinary! On paper, this could help a think tank structure their work according to clear priorities. Almost for free. For good measure, I asked GPT4 to compare this to a hierarchy of human needs such as Maslow’s. The answer came thusly:

1. Economic Impact and Health Impact could correlate with the Physiological and Safety levels, as they concern fundamental needs for survival and well-being.
2. Social Well-being could be related to the Love and Belongingness level, as it deals with relationships and community.
3. Moral Values, Accountability, and Justice and Equity could link to the Esteem level, as they touch on respect and fairness.
4. Technological Progress, Human Control of Technology, and Transparency and Explainability could connect to the Self-Actualization level, as they involve reaching potential and understanding.

And, finally, asked for reasons why the two lists differ, GPT4 told me:

The difference between the impact of AI on society and a hierarchy of needs like Maslow’s could be due to a few factors:

1. Complexity of AI Impact: AI’s effects on society are wide-ranging and interconnected. They don’t fall neatly into categories or a hierarchical order because they often occur simultaneously and impact multiple areas of life. For instance, an issue like data privacy has economic implications (potential fines for data breaches), social implications (individuals’ trust in institutions), and personal implications (potential identity theft).
2. Divergence from Individual Needs: Maslow’s hierarchy is primarily focused on individual needs, whereas our analysis is looking at societal-level impacts. While there’s overlap, the priorities can differ. For example, economic growth is often a societal priority, even though it doesn’t correspond directly to an individual need in Maslow’s hierarchy.
3. Adaptive Nature of Society: Societal wellbeing programs often have to respond to the current state of society, which is influenced by many factors beyond basic needs, including technological advancements, cultural shifts, and political climates. For example, 50 years ago, internet access wouldn’t have been part of societal wellbeing discussions, but now it’s seen as a key factor in social and economic participation

Important point: This feels like magic. Had I asked a monitoring company to find all articles on the topic and create a similar statistical analysis, it would have taken weeks and cost well into 5 figures. But is any of it true?? Is any of it blatantly wrong?? I sure don’t know and do not have the necessary expertise to evaluate the results of this quick experiment.

Using GPT (over a period of hours) for an analysis of value creation in railway systems, I was incredibly pleased to see it produce abundant bibliography. Something I had not been able to do myself using traditional means (which was my job, early on). I had attributed this to my lack of expertise in the topic and not knowing where to find the proper scientific publication databases. The answer was far worse. When I checked the bibliography, all 15 articles were fake. Totally credible, as the authors exist and do write about the topic. But the specific articles do not exist. They were merely dreamt up as ordinary text, as GPT does. A lawyer recently experienced this first hand.

Takeaway: The takeaway is simple. Use GPT to accelerate your work freely when you have the ability to judge the trustworthiness of the results it gives you. It’s fun, it’s a huge time saver and you’ll often be stunned at the quality of the results. But for breaking new ground, you can only use it as a tool for broadening your horizon, knowing what to search for, finding and framing interesting ideas. And certainly not as a source of truth. At least not as this is being written, mid 2023.

Final note: If anyone reading this has some understanding of ethics or, more specifically ethics in the field of AI, please let me know how ChatGPT did on this (or how I did, considering I may not even have asked the proper questions).

If you would like to understand how ChatGPT works and what its emergent properties are, you can read the two other articles I linked to.