An Idiot's Guide to Line Editing

Chapter Eighteen -- Beep boop I'm a robot

Interesting, right? It's actually worse, overall. However, if you look more closely, its getting some stuff right.

Firstly it has nixed all but one of my 'was'. It's really embraced that mantra. I don't like all of its techniques: 'stretched' to me implies movement, but it's a great place to think about other verbs. 'Within its confines' is also quite nice, but again, not my writing style.

Then, some of the sentences are probably better structured than mine. Look how it elided the 'was' in the first sentence of the second paragraph by using the verb 'danced'. That's not the verb I'd use, but it's a good foundation for a restructure of that sentence.

But that's all overshadowed by its choice of words. It's chosen synonyms which are generally worse. It's like it took a thesaurus and randomly substituted words.

There's actually a very good reason for this, and you can control it. To do that we're going to need to go in deeper into how chatbots work.

#

So, a slight correction to my previous description. When the bot looks at the text its been given, it doesn't actually pick one word to go next. They actually suggest a small number of words most likely to come next, and then select one to be the next word.

There are a few things which go into this decision. The easiest to understand is its 'temperature.' A low temperature bot will be very conservative and will tend to choose the favourite from the proposed selection. When they're like this, they tend to be repetitive because they don't have much randomness. A high temperature bot will choose from less likely options. These are less repetitive, but can be weirder. At the time of writing, ChatGPT doesn't let you tweak this directly (it used to), but you can ask it to provide answers with a lower or higher temperature. I imagine this customisation is something that will disappear completely from the chat interface, like how Google removed a lot of the fancy search operators, though.

Then there is another, more important parameter that also affects ChatGPT: its so-called 'diversity penalty.' This is a way of making its output interesting. ChatGPT is penalised for echoing, so previously selected words are less likely to be chosen from the candidate set. Now, this is great for getting rid of echoing! That's why it did such a good job of removing 'was'.

But remember how it doesn't really distinguish between your request and its answer when it comes to generating the next word? All it sees are sequences and probabilities of next words. So that ability to reduce echoing really gets in the way with editing; we want it to spit out some of the same text, but it is constrained to not repeat itself.

Will this be a problem forever? Maybe not. These things are being evolved very quickly. I imagine you might see better results in the future. Again, note that it used to let us tweak this parameter, but now you can't, so the makers of ChatGPT are confident that it mostly gets it right. I would say for line editing, not so much.

But: rule one. Just because ChatGPT suggests a different word, doesn't mean its better! It might be because it's trained on bad writing, choosing a random less-optimal word, or it might be changing your word just to avoid echoing itself.

#

Let's go back to another thing we mentioned, the quality of the corpus. I asked ChatGPT for some adjectives. This is our conversation. (For full disclosure, note that I elided some of its responses because it wanted to tell me what adjectives were and how to use them and that didn't seem very useful to repeat; then I reformatted its response to make it easier to read. The substance is unchanged, though.)