Without reading the linked paper, I thought to myself "How would I ask ChatGPT to run an addition program?" So I tried the following prompt:
> Imagine a finite state machine with two states A and B. The machine reads a long list of digit-pairs, one pair at a time, and depending on its current state and the digit pair, it outputs some digit and it transitions to a new state.
> When the machine is in state A and it reads a pair of digits, it adds together the two digits. If the sum is less than 10, it outputs the single-digit sum and it stays in state A. Otherwise, it outputs the least significant digit of the sum and it goes to state B.
> When the machine is in state B and it reads a pair of digits, it adds together the two digits and adds 1 to the sum. If the sum is less than 10, it outputs the single-digit sum and it goes to state A. Otherwise, it outputs the least significant digit of the sum and it stays in state B.
> With the machine in state A, and the two input digits being 4 and 4, what is the output digit, and what is the next state?
Unfortunately, ChatGPT failed to apply the transition rule correctly - its output:
> When the machine is in state A and it reads the digit pair (4, 4), it adds the two digits together to get 8. Since 8 is greater than 10, the machine outputs the least significant digit of the sum, which is 8. The machine then transitions to state B.
> So the output digit is 8 and the next state is B.
It seems like my approach is "instruction-only" if I try to follow the paper's nomenclature, but the paper doesn't seem to give examples of what their "instruction-only" prompts look like - those would be interesting to try to feed into ChatGPT.
> Imagine a finite state machine with two states A and B. The machine reads a long list of digit-pairs, one pair at a time, and depending on its current state and the digit pair, it outputs some digit and it transitions to a new state.
> When the machine is in state A and it reads a pair of digits, it adds together the two digits. If the sum is less than 10, it outputs the single-digit sum and it stays in state A. Otherwise, it outputs the least significant digit of the sum and it goes to state B.
> When the machine is in state B and it reads a pair of digits, it adds together the two digits and adds 1 to the sum. If the sum is less than 10, it outputs the single-digit sum and it goes to state A. Otherwise, it outputs the least significant digit of the sum and it stays in state B.
> With the machine in state A, and the two input digits being 4 and 4, what is the output digit, and what is the next state?
Unfortunately, ChatGPT failed to apply the transition rule correctly - its output:
> When the machine is in state A and it reads the digit pair (4, 4), it adds the two digits together to get 8. Since 8 is greater than 10, the machine outputs the least significant digit of the sum, which is 8. The machine then transitions to state B.
> So the output digit is 8 and the next state is B.
It seems like my approach is "instruction-only" if I try to follow the paper's nomenclature, but the paper doesn't seem to give examples of what their "instruction-only" prompts look like - those would be interesting to try to feed into ChatGPT.