We inherit the word “induction” from logic and philosophy. Let us discuss the general vocabulary around this choice, especially with regard to the interpretations coming from contemporary work in machine learning.

Logically and philosophically, inference might be understood as the general process of drawing conclusions by reasoning. Meanwhile, each of logical deduction, induction, and abduction represents a particular kind of inference. Logical deduction involves drawing conclusions which are logically necessary. Logical induction involves inferring a general rule explaining some given observations. And logical abduction involves inferring from an observation a probable cause or explanation for it.

These terms have been repurposed and reinterpreted throughout the millennia. For instance, mathematical induction is a proof/axiom scheme employed in mathematics to conclude that a property holds in the limit over some nicely ordered set so long as it holds once and satisfies some productive rule like “holding over one instance implies holding over the following instance.” Yet, mathematical induction really constitutes an instance of logical deduction, for the conclusion it reaches is actually logically necessary under mathematical axioms. Still, calling this method induction isn’t too unreasonable: in effect, mathematical induction concludes a general rule from only a few cases, matching the goal of logical induction.

Inference has recently taken on a narrower meaning in the context of machine learning. We might now understand inference as the process of LLMs responding to queries with some version of autoregression, assigning probabilities to (and then, via some rule, predicting) next tokens to continue generating its response. While this process certainly involves reasoning and so indeed qualifies as logical inference, it hardly captures the breadth of alternative methods for using reasoning in responding to queries. So, we might prefer to call this autoregressive process: response inference.

Now we’ll cast template induction in more precise terms. Butter’s template induction pipeline employs models (which perform response inference in their own right) all to help separate dynamic content from structural content in user queries. It claims that the produced query and response templates accurately capture how to respond to any followup queries matching the format of that query template. This process effectively constitutes an instance of logical induction, seeing as how it essentially does one-shot or few-shot generalization on a collection of messages. This is how we justify the term template induction.