What is automatic prompt engineering?

Based on the sources provided:

Automatic Prompt Engineering (APE) is a method that automates the intricate process of prompt creation or prompt engineering. It works by harnessing the capabilities of Large Language Models (LLMs) themselves to generate, evaluate, and refine prompts.

The methodology involves a series of distinct yet interconnected steps. Initially, the LLM produces a variety of prompts tailored to a specific task. These generated prompts then undergo an evaluation phase, where they are scored based on key metrics. Subsequently, the prompts are refined and adjusted based on their scores in an iterative process. This process, illustrated as having a cyclic nature of prompt generation, evaluation, and refinement, aims for continuous improvement in prompt quality. One specific approach uses a set of exemplars to generate a Zero-Shot instruction prompt, generates multiple prompts, scores them, and then creates variations (e.g., using prompt paraphrasing), iterating until desired outcomes are met.

By automating prompt engineering, APE aims to optimize the prompt design process. It not only alleviates the need for human input and the burden of manual prompt creation, but also enhances the model’s performance in various tasks by introducing a level of precision and adaptability.

For example, you can use APE to help train a chatbot for a merchandise t-shirt webshop by figuring out the various ways customers might phrase their order. Innovations like APE could become standard practice in the future.