Prompt Engineering Guide

When creating prompts, there are a few parameters that users can adjust to get different results from prompts. Tweaking these settings is important to improve reliability and desirability of responses, but it takes a bit of experimentation to figure out the proper settings for use cases. Below are the common settings encountered when using different LLM providers:

• Temperature: Lower values lead to more deterministic results, with the highest probable next token (or word, number, etc.) always picked. Increasing temperature could lead to more randomness, encouraging more diverse or creative outputs by essentially increasing the weights of other possible tokens. In other words, a low “temperature” is going to provide the output that the LLM determines to be the most probable according to the data used to train the LLM. A high “temperature” is going to produce responses (i.e. outputs) that are more random and diverse. For example, the prompt “The rain made the road” might produce the output “wet” if the temperature is low. If the temperature is high, the output/response might be something like “difficult to handle curves, particularly on country roads.”
  • To adjust the temperature in prompts for ChatGPT and Copilot, add this information to the prompt. For example, “Provide recommended readings for an introductory American History class, doing so with a low temperature.”
• Max Length: Users can manage the number of tokens (or words, numbers, etc.) the model generates by adding specifics in the prompt. Specifying a max length helps prevent long or irrelevant responses and helps to limit wasted compute power.
  • To adjust output or response length, add clarity to your prompt. For example, “Provide recommended readings for an introductory American History class but keep the response to 100-150 words.”

A prompt can contain any of the following elements:

• Instruction: a specific task or instruction the user wants the model to perform
• Context: additional information or context that guides the model towards producing better outputs/responses
• Input Data: the input or question to which the user is requesting a response
• Output Indicator: the type or format of the response/output

Here is an example of a prompt with these elements at work:

• Prompt: Classify the text into neutral, negative, or positive. Text: I hate eating stale bread. In words, tell me what the classification is?
• Response: The classification of the text “I hate eating stale bread.” is negative.

In this example of a prompt engineered to produce a sentiment analysis, the prompt had instruction (Classify the text into neutral, negative, or positive), input data (Text: I hate eating stale bread), and an output indicator (In words, tell me what the classification is).

No context was added as the language model used (ChatGPT 3.5) had sufficient training data to produce a reliable -and contextually accurate- output without any overt context necessary in the prompt. Users experimenting with other models will need to pay attention to the outputs and when/if necessary, add context in the prompt input.

Start simple when you dive into crafting prompts. It’s a process that involves a lot of trial and error to find what works best. Begin with straightforward prompts and gradually add more elements and context as you aim for improved results. Iterating on your prompt as you go is crucial.

The examples used throughout this guide are very simple, but adding more detailed instructions, more context, more data/information, and clarity in the output indicator will yield better responses and allow for the kind of prompt engineering necessary to really leverage the power of these models.

For larger tasks with multiple subtasks, consider breaking them down into simpler components. This approach prevents overwhelming complexity in the prompt design process from the outset.

Use Clear Instructions

For straightforward tasks, use clear commands like “Write,” “Classify,” “Summarize,” “Translate,” or “Order” to instruct the model.

Remember, experimentation is key to finding what works best. Try different instructions, keywords, contexts, and data to determine the optimal approach for your specific use case. Generally, the more precise and relevant the context is to your task, the better.

Some experts suggest placing instructions at the start of the prompt. Another tip is to use a distinct separator like “###” to delineate the instruction and context. Models such as ChatGPT and Copilot operate well without needing a distinct separator in the prompt, but when trying out other models, using these separators may be helpful.

Here is an example with a distinct separator that was used with ChatGPT 3.5:

• Prompt: ### Instruction ### Translate “Hi” into Spanish.
• Response: “Hi” translates to “Hola” in Spanish.

Now without the separator (but also using ChatGPT 3.5):

• Prompt: Translate “Hi” into Spanish.
• Response: “Hi” translates to “Hola” in Spanish.

With ChatGPT 3.5, the training (and retraining) has reached a point where direct separation is not always necessary. Users will need to test out prompts depending on the model being used to figure out when and where these separators are needed.

Be Specific

When crafting prompts, precision is key. Provide clear, detailed instructions to guide the model effectively. The more specific your prompt, the better the generated results, especially if you have a particular outcome or style in mind. While there are no magic keywords for better performance, a well-structured and descriptive prompt is crucial. Including examples within the prompt can significantly enhance the likelihood of obtaining the desired output, particularly in specific formats.

Consider the prompt’s length, as there are limitations to how much information the model can process. Balance specificity with conciseness, avoiding too many useless details that don’t contribute to the task. Experimentation and iteration are essential for fine-tuning prompts to suit your needs. Embrace the process of testing and refining to optimize prompts for your specific applications.

• Prompt: Take these names currently listed as Last Name, First Name and re-list as First Name Last Name: Garcia, Joe; Smith, Tonya; Doe, Juanita
• Response: Sure, here are the names listed as First Name Last Name: Joe Garcia. Tonya Smith, Juanita Doe

Avoid Being Imprecise

When considering the advice on being detailed and enhancing format, it’s tempting to get overly creative with prompts, which can lead to vague descriptions. However, it’s usually more effective to be straightforward and precise. This is akin to effective communication: the clearer and more direct the message, the better it’s understood.

Let’s take an example:

• Imprecise Prompt: Explain ethos, pathos, and logos.
• Imprecise Response: Ethos: Ethos refers to the credibility or character of the speaker or writer. It involves convincing the audience of the author’s credibility, authority, or ethical stance on the topic. Ethos relies on establishing trust, credibility, and expertise to persuade the audience. This can be achieved through the speaker’s reputation, professional credentials, or presentation of themselves as knowledgeable and trustworthy. Pathos: Pathos appeals to the emotions and sentiments of the audience. It involves using emotional language, vivid imagery, and personal anecdotes to evoke feelings such as sympathy, compassion, anger, or joy. Pathos aims to connect with the audience on an emotional level, appealing to their values, beliefs, and experiences. By tapping into the audience’s emotions, speakers can elicit a stronger emotional response and effectively sway their opinions or attitudes. Logos: Logos refers to the use of logic, reason, and evidence to support an argument or claim. It involves presenting facts, statistics, logical reasoning, and rational arguments to persuade the audience. Logos aims to appeal to the audience’s intellect and critical thinking skills by providing logical and coherent arguments supported by evidence. This can include citing research studies, expert opinions, historical examples, or logical deductions to build a persuasive case.

Time to clean it up:

• Precise Prompt: Explain ethos, pathos, and logos. Do so in one sentence per element and do so in a way that can be understood by a high school student. 
• Precise Prompt: Ethos is about showing that the speaker is trustworthy and credible, like when a teacher uses their experience to explain a topic. Pathos is appealing to the audience’s emotions, such as telling a sad story to encourage donations to a charity. Logos is using facts and logic to make a point, like when a scientist presents data to support a theory.

For additional examples on basic prompting, check out this video:

Simple prompts can produce valuable results (i.e. outputs), but their effectiveness depends on the information provided and how well the prompt is crafted. A prompt typically includes instructions, questions, context, inputs, or examples to guide the model effectively and enhance result quality.

Here is a basic example of a simple prompt:

• Prompt: The road is
• Response: a symbol of journey, progress, and exploration. It represents the path we take in life, leading us towards our goals and aspirations.

But even that simple prompt can be improved depending on the user’s needs:

• Prompt: Complete the sentence as if it were going to be used in a children’s book: The road is 
• Response: like a long, winding ribbon stretching far and wide, inviting us to explore the wonders that lie beyond each bend.

Adding a bit more context in the prompt (i.e. needing a response that could be used for a children’s book) clarifies the request and allows the LLM to produce an output more specific to the request.

This process of tuning a prompt (i.e the input) to give the right instructions to the large language model (LLM) and make a valuable response (i.e. output) possible is called prompt engineering. However, just as this prompt was focused on generating words, prompts can be engineered to handle mathematical reasoning, code generation, and text summarization.

This form of prompting is known as zero-shot prompting as the user is prompting the model without including any examples or demonstrations about the task you are asking the model to complete. The ability for an LLM to handle zero-shot prompts is dependent on the model’s design and knowledge (i.e. the data it has been trained on). In other words, experiment with different models to see how they handle zero-shot prompts.

Few-shot prompting involves the inclusion of examples (sometimes called demonstrations) in the prompt. A user would use few-shot prompting when the examples provide the context or framework they want the LLM to use when creating a response.

Here is an example of a few-shot prompt:

• Prompt: Rain is bad is “negative.” Sun is good is “positive.” Kittens are awesome is “positive.” Knowing this sentiment, what is Dogs are amazing?
• Response: Based on the sentiment provided, “Dogs are amazing” would be categorized as “positive.”

In this example, the prompt established the rules for sentiment analysis with these rules then used to appropriately answer the question, “What is ‘Dogs are amazing’?”

Few-shot prompts make in-context learning possible for the model, giving the language model the ability to learn the task being prompted based on the demonstrations/examples added to the prompt.

Now that the basics have been covered, we will turn to DAIR.AI, a non-profit committed to “democratizing AI research, education, and technologies” and “enable the next generation of AI innovators and creators,” for information on additional prompt engineering approaches.

Chain of Thought (CoT) Prompting

Zero-Shot CoT Prompting

Automatic CoT Prompting

Auto-CoT consists of two main stages:

1. Question clustering: partition questions of a given dataset into a few clusters
2. Demonstration sampling: select a representative question from each cluster and generate its reasoning chain using Zero-Shot-CoT with simple heuristics

Here’s an example:

For those looking to experiment more with advanced prompting, DAIR.AI has assembled some fantastic examples applicable to all models.

Check out the Techniques area of their prompt engineering wiki.

Once ready to take things up a notch, DAIR.AI has also developed excellent how-to videos and guides on leveraging prompt engineering for data analysis, including support on how to use models to evaluate and assess data, how to visualize data, and how to handle unstructured data.

Both the Applications and Prompt Hub areas of their wiki offer step-by-step guide on using prompts to gain valuable insight into data as well as how to do things that prompt engineering for coding (a great place to start for the most inexperienced of coders).