The 10 Rules for Reliable Use of LLMs at Work

Generative AI tools, like LLMs (Large Language Models), have become indispensable in modern workplaces, helping us write reports, create code, brainstorm ideas, and more. To better understand how to use these tools effectively, we surveyed LLM best practices for the Savant community. The goal? To uncover what works best to avoid common pitfalls and maximize results. We compiled what we learned into a guide to avoiding the LLM hallucination black box and making Gen AI operate safely and reliably for you at work:

1. Give Context

The #1 mistake people make when using LLMs is writing prompts without context. We often don’t realize just how much tribal knowledge we rely on in our day-to-day work. Take a moment to write it down and share it with the LLM — it makes a huge difference.

• What to include: Explain your company, your role, and the project you’re working on. Provide key considerations for the task.
• Use examples: Show it examples of what good and bad outputs look like. LLMs learn from patterns in your instructions, so the more detailed and specific you are, the better.

Example prompt: “I work at a B2B SaaS company targeting enterprise customers. I need help drafting a blog post about our new product feature — focus on benefits for IT teams, not generic marketing speak. Here’s an example of our tone and style.”

2. Define the Role

LLMs thrive when you give them a specific role to act in. This serves as additional context to guide their outputs.

• Set the stage: Start your prompt by specifying the LLM’s “job” — this will shape its outputs in line with the expertise you need.
• Why it works: Giving the LLM a role adds structure to its reasoning and output style, making responses more aligned with your expectations.

Example prompt: “You are an IT consultant helping a company choose cloud storage solutions. Create a comparison table highlighting the pros and cons of three popular options.”

3. You’re a Dictator — Act Like It

Take control of the entire process by providing detailed and precise instructions. Treat the LLM like a junior employee who needs exact guidance to perform well.

• Write detailed instructions: Break down the task into clear, step-by-step directives, leaving no room for ambiguity. Instead of saying, “Summarize this document,” say, “Write a 200-word summary focusing on the three main arguments and include a key takeaway for each.”
• Control the outcome: Specify the tone, style, format, and audience to ensure the output aligns with your needs. This reduces the risk of misinterpretation and hallucination.

4. Eat the Elephant One Bite at a Time

When working with LLMs, taking things step by step is better than asking it to tackle a massive task in one go. 

• Why it works: Breaking tasks into smaller parts reduces the chance of errors or hallucinations. It also allows you to catch mistakes before they snowball.
• How to approach: Start with small questions or outputs. For example, instead of saying, “Write a full project proposal,” ask it to draft an outline first, then expand on each section one by one.

5. Iterate and Refine Prompts

Getting the best out of an LLM is often an iterative process. After the first draft or output, refine your instructions based on what’s missing or incorrect.

• Why iteration works: LLMs get better at aligning with your needs as you give them feedback.
• How to refine: Say, “I like this section, but it’s too technical for my audience. Simplify it and add a metaphor.”

6. Inspect the Reasoning Chain

Reasoning capabilities in LLMs are improving rapidly, but they’re still new and prone to unexpected hallucinations. If you’re using an LLM for complex reasoning tasks, always review its chain of thought.

• How to review reasoning: Most advanced LLMs can show their thought process or reasoning chain. Take a moment to read through these steps to spot errors early.
• Why it matters: Hallucinations often creep in during reasoning. A quick check of its internal monologue can save you from relying on a flawed output.

7. Understand the Abundance Paradox

The more mainstream a topic, the more generic the information an LLM will provide. Conversely, niche topics often lead to specific and high-quality information. But be cautious — if you ask about a niche topic and receive very general content, that’s a red flag for hallucination.

• Know the difference: Mainstream topics often surface generic summaries. If you need deeper insights, clarify your prompt. For niche topics, expect detailed and precise answers. Generalizations in these cases signal a potential hallucination.
• How to approach: Always sanity-check niche outputs by verifying against trusted sources or prior knowledge.

8. Include Examples of What You Want

LLMs excel when you provide positive and negative examples of the desired outcome. This ensures alignment with your expectations.

• Be clear: Reference the examples given in each recommendation in this article as a guide for crafting your own examples.
• How to use examples: Show both what you want and what to avoid. For instance, if writing an email, provide an example of a well-written email and a poorly written one.

9. Use Gen AI to Engineer Your Prompt

When in doubt, ask the LLM for help in crafting the best prompt for your needs.

• Example prompt: “Help me write a detailed and specific prompt to generate a blog post about the benefits of AI in education.”
• Why it works: The LLM can iterate on itself, helping you structure better instructions for your task.

10. Recognize What LLMs Are Bad At

Sometimes, even after multiple attempts, an LLM may fail to produce the results you need. When this happens, it’s important to stop forcing it — some tasks might simply be beyond the technology’s current capabilities.

• Adjust your expectations: If the LLM can’t handle a task, try thinking about other parts of the problem it can assist with. For instance, it might not solve a technical challenge but could help brainstorm solutions or structure a plan.
• Ask for advice: When in doubt, ask the LLM what it recommends. Prompts like, “What’s the best way to approach this problem?” can help identify areas where it might still provide value.

Final Thoughts

Generative AI is a powerful tool, but like any tool, it’s only as good as the person using it. By providing context, being specific, managing tasks step by step, and having a solid QA plan, you can avoid the pitfalls of hallucinations and get reliable, high-quality results. Treat your LLM like an intern: train it, review its work, and guide it closely.

This field is evolving fast, and so are the best practices for using LLMs effectively. What works today might be improved tomorrow. Sharing insights can help all of us stay ahead and get the most out of this transformative technology.