Human in the loop (HITL)- best practices for agentic worklows

Human in the loop gives the best of both worlds. Best practice in agentic workflow is testing, verification,quality assurance. You name it. Workflow agents are automatons and can significantly enhance productivity, but their true potential is unlocked when combined with a human-in-the-loop (HITL) approach. This ensures that the agent’s actions are verified and refined by human expertise, leading to better outcomes. In addition it provides a feedback loop for learning and enchancing workflow processes.

Why Use Workflow Agents with Human-in-the-Loop?

Enhanced accuracy

Agents may (or may not 🙂 ) on occasiona misinterpret instructions or context and provider infered data that is not what we wanted it to. Human verification ensures that errors are caught early and adjusted for the future maintaining high accuracy and instant improvment.

Collaboration

The Human in the loop approach provides collaboration between humans and the machine. Humans provide context, judgment, and creativity, while agents handle repetitive or computationally intensive tasks.

Continuousi improvement

By review and verification work of the agent’s actions, humans can identify areas for improvement in the agent’s logic or instructions, leading to iterative enhancements.

In system we trust

Human oversight builds more trust in the system. Ensures accountability and responsibility by allowing to intervene and correct the agent’s actions when necessary. We need to have the possibility to intervene by defining proper steps.

Example workflow of Human-in-the-Loop

Example of a workflow where an agent assists in generating code, writing documentation and a human verifies and refines the output and or instructions.

Agents

This of course is a simple example. You can define most of the rules in Your copilot instructions or any AI agent prompts, skills and such. This can be automated by rules to work in semi automated mode or even fully manuall and accept every change in every file one by one only after a full solution is presented.
Remember You have to do it in regression.
Automation is Your ‘friend’ 🙂

Step 1: Define the Task – Prompt engineering

The human provides a clear and detailed description of the task for the agent. Prompt example using RICO approach :

“You are a full stack developer implementing a simple service for authentication.
Use context7 mcp server to get information about vue authentication methods.
Ask me to verify every step of Your work so we will accept it and refine the solution.
1. Generate a Vue.js composable for managing user authentication, including login, logout, and session management. Think about edge cases.
2. Create a simple test to check the generated code, it can be simple , more as a place holder for future refinment
3. Create a simple documentation that described the solution selected for authentication.
Iterate until the tests are working. When in doubt ask me, do not assume if You are uncertain.
In Addition:
If you need to use a 3rd party library ask me, provide some choice and descrbie why would i select a given library.”

Step 2: Agent executes the task

The agent generates the requested code based on the provided instructions. For example:

import { ref } from 'vue';

export function useAuth() {
  const user = ref(null);

  function login(credentials) {
    // Simulate login logic
    user.value = { name: 'John Doe', ...credentials };
  }

  function logout() {
    user.value = null;
  }

  return {
    user,
    login,
    logout,
  };
}

Step 3: Human verification

The human reviews the generated code to ensure it meets the requirements and adheres to best practices. For example:

• Check for Security issues: Ensure sensitive data is handled securely.
• Validate Functionality: Confirm that the code fulfills the task requirements.
• Refine Logic: Adjust the code to align with project-specific conventions.

Step 4: Ask for feedback and continue

The human provides feedback to the agent, either approving the output or requesting modifications. For example:

“The login function should include error handling for invalid credentials. Please update the code.”

Step 5: Iterate over steps

The agent refines the code based on the feedback, and the human verifies the updated output. This iterative process continues until the task is completed satisfactoriy.

Comparison of HITL Workflow with Fully Automated and Manual Approaches

Aspect	Human-in-the-Loop Workflow	Fully Automated Workflow	Manual Workflow
Accuracy	High – Human verification ensures correctness	Medium – Errors may go unnoticed	High – Fully controlled by humans
Efficiency	High – Combines speed of automation with human judgment	Very High – Fully automated	Low – Time-consuming
Flexibility	High – Humans can adapt to changing requirements	Medium – Limited by agent’s logic	High – Humans can adapt easily
Scalability	Medium – Requires human involvement	High – Fully automated	Low – Limited by human capacity
Trust and Accountability	High – Human oversight builds trust	Medium – Requires extensive testing	High – Fully controlled by humans

Best Practices for HITL Workflow

1. Define Clear Instructions: Provide detailed and unambiguous instructions to the agent.
2. Write a plan : Writing prompts already ? Write a plan with steps.
3. Set Verification Points: Identify critical points in the workflow where human verification is required.
4. Use Feedback Loops: Establish a process for humans to provide feedback and for agents to incorporate it.
5. Leverage Agent Strengths: Assign repetitive or computationally intensive tasks to the agent, allowing humans to focus on higher-level decision-making.
6. Document the Workflow: Maintain clear documentation of the workflow, including instructions, verification points, and feedback.
7. Benchmark : Benchmark the solution – test the code from AI logic perspective.

Conclusion

Workflow agents with human-in-the-loop verification offer the best of both worlds: the efficiency of automation and the accuracy of human judgment. By combining the strengths of humans and agents, organizations can achieve better outcomes, foster collaboration and continuously improve their workflows.

Human in the loop also lowers a bit the cognitive load. The person has a bit less amount of code to adjust per single outcome.

While fully automated and manual approaches have their place, the HITL approach is ideal for tasks that require both precision and adaptability.