A functional prototype for this idea is still WIP which will be available soon in my GitHub and uploaded to Streamlit Cloud. However, I’m sharing the idea here.

Background Link to heading

I was doing prompt engineering for a project and I found myself repeating the same steps over and over again. Here is what I was doing:

1. Wrote an initial prompt for a task using Claude 3.5 Sonnet.
2. Used that prompt to see if classifies/extracts the information as I wanted.
3. Looked the output and decided, it’s not working as expected.
4. Copied the wrong extraction back into Claude telling it to fix the prompt.
5. Repeat the same steps again and again until I was satisfied with the output.

Then I started to think, what if I can make this process more efficient? What if I can use a tool that can help me optimize the prompt? Why can’t I just show claude my what output I want to get from an input and tell it to keep optimizing the prompt until it’s working as expected with high accuracy?