Vibe Prompting - A Two Part Series is sponsored by Agent.ai - Discover, connect with and hire AI agents to do useful things.
I recently came across a post from Andrej Karpathy where he talks about this concept of "Vibe Coding". To paraphrase what he talks about -
I just see stuff, say stuff, run stuff, and copy paste stuff, and it mostly works. Where you fully give in to the vibes, embrace exponentials, and forget that the code even exists.
This is something I can totally relate to as I have had ChatGPT generate python code, Google AppsScript code etc. and I pretty much follow the same "vibe coding" experience that is described above.
There is a whole separate discussion we need to have about whether this is making us lazy / dumb. On one hand I am punching above my weight when it comes to coding, but on the other hand I am not learning anything about programming. That being said, I am solving problems that I would never have solved before. So in some ways, I am becoming a really good manager. I am explaining what I want, I validate the output but I am completely delegating the 'doing of the work'.
This is "Trust but Verify" in action
As I was thinking about Vibe Coding, I realized that when it comes to building agents, I have been following this approach in coming up with prompts. I see a lot of posts that talk about prompt engineering and I strongly feel that 'prompt engineer', as the job that we know today, will go the way of the people who were really good at writing search queries for AltaVista and Yahoo, before Google came along and let anyone who could type in a question, get the answer they want.
Part of the reason I wanted to write this newsletter was because so many people are still writing prompts by hand and I know a lot more people are building agents now (I get this if you have simple prompts, but if you are building agents and you want really good prompts, that is a sub optimal way).
To make this real, let me share an example of how my "Vibe Prompting" approach works for building agents. The agent I am trying to build is one that will take any CSV as input (P&L, Expenses, Fitbit data) and then analyze it.
While I said Prompt Engineer as a job will go away, and people who are writing prompts by hand are taking a sub-optimal approach, I should clarify that while I am still writing prompts, they are not your typical prompts.
So here are the prompts I used to build my agent -
Without going into every detail of what it came back with, here is the high level summary -
Yes, you can build such an agent! The process would involve several key steps:
Understanding the File Structure and Content
Asking Clarifying Questions
Conducting Analysis
Returning a Detailed Analysis
What stood out is that I would have started with Step 2 (if I had come up with the steps).
Here was my 2nd prompt and I will paste all my prompts (I am not pasting all the responses as they will take up too much space) so you can see how different this is from traditional prompting and it talks to the concept of Vibe Prompting and me having a conversation with ChatGPT to generate the prompts -
Prompt 2 basically takes the first step in the process that ChatGPT had come up with and Prompt 3 is a clarifying question based on the output of Prompt 2.
Any time the prompt is a bit off I ask it clarifying questions. I even go back to Prompt 2 later in the process to make a modification.
As I review these prompts, what stands out to me is that I am having a conversation with ChatGPT on the job I gave it and I am reviewing the output and asking questions (all those years of learning how to be a coach and not just tell people what to do, are finally paying off) and it gets to agree or disagree with me and when we differ we can have a conversation about our different approaches and we both learn.
The other learning here is that for someone to build this agent and to use this prompting approach, one needs to be a subject matter expert. By looking at the prompt you need to know what the gaps are. I could look at Prompt 3 and know that is should break it up into 2 etc. So the skill one needs is to understand the domain and be a good manager.
Maybe this is overkill if you have simple needs but this approach is especially useful when one is building agents. The reason is because agents are digital workers that need to consistently execute the same task again and again and produce repeatable results.
If we take our example of reading any CSV file and analyzing it in a generic way, there are a series of steps that need to take place and they are all combined together in an agent. Here is the agent in Agent.ai.
Steps 2, 6, 7 and 9 are the 4 prompts that ChatGPT generated.
If you want to see the output of any of these prompts, here is what Step 9 (the final prompt) looks like. You will see that ChatGPT does a good job of following prompting best practices (those skills that you learn in prompt engineering classes). If you've seen examples of people who write these elaborate prompts, now you know how they do it -
This is the first part of the newsletter and in the second part, I will get into how I go about debugging these prompts till I have the output the way I want it. I don't want people thinking that we are done. All we have done is built the initial prototype. The next step is to test the agent with real data and that is when we will go through a whole process of making these prompts even better and targeted.
So my call to action for anyone who has got this far is to go sign up for Agent.ai and start building some agents and play around with some vibe prompting.
