A few months ago I had posted about GenAI for Data Analytics and the challenges of using something probabilistic to return something deterministic. Here are the 2 articles - Link 1 and Link 2. For the most part, I have stayed away from GenAI on structured data but I am getting on a call with a dear friend and former colleague John Jensen today to talk about Zenlytic. I do feel that GenAI on Structured Data will unlock a ton of new applications and was curious to learn more before my meeting.
I came across a case study on how Pinterest was using GenAI to do Text to SQL, written by Lewis Walker ➲ that was quite illuminating. They talked about how they went from 40% accuracy to 90% accuracy by moving from just LLM to LLM + RAG. For those interested, here are a couple diagrams that talk about their approach.
Here is the text that describes the approach
Here is the 2nd approach -
Here is the text describing the approach
While this case study gave me a high level understanding of the approach, I was not sure I completely understood it. I am a learn by examples kind of person. If someone gives me an example I can really understand the topic. But the case study did not have an example. So I asked GenAI to explain this to me with examples.
This is the output it came up with - https://docs.google.com/document/d/1JRwWGjShq4NXzupOC3xLki_q_nMQFhJ7Nrung-0kCgo/edit?usp=sharing
This was very helpful as it really broke down the process in a way I could understand it. What is fascinating is that the author of the case study gave me some information and then GenAI was able to uses it's knowledge to extrapolate and expand on it.
I've always wondered how Text to SQL works and seeing some of the code examples also helped a lot.
So next time someone talks about using LLMs to generate SQL queries, I can be much more knowledgeable and also, now I will always use GenAI to explain things to me with examples.
The question still remains - if it is 90% accurate, what happens with the 10% errors? Another question for GenAI.