expectedwrong hindsight

You Spent a Weekend Building What Salesforce Ships in the Box

On the particular joy of reinventing enterprise software from scratch and then finding the receipt.

1 min read 209 words #ai #salesforce #llm #nlp #gpt-4
hindsight — still happening

the pattern of building what the platform ships for free accelerated dramatically. every major platform added natural language interfaces to their query languages. the four-minute satisfaction window before someone says 'salesforce already has this' never got any longer.

I built a pipeline that introspects a Salesforce lending engine — all the custom objects, their fields, their relationships — feeds ~10k tokens of schema description through GPT-4, compresses it down to a 500-token instruction set, and uses that to translate natural language into SOQL queries that run directly against the org.

Zero-shot. Almost perfect. No hand-tuning, no few-shot examples, no retrieval layer. Just schema in, English in, query out, results back.

It works — which is genuinely satisfying for about four minutes.

Then someone tells you Salesforce has this built in.

Not a third-party plugin. Not a beta feature buried in a sandbox org. Out of the box. Einstein, or whatever they're calling it this month. Natural language to SOQL, ship it, done.

The thing is, the pipeline I built is exactly what you'd design if you were Salesforce and you had access to a capable LLM and needed to solve this problem. Compress schema, constrain the query space, execute — it's obvious in retrospect, which is how all correct designs feel once someone else has already shipped them.

What I actually built was a proof of concept for a product that already exists, which is a genre of engineering outcome I'm getting more familiar with every month.