How LennyKache came to be

Last updated: September 9, 2026

Two weeks after launching IdeaKache, I was sitting at my desk on a Friday afternoon, thinking about what I do next. I went on X and saw a Tobi Lütke post in my feed, which isn’t uncommon because I’m a follower and a long-time admirer of his thinking.

In that moment, I had an idea. I remembered how Tobi originally started an online snowboard shop and then used what he learned to build online shops for others. I wondered whether it would be possible to do something similar with IdeaKache. I had no idea if it would work, but thought it was something worth trying.

The best idea I could think of was to use the transcripts from Lenny’s Podcast. I knew he’d open-sourced many of his podcast transcripts, and I thought that was a great first candidate to try because this podcast met several criteria that were really interesting to me:

• Lots of episodes;
• Thoughtful and talented guests; and
• A large audience.

With those criteria, I felt confident that the ideas in the transcripts were going to be good, and so the question was just whether I could extract them in a way that was interesting to people.

Starting with a download of Lenny’s transcripts, I needed to make some minor changes for this to work. I had Claude make a working copy of every transcript (one file per episode) and store that in my LennyKache folder.

1. In each working copy, I grouped the transcript into individual ideas rather than segments marked by timestamps. That required stitching consecutive segments from the same speaker into a single turn. I did this because I’m trying to find individual ideas, and an idea doesn’t necessarily stop at a timestamp. Timestamps were kept separately so that each quote, each idea in LennyKache, still links to the moment it starts on YouTube, where that was possible.
2. I skipped the short teaser at the start of each episode. This was typically the first couple of minutes. The teaser is often edited highlights of lines said later in the conversation and I thought there was a reasonable chance it could cause mistakes or double entries.
3. I wanted to run some tests to make sure the lines attributed to a person were said by them. I measured Lenny’s share of the words in every interview. Roughly a quarter is typical, but for five transcripts, that number was much higher, sometimes more than half. In those cases, the language model set up to run through those five transcripts was told the labels could not be trusted, so it had to decide who was speaking from what was being said.

After going in circles trying a bunch of things that used a lot of tokens and weren’t good enough, I did a calibration, which helped a lot. I read the Marc Andreessen, Ben Horowitz and Tobi Lütke episode transcripts and then manually selected the entries. I picked those three because I’ve listened to many of their interviews and feel like I had a well-calibrated sense of what was worth keeping.

Comparing the results from those three episodes with some of the different methods I tried, such as an embedding model, a panel of judges scoring every candidate against a written standard, and a language model reading each transcript, the language model worked best. On those three episodes, a single reader found about 8 in 10 of my picks, and three readers running independently found 24 of my 26.

The embedding model could recognize entry-shaped writing really well, but couldn’t tell where I’d cut. The judges could score entries really well, but when I tested their scoring against the entries I’d keep or trim, the results weren’t much better than chance.

The language model worked by reading each transcript from start to finish, with 50 real entries from IdeaKache, chosen to span the range. The idea was that these examples carry the standard rather than setting arbitrary rules.

The fifty entries sat in one file in my LennyKache folder, each with its type, title, author and text. A second file contained a description of what the IdeaKache entries have in common, such as median length, only being one idea and how often they stop mid-sentence rather than at a full stop. This was measured against the ~2,500 entries in IdeaKache. A third file held the quoting rules mentioned above, to work from the stitched turns, skip the teaser and take the span that carries the whole idea.

I started with 8 agents, and then moved up to 12, through Claude, that operated in parallel. Each agent took one episode to itself. It opened those three files and that episode’s working copy at the start of its job, then read the transcript from beginning to end.

Once the agents finished going through the transcripts for all 300+ episodes, the first cut was 10,954 entries. That’s too many to manage, so the next job was to order each episode’s potential entries.

To do the ordering, I went back to the three episodes I had read (Andreessen, Horowitz and Lütke). The agents had also been over the same three episodes, and between them they had found a lot more potential entries. Matching what I picked against what the agents found left two piles and the difference was the threshold line in my head for what was good enough for LennyKache.

I gave an agent both piles and asked it to work out that difference for itself before ordering any entries. Each agent wrote a different rule but they were often in the same ballpark (see two examples in the box).

Based on the rule it created, each agent then took its own episode’s candidates and put them in order, with the most likely to be picked first. To test the ordering, an agent used two of the Andreessen, Horowitz and Lütke entries and tested it on the third episode (so three separate runs, one at a time). Two thirds of my picks came back in its top 20, which was a reasonable start.

Scoring each entry was next. I didn’t think scoring ~11,000 entries was reasonable and so picked a 40% cutoff. This was art and a bit of a guess. 40% of a median 33-candidate episode was 13 and I knew most episodes wouldn’t have 13 entries. From past experience, a truly stellar podcast might have 15 entries for IdeaKache and most have fewer than five.

The 40% cut, or 4,513 entries, were scored out of 100 against the same two piles from before, the entries I chose from those three episodes, and everything else the agents found in those same episodes. The scoring was done by an individual agent looking at 40 potential entries at a time.

A quote that belongs beside the ones I picked scores say 80 to 95, one that reads like the ones I passed scores say 30 to 50, and anything unusable on its own scores below 20. The scores came back spread from 5 to 96 with a median of 58.

Separately, I scored 120 entries myself, which were spread across the score range. Originally I did a keep or cut but that was too noisy given a bunch of borderline entries. I did a follow up on 120 entries where I scored each from 1 to 5. For the scoring, 1 is bad, 3 maybe just makes it, 4 is good and 5 is great.

My scores of 1 to 5 could be compared against the agent’s 0 to 100 for each of these 120 entries. To decide the score where I “snap the line”, I wanted it high enough that I would keep 4 of 5 entries for LennyKache. That score landed at 60.

Snapping the line at a score of 60 gave 2,157 entries. That, plus the 35 I’d chosen from the three episodes myself, came to 2,192. A speaker attribution check removed 13 entries. That left 2,179 entries, from 309 episodes and 305 people.

With the collection of entries “final”, I checked 40 random entries I hadn’t scored before. I scored them from 1 to 5. There were no 1s or 2s and 80% of them were at 4 or better.

A final test, out of curiosity more than anything, was run on the transcript of David Senra’s interview with James Dyson (it’s great). I’d already added those entries to IdeaKache months ago. An agent followed the same language model process as for Lenny’s podcast transcripts and selected the quotes it would take. A single reader caught around 10 of the 14 entries I’d collected.

Lastly, I ran the same language model on two new episodes that arrived after I’d done all the above. LennyKache now has 2,205 ideas from 311 episodes with 307 guests. It’s good. Go through it and I promise you’ll learn something new.

If you have any feedback or questions, please email Jeremy at jeremy@gemka.co