Will LLMs abstract away humanity?
David Peterson

Our hearts go out to the people of Israel as the depth of Hamas’ cruelty continues to be unearthed. As Gil wrote in his note last week (which I urge you to read, if you haven’t had a chance yet), you didn’t sign up to this newsletter for our thoughts on politics, but there are some moments that demand continued attention, and this is one of them.

Now back to our regularly scheduled programming:

I’ve been speaking with many founders lately who are dealing with the trials and tribulations of building with LLMs. And I’ve noticed a pattern.

From the outside, there’s this sense that software branded as being “powered by generative AI” is fundamentally different from the software that came before. But the reality is that many of the table-stakes features required for any product have nothing to do with AI. Invite flows, CSV importing, activity feeds, etc. All this stuff still needs to be coded up just like before.

That being said, LLMs are scrambling the instinct many founders have for what problems to solve locally with code and what problems to solve with AI.

As an example, it’s probably true that your product’s invite flow should be built the old-fashioned way, but what about the CSV import? CSV import doesn’t initially scream “LLM” to me, but I bet field mapping could be handed off to an LLM and create a smoother and simpler experience that’s less prone-to-error as well.

We’ve been building some AI-powered tools for our own firm’s operations internally and have faced our own version of this challenge as we’ve tried to throw LLMs at every problem we
can think of. Creating themes for companies is a pretty good use case for LLMs, but building smarter company search? Well, that’s still just a search problem…no matter how much we want another excuse to play around with AI.

What, then, is the intuition for when to lean on an LLM and when to keep the logic local?

This dynamic reminds me of the emergence of API-first companies over a decade ago. With the rise of API-first companies, all of a sudden developers began to rely on third-party APIs for solutions that were previously coded locally.

The pitch for any API-first company was simple. If they could abstract away something complicated behind an API, developers would happily pay to hit that API, receive a standard result, and never have to think about that complexity ever again. Twilio did this with telephony, Stripe did this with payments, Shippo did this with shipping…all famously complicated, old-school, industries.

It’s easy to think of LLMs as a continuation of this model. So if API-first companies are about abstracting away complexity, you might argue that LLMs are about abstracting away humanity.

(However dire that sounds…)

That’s what I thought, at least, until I read this interesting thought experiment from Martin Casado of a16z related to writing games with AI NPCs. What logic should live in the game engine, and what logic should be outsourced to LLMs, Martin asked?

Backstories and conversation…those are all things that used to be painstakingly coded by hand, but the results were still staid and stilted. With LLMs, AI NPCs have a real shot at being dynamic, deep and additive to the game.

In other words, by leveraging LLMs, game developers aren’t abstracting away humanity, they’re adding it back in at a scale, and at a fidelity, that was previously impossible.

So the next time you’re thinking about whether to use an LLM or not, don’t think about abstracting away humanity…think about adding it back in. Where do you wish you could sprinkle a bit of creativity? Where would you like to add in a layer of judgment? That’s where you may want to outsource to an LLM.

Until next time,
David

FROM THE BLOG

No Words
The heartbreaking situation in Israel.

2023: The Crucible Year
What is in store for the next 18 months?

Kafka Survivors of the World, Unite!
Why we backed Memphis.dev.

The Problem with Startup Advice
The best founders and investors know the rules, but also know when to break them.

EUROPE & ISRAEL FUNDING NEWS

Israel / Databases. ScyllaDB raised $43M to scale its NoSQL database platform optimized for high-throughput, low-latency workloads.

UK / Fintech. Kennek raised $12.5M to grow its operating system for private debt market lenders.

France / Insurtech. Orus raised $11.6M to scale its insurance platform targeting long-tail SMBs, starting with restaurants.

Germany / SaaS. Deta raised $3.6M to build its “personal cloud computer.”

WORTH READING

ENTERPRISE/TECH NEWS

The high cost of AI. Paul Thorrott took a look at the high cost of generative AI in a thought-provoking post: “Individuals pay $10 per month for GitHub Copilot, but multiple sources told it that the service loses an average of $20 per user per month, with some users costing Microsoft as much as $80 per month. So it’s likely that this situation played a role in the company’s decision to charge a lot more for the AI capabilities it will soon provide via Microsoft 365 Copilot. That service will cost customers $30 per user per month on top of the normal monthly Microsoft 365 subscription fee (which varies by tier). It’s not coincidental that Google will charge an identical additional per-user fee for its similar Duet AI offering. The extravagant cost of AI also explains why Microsoft is working to develop its own in-house AI chipsets for use in its datacenters and is pushing the PC industry to adopt what it calls NPUs — Neural Processing Units that accelerate AI operations independently of the CPU — that will usher in the new era of PC computing that HP discussed last week: These coming PCs will be able to offload some AI tasks from the cloud and process them locally, reducing the expenses on the backend.”

A techno-optimist’s manifesto. Marc Andreesen published a lengthy and upbeat manifesto on why he’s an eternal techno-optimist. Despite the challenges we face, this is a voice we need to hear. He starts with the observation that “there are only three sources of growth: population growth, natural resource utilization, and technology,” and he moves on from there. It’s worth a read. This passage struck me particularly this week: “We believe America and her allies should be strong and not weak. We believe national strength of liberal democracies flows from economic strength (financial power), cultural strength (soft power), and military strength (hard power). Economic, cultural, and military strength flow from technological strength. A technologically strong America is a force for good in a dangerous world. Technologically strong liberal democracies safeguard liberty and peace. Technologically weak liberal democracies lose to their autocratic rivals, making everyone worse off. We believe technology makes greatness more possible and more likely.”

Towards decentralized identities? We continue to look for solid use cases of web3/crypto. While we have yet to see many in the real world, efforts to get there persist. This piece on decentralized identities by Michal Trojanowski makes the case for decentralized identity documents such as passports and tickets. With decentralized identities, “there is no direct link between the verifier and the issuer, only a trust relationship. This is a crucial privacy enhancement compared to present solutions based on federated logins because the issuer does not know how or when the holder uses the credentials. The issuer thus cannot collect data about the user based on their online behavior. Moreover, with proper implementations, the holder will have much more control over their data and how it is shared with the verifier. The user can share only a subset of identity data or even use zero-knowledge proofs (ZKP) to cryptographically prove some identity information without revealing any data.” Wei Lien Dang of Unusual Ventures believes there is opportunity in the LLM infra stack as well: “We’ll see more startups providing tooling for LLM deployment that doesn’t require a dedicated AI infrastructure team, too. This is what’s driving the idea of an “AI engineer” — AI will have its largest impact when the 30 million+ developers out there can easily use LLMs to build applications. OpenAI, Anthropic, Cohere, and others have set a certain expectation among users around making it easier to get started with LLMs.”

HOW TO STARTUP

AI startup strategy. Techcrunch Plus interviewed a few VCs on how startups can capture and defend market share in the AI era. Rick Grinnell of Glasswing Ventures is focussed on the application layer: “We think that most of the opportunity lies in the application layer, and within that layer, we believe that in the near future, the best applications will harness their in-house expertise to build specialized middle-layer tooling and blend them with the appropriate foundational models. These are “vertically integrated” or “full-stack” applications.”

Tips for converting SaaS homepages. Kyle Poyar at Openview offers some practical tips. For example, Kyle suggests focusing on a champion as opposed to a decision maker in many cases: “A common mistake is writing the page for the buyer or decision maker — an executive who is higher in the org chart. While they have authority, in most cases they are too far removed from the problem to care about your product. Most B2B sales cycles are 6–12 months long, which means you want to pick a person who has the time, energy, and motivation to convince all relevant stakeholders, work with legal and procurement, and ultimately get the deal pushed through. This person is known as the champion, and they are the ones you need to write the page for. The champion isn’t always the end user, but they do need to be close enough to the problem to see its negative effects and understand how a product would solve the issue.”

HOW TO VENTURE

General Catalyst just bought La Famiglia. The US VC General Catalyst announced that it would acquire La Familigia and convert it into its European seed investment arm. Who says VCs can’t exit? “La Famiglia, an increasingly active seed investor out of Berlin, is to become a seed investment arm, and — at least for now — retaining its brand as such, aligning with General Catalyst, a VC with a 20+ year history and a significant European investing footprint. In practical terms, La Famiglia’s existing fund will run its course, and new seeding investing for La Famiglia will be done as if it’s a seed arm of General Catalyst.”

PORTFOLIO NEWS

Aquant’s CEO, Shahar Chen, shared why it’s important to diversify the voices contributing to the AI regulatory conversation by including vertical AI vendors.

Snyk and Nightfall AI announced a strategic partnership to offer developers state-of-the-art AI-powered secrets scanning capabilities.