Podcast

Dhaval:
This former meta engineer turned a hackathon product into an AI startup with 150% growth month over month. In this episode, we talk about her launch story, how she built the initial team, and her product development journey. Specifically, we discussed an interesting learning lesson around innovative approach she uses for data labeling and creating the golden data set for building your ML model. She shares a few examples of how product creators can get started with building ai. Without deep technical expertise, she also shares a tangible workflow that they can use to do so. Today my guest is Lily Chan. She’s a former software engineer at Meta. She’s the founder and CEO of Contenda. Contenda is an artificial intelligent tool. Reimagine your workflow in new formats for audience to discover with no extra work from you, and scale your existing technical content faster.

Welcome to the call, Lilly. Thank you for joining us. Tell us about your product.

Lilly:
Contenda Scales, technical content marketing for developer advocates.

Dhaval:
Okay. What does that mean? Tell me. Tell me, that’s very interesting. You got that. You nailed it. You nailed the position there.

Lilly:
I mean, the buzzword of the day is generative ai. We do fall in that category. Contenda uses large language models to generate technical content with a high degree of accuracy.

Dhaval:
Wow. Technical content is the hardest content to create of all the content categories, right? It’s very easy to spit out sales copy. But technical content that’s pretty challenging. So your audience is developer is that right? is a typical developer who wants to create documentation technical documentation. Is that your audience? Tell me a little more about your target user.

Lilly:
Our target users they’re called developer advocates, so they write content for other developers. They will oftentimes do a live stream, a lecture, or a conference talk, and then they need to transform that content into a written form, such as a blog tutorial and sometimes documentation.

Dhaval:
Got it. Yeah. So I, as a product person, I have to write a lot of documentation based on. Product launch based on the product demo. And that’s the type of a content, you help advocates with. Is that? Did I get there, right? Correct. Okay. Now, tell me where you are in that journey of your product. Is that something you have launched? Are you working on it, et cetera

Lilly:
When we first started building this product, we served everything through email and Google Docs. That meant that you can’t sign up on our website to use the product. You just had to email me personally. We spent all of our time building the machine learning and backend infrastructure to do that. Now we have reached the point where our interest form on our website has grown 150% month over month for over the past quarter and a half. And these hundreds of people can’t get on our platform unless we build one. So that’s what we’re currently doing, building a platform.

Dhaval:
Wow. Okay. So you have productized it. To your manual workflows, and right now, you are automating those workflows to create a user experience. How many users do you have? , is that, can someone use your product now? If they want to use it, how many users do you have? How much revenue do you have, et cetera?

Lilly:
We currently only do enterprise-level conversations. That’s something that we’re looking to change. We would love to get in touch more with developer advocates who have a personal brand and are thinking about using contender for their own Twitch stream or Twitter or blog, and then eventually get to a point where we can say, Hey, would you like to bring our product to work? Would you like to use it on the enterprise level?

Dhaval:
Wow. Okay. So would you say that you have any revenue at this point? Do you have customers that are using this at enterprise level? And, if so, like where does that sweet spotlight for you right now and in the future?

Lilly:
We do have a few enterprise customers. I won’t disclose what each person is paying. But the total ACV value is somewhere between 50 k to 100k plus.

Dhaval:
Got it. Is this something that you have bootstrapped? Have you built this on your own? Have you raised a round of funding? Tell us a little bit about how you got to it. Started it.

Lilly:
Well, funny enough, it actually started off as a hackathon project. I was working as a full-time machine learning infrastructure engineer at Meta, and I had just flown out some friends to California for a hackathon project. That hackathon project was for Twitch streaming. It was a retention project for Twitch streamers, and our project went viral. It ended up helping a Twitch streamer break a Guinness World record for most subscribers in a month. It went viral, and a couple of news outlets picked up on the story. So investors reached out from there, and that’s how we became a venture-backed business.

Dhaval:
Wow. So what was this hackathon project trying to do? You said something about retention for streamers. Tell me a little more there.

Lilly:
Right So our idea was to build something HubSpot esque, but for content creators on the individual level, Twitch streamers have really, really high churn on their subscribers, and so we wanted to build a product that could help a Twitch streamer retain subscribers over time with the project that we did.

Lilly:
His name was Ludwig. He’s. It’s currently the Guinness World Record holder for most subscribers on Twitch. We wanted to see if he spent a dollar with us, how much we could retain for him over time. And in the first month, for every dollar he spent with us, he earned a dollar 70 back, and then the following months it would drop off, 50 cents, 20 cents, so on, so forth. But overall, it was very good margins for both of us.

Dhaval:
Wow. So how did you do that? What was the recipe to increase the retention there?

Lilly:
Right. So if you are a Twitch subscriber to Ludwig, you received a notification from us that you could come and fill out this form. On this form, you would be randomly distributed into one of two groups. In one group, you received a digital hello from Ludwig, and in another group, you received a physical sticker in the mail that had a Ludwig emote on it. And we basically ran this ab test to discover that these real-life people, the people that you interacted with through the physical stickers, retained much, much higher over the period of a quarter than the other group did.

Dhaval:
Oh wow. So then this project went viral. You helped that streamer make more money. What happened after that? Like, did you keep the team together, and you continue to build the MVP? Did you raise the around of capital? Tell me a little more. Tell me all the juicy stuff that went into after that.

Lilly:
Yeah, we got our first million-dollar check shortly after that project was released. The funny thing about working with a Guinness World record holder is it only gets worse from there. The market only gets smaller, so pretty quickly, I would say by that summer, we realized this project had no legs in the venture market, and we needed to pivot. The team stayed together. But that’s when we ended up discovering. Developer advocates because they were streaming on Twitch under the science and technology section.

Dhaval:
Got it. So you kept the team together. You used that initial funding that you got to pivot towards advocates. And that’s what you’re, you build a product, you got some more enterprise customers, and now you are trying to further create a UX experience around that so that you can have more, more users. Did I get that product journey right?

Lilly:
Absolutely nailed.

Dhaval:
Awesome. Wonderful. So we got the context of what you were working on. We got the context of what problem you were solving. Let’s unpack how you built the initial team. What was the process like? How did you attract those people in your team? And then, before you answer that question, tell us a little bit about your background. Are you a technical co-founder? Are you a technical founder? Are you the, playing the CEO role, the business role? So tell me your role, and then tell me about how you got other people to join this interesting project with you.

Lilly:
I’m the CEO and founder. I have a technical background. I used to be a machine learning infrastructure engineer at Meta as well as a DevOps engineer at Rapid Seven, a public cybersecurity company, and I’ve been the first software engineering hire at a gaming startup. That being said, I don’t have a CS degree. My background in undergrad is actually economics and math. I thought someday I might pursue a Ph.D. in economics. I’m also a high school dropout, so education, formal education-wise, I have very little. That being said, I would say my team is relatively diverse in their backgrounds. Our CTO, Cassidy Williams is a prominent developer advocate. She took over all of the managing of engineers from me, so I am free from managerial work.

Dhaval:
How did you get Ca Cassidy? How did you, how were you able to get her on board? What was that experience like? Did you how did you build a relationship with her to trust you?

Lilly:
I met her husband playing video games. And her husband said you should meet my wife. She’s pretty cool. And I was like, yeah, that does sound pretty cool. And that’s how I met Cassidy. We started hanging out, just chatting. Funny enough, when I floated her joining the team, I said, wouldn’t it be crazy if you like came and worked here? Unless and I sort of made that joke, like I wanna say, over a period of a couple months and at one point she started making the joke back at me and then I thought, wait, are you being serious? And she was trying to figure out if I was being serious. And turns out we were both being serious. And so she joined.

Dhaval:
Wow. Interesting. Very cool. Let’s change the gears a little bit there. Tell me a little bit about how you built the current product. Is that built on top of large language models? Is that something that you have built on your own Contenda is a product that helps developer advocates create technical documentation. What was the process of building that product?

Lilly:
So one key thing to note is we only repurpose existing content, meaning we have to work off a source of truth, unlike other LLMs that generate content from the training dataset that they were built off of. We actually generate content from someone’s original word. That’s really important because we actually have an evaluation model that we run on top of the generative model. So, for example, a generative model creates the original writing. Actually, let me restart on that. Let’s take the transcript from a conference talk that you did. We would break that transcript, say, into 10 sections. That first section. We would then put into a prompt as an input, say, transform this into a written paragraph of some sort. That paragraph gets generated 10 times. We score each of those paragraph. With our own in-house evaluation model, that’s the determine the quality of the writing, which is subjective. The way that we determine if a number is good or not is we run that same scoring model on all of your existing content. That gives us an idea of what you determine subjectively to be good publishable writing. From there, we take the highest winner, and we feed that and the next section of the transcript to create that next piece and so on and so forth. So that’s how we create a blog using both LLMs and our in-house evaluation model.

Dhaval:
Yeah, that was very helpful. One thing I would like to gain further clarity on is how you evaluate. Is there a human in the loop who decides to score in the evaluation model? How? How does that happen?

Lilly:
So what happened when we were delivering our content through email and Google Docs? We asked all of our early testers to leave it as a suggestion in Google Docs. Then we manually went through and labeled all of that data, and that created our golden test set for how people were viewing content. We had a variety of writers from notable different roles, whether it was software development, marketing, VP of engineering, and so on and so forth at multiple companies. This created a data set of below thousands, which we found was sufficient enough to train a classification model from.

Dhaval:
Did you build that classification model on your own, or did you build fine tune an existing LLM for classification?

Lilly:
That is our own model.

Dhaval:
Great. Now, where in this journey, Does AI gets infused? Is it from the very beginning? Tell me a little bit about the user journey. What does that look like? When does a user come in? What is that awareness? When does a user gets delighted with the content? Yeah. If you could paint a picture of user, typical user journey, and where is AI infused in that journey?

Lilly:
Right. When we onboard users, the first thing we do is go collect all of their content data through whatever CMS they’re using, and we go ahead and score that using our in-house evaluation model. From there, we have a benchmark of what they consider to be good content. Then they can give us a video of anybody, anybody speaking at all, and we can transform it into writing that they would find publishable on their website without any additional work from them.

Dhaval:
Wow. Okay. That’s pretty streamlined workflow. Thank you for sharing all of that. Let’s shift the gears a little bit. What advice do you have for creators who are either building AI products from ground up? They’re like excited about all this revolution that’s happening. They wanna build something. They don’t have a whole lot of technical background like you, but they are excited. They have motivation. They wanna build an AI product, or they wanna infuse AI in their existing product. In other words, they work at some company, and they wanna figure out a way to tap into this revolution to be able to improve their user experience. So AI creators, right? How can they start on this journey? What is your advice for them?

Lilly:
I really believe that OpenAI has lowered the barrier of entry so that anybody who is even remotely technical can really attain value from the product. My advice to you would be there’s a lot you can do with just the base API of GPT and that you should be looking for low-hanging fruit where you can use the model to do manual tasks that people are performing. Anyway, anything that is highly repetitive yet manual, automating that with an LLM, is probably the lowest hanging fruit to value ratio right now. I think a good example actually would probably be cleaning form data. So let’s say you had a survey that you sent out to a bunch of people. People have in the leave more comment section. People will write literally anything in that box. Having a person go through and read all of those could be really hard. I think having an LLM summarize what those things are into a certain category could give you some quantitative data that’s really easy for you to work with. So you could just even do something like you could just categorize into sentiment. You could do positive and negative, like figure out if the comment box is that someone saying something, a glowing review like they had more to say that was really positive. Or are they leaving you feedback that they want you to work on because it’s negative? Even just seeing that split can give you a better idea of whether or not it’s worth spending any time on reading those form submissions.

Dhaval:
Very interesting. So there. Hundreds of opportunities that product creators could be working on. One, you just gave us how to clean up the form data, the survey data. What is your framework of choosing to work on what you choose to work on? How will you approach that? How would you rank the opportunities in this space?

Lilly:
Oh. When we started working on this particular product chat, GPT had not been released yet. Generative AI was not a hype train word, so for us, it purely had to do with our backgrounds. I’m a self-taught developer. I only, I was only able to break into software engineering into machine learning because Really nice people put up free content on the internet. People like who, who write for product hq. You give me your expertise, you tell me your stories, you tell me how to learn things where I can find more. And that ultimately I ended up working at Meta, which is a pretty highly competitive place to find a job for someone with no CS background.

Dhaval:
Interesting. Thank you for sharing all of that. I would love To hear about your thoughts on where is a good starting point or for a product creator who’s interested in creating in this space, what is a good starting point for that creator? To learn?

Lilly:
If you are not technical and you want to start exploring some possible business solutions, I would probably recommend no code tools such as Zapier or low code tools such as auto code. In those situations, you can play around with integrations, set it up with a Google sheet, with email, with a CMS that you’re working off of, and try finding a problem that you have to solve every single day at your job. Maybe you spend 20 minutes every single day at your job solving this particular problem. Try automating that solution. See if it works for you. If you like it and it solves your problem. In the worst case situation, you just saved yourself 20 minutes every day for the rest of this job. That’s fantastic. And in a better case solution, maybe you find out that this other team has everyone has to do that every day for two hours. They have that same exact problem, and then you can find some way to scale it from there.

Dhaval:
Got it. Thank you so much. One thing we’ll end on is your vision of your company, your future vision of the product. Where do you think your product is going? Tell us a little bit about the future vision, like five, 10 year vision.

Lilly:
Yeah, we’re actually already right now, so we’re working on the platform, but we’re also working on building extensions with popular CMSs. The idea here is not only does Contenda generate this content for you, but we could dynamically generate it for your audience. We’ve been talking a lot today about what if I’m technical, what if I’m not technical, and what that content looks like for those different audience groups. So imagine that when you go to look on a blog and you’re reading something about a guide, how to break into AI software, building your own company, so and so. Depending on whether or not you’re technical, it’ll display different resources. That’s something that contenda has the ability to do, and we can learn from what the audience wants by dynamically generating content,

Dhaval:
Dynamic content generation for content creators for different audiences or all types of content.

Lilly:
For our customer’s customer. So it would be like people like us when we go and view content on a certain landing page. It could dynamically adjust the resources and fill in the blanks for what makes the most sense for our background uniquely.

Dhaval:
Wow. Okay. Highly personalized, individualized content for the reader or the consumer impressive. I wish you all the best in that journey. Thank you so much for making time for this show. I’m looking forward to hear back from you in a future podcast where you know you come back and share more learning lessons with us. Thank you, Lilly.

Lilly:
Thanks for having me.