Name: Lessons from Figma: Measuring product health with the Hex Agent
Uploaded: 2025-11-21T20:16:03.153Z
Duration: 1 h 28 s
Description: Lessons from Figma: Measuring product health with the Hex Agent

Transcript for "Lessons from Figma: Measuring product health with the Hex Agent": Hello. Welcome, welcome, and happy Friday. Welcome to our live session lessons with Figma, measuring product insights with the HEX agent. We will get started in about a minute or two. I'm just going to give folks a chance to trickle in, get situated. In the meantime, would really love to see where folks are tuning in from. I'm personally tuning in from New York City. Please feel free to share in the chat where you're located. We will also launch a poll in just a sec to get a sense for what you're all hoping to get out of this session. You can actually select more than one answer choice. You'll see the poll pop up if you're not as familiar with this platform and the top right hand corner, you should see a poll. It's got a little red dot next to it. If you click on that tab, you can go ahead and engage with the question. Again, we will give folks a moment to join and please share your location in the chat. Always love to see where people are dialing in from. And I think Molly and Ree, where where are you all calling in from today? Molly, you'll go first. Where I can start? I'm calling in from, San Francisco Bay Area, Oakland. Woo woo. And I am calling in from Greenpoint, Brooklyn, New York. it. Love it. Amazing. I see we also have some other folks calling in from New York. That's very fun. Houston, New Orleans, Chicago, love to see it. Indiana. Awesome. Let's take a look at the poll results. Let's see where that is at. Hopefully everyone's had a chance to engage with it. The results I'm seeing, it's a mix across the board. I don't know if you all are seeing the results on the screen or not, but I can just voice it over either way. I'm seeing a bunch of folks saying that they're really excited to get inspired by the agentic workflows and also want to see specific use case examples of the Notebook agent. Then a mix of folks also looking for the demo of the product and specific how tos of how to use the product. We definitely will have time for live Q and A at the end as well. Not an either or, it's actually a false choice. We will actually be covering all of this content in this session. It'll be really action packed and super excited to dive in. With that, next slide, please. I'm Nicole. If you joined webinars in the past, I might be familiar face, but I'll be your host for today's session. A little anecdote is that I actually discovered Figma in college and it was truly the first SaaS product I ever fell in love with. So it is really truly an honor to introduce our speakers today. Rhee is a product insights researcher at Figma, and Molly is a senior researcher at Figma, and they both have such a wealth of knowledge to share. And so I will be very shortly handing the mic over to them so that they can really speak to how they've approached their data journey at Figma, how they're using hacks as well as our notebook agent. Next slide, please. I'm just going to briefly run through the agenda. We have a one hour session ahead of us. Rhee is going to start by telling us more about research and data at Figma, and they're going to walk us through the journey that they've been on to build out scalable programs that ultimately deliver product insights to the business. Then I will jump in with a quick overview of HEX and the Notebook agent that we launched a few months ago. Really, that's just to tee up Molly, who is going to be the one to demo how she uses the agent to really supercharge her analytics workflows. Then after the demo, we will wrap up the presentation portion of the session by sharing what's next on their data roadmap and some final thoughts on really how the HEX agent has changed the way that they fundamentally work with data at Figma. Then the rest of the time we have will be for Q and A. We're really, really excited for questions that you will have. Please put them in the Q and A box that you will also find in that top right hand corner of your screen. We'll try to address some of those questions as we go along, but really plan to answer the rest at the end. Then the final reminder I'll share is that the session is being recorded, and you will all receive the recording via e mail afterwards. You can always play it back at your own speed, especially that demo portion that Molly will be walking through if you want to get into the details. With that, I think I will just be passing it over now to Reid to take it away. Thanks so much, Nicole. Hi, everyone. My name is Rhee McGuire, and I'm a researcher here at Figma. Figma is where teams come together to turn ideas into the world's best digital products and experiences. Founded in 2012, Figma has evolved from a design tool to a connected AI powered platform that helps teams go from idea all the way to shipped product. Whether you're ideating, designing, building, or shipping, Figma makes the entire design process and product development process collaborative, efficient, and fun, while it keeps everyone on the same page. So at Figma, I own two continuous measurement programs, NPS and a product insights and health platform. I also work to create scalable paths to empower researchers across a spectrum of data fluency to gather feedback from key audiences and then build a much richer world around the information than survey data alone could ever provide. Ultimately, this is all in service of a goal to turn millions of survey data points as well as other critical user signals into accessible insights that product builders can run with to surface opportunities, garner alignment, measure their progress, and generally build data informed product. In a moment, I'm gonna dig into the how it all happens. But for starters, I'm gonna give you a little glance into our data stack for the research team. So we continuously capture high context signal through a light touchpoint survey platform called Sprig. That information then flows seamlessly into Snowflake so it can sit alongside our other data. After that, it is transformed into DBT to create structure, order, meaning, and overall governance. Then it is brought to life in a digitally tangible product. In this case, it is a well designed hex dashboard that's accessible by anyone in the company. And then finally, when there are changes to the data or insights worth knowing about, insights and information can then be pushed directly into Slack. Now none of this happened overnight. It's a story of being scrappy yet strategically clear eyed about the future, of being persistent but also willing to evolve. And most importantly, it is about leading by example, regularly demonstrating your vision, rolling up your sleeves, I guess mine are already up here, to work manually as a means to identify the critical paths for automation and making information accessible to others over time. So when our research team first started using Sprig as our light touch point survey tool in 2023, the ability to world build around the data was limited to either, one, information we were already piping into Sprig already, which required both foresight and engineering assistance, or two, manually joining that data via CSV export, threading things together, and that required both patience and often data science assistance. It was incredible to have these insights pulling in, but the threading of that needle was expensive and really time consuming. So I took what I learned about the CSV export, how the data was formatted, how it got passed through, and how we would want it to be passed through, and I used this to build a comprehensive, direct, and specific integration ask of our data engineering team. While the ask in and of itself was complete, the thing that actually motivated the team to deliver the work was me documenting exactly what it took today to bring the information to life versus what it could be tomorrow. And then just like that, Sprig data was inside of Snowflake. So with our data now flowing into Snowflake automatically, the work transformed from exporting CSVs and manually threading together information using SQL into using SQL to build queries that could actually join sprig data with p zero information on our users. What tier were they on? What was their job title? And when did they sign up for Figma? Now I wanna take a moment to say that I came up through a qualitative research discipline. Languages like SQL and Python were brand new to me. But coming back to that idea of having that clarity of vision, because resources were there like ChatGPT or a friendly, albeit busy, neighborhood data scientist, I was able to really build towards the tables that would ultimately unlock the v zero builds of Figma's continuous measurement platforms, and these would ultimately live as these interactive hex dashboards. At this point in time, product builders were becoming pretty hip to the insights that they could get access to through these triangulated data points, but we were only at the starting point. Over the next few months, our survey columns grew from sign up date to age of user at time of survey, from product used to self identified use case, from one to five scale to Figma fandom rating simplified. Likert scales evolved to help to turn into designed, defined products like percent favorable metrics, and we developed health ratings to help teams actually contextualize how good is 70%. These ratings were assigned a shade of red to green color scales, and it helped people instantly know where they should be focusing. But this also created the need for consistency, governance, and not repeating myself so many dang times in code. And yet again, because I had gone through all of this, I knew exactly what to ask of data engineering. We could come to them and show them the pains and the risks without going into DBT. And so just like that, we had information tables in DBT. And here we are today with a single reusable component in hex, which leverages dbt, adds a few minor transformations, and then we can feed the foundation of 10 modularly designed, both visually and technically, HEX dashboards. And what's more is HEX's notebook agent combined with the intentionally structured data, this really has a big yes and power. It's incredible. In a few moments, my co conspirator, Molly, is gonna bring this agent workflow to life in front of your very eyes. But so far, I've been able to use it to build automated markdown summaries, to create scaffolding for way more rich analyses, and even to figure out how to smartly bind the data to Hex and Zapier to make really super custom Slack workflows. But finally, you can see the areas where, over time, we're going to increasingly automate by identifying more friction points in areas where we're over indexing on manual approaches. And through there, we'll continue to automate. Now at this juncture, I've given you a pretty solid glance into what has happened over time to bring the survey data to life, but I wanna take a step back and ground everyone in the why behind. Here, we're looking at three critical user journeys that Figma's continuous measurement programs must be able to deliver on in order to be successful. Our leadership team has to be able to understand the health of the product. In aggregate, how are we doing through the eyes of our users? They have to be able to understand our overall portfolio and know where to drive attention. Our product builders have to be able to dig deep into specific product areas. They have to understand what's happening. Who is it happening to? And what kind of qualitative intel do we have on the why this could be actually happening? And finally, with clarity around the problem, researchers and product builders have to be able to drive alignment and get others to rally around the same problems that they see in order to move forward towards a solution or even prioritize net new discovery research work. So how does this all happen? Well, I'm gonna show you a multilayered systemic approach to building programs that can deliver on our critical user journeys. It starts with a signal, either a net new customer listening post or an existing post that you wanna operationalize and build a system around. Eventually, there's gonna be many signals. Think CX data, social media mentions, other kinds of survey data, behavioral data. And you're gonna wanna design your system in a way that has an eye for that eventual expansion. But the goal here is to be constantly putting that proof in the pudding, demonstrating how you can bring all of these different layers to life even while you're starting with a single source of information that you're gonna pull all the way through. So radical focus and constraint is your friend here. At Figma, we start with a signal that the research team had a lot of control and ownership over, and that was these low low touchpoint surveys. To ground it in reality, so for my colleague Molly, this meant that we were able to start routinely gathering information from developers who are a historically challenging audience for us. After signal comes meaning. The goal here is about familiarity, ease of understanding, and creating a framework that you can largely repeat and scale. It's also the prototype for what can eventually feed a robust semantic model and layer, something that I, in my work, am only beginning to scratch the surface of. Shout out to Carlos at Hex for telling me what these were to begin with. So we developed four core metric areas based off of the kinds of lenses that we understood our product builders would want to judge their work by. Three of them are portfolio wide, which allows for apples to apples comparison across products. First up, we have product market sentiment product sentiment, or that's Figma's overall effectiveness. Then we have product market fit. That's how disappointed people would feel if they lost access to any individual Figma product. Then we have initial impressions or how well are we delivering on user expectations within the first thirty days of a journey? And then we have a metric that is product specific, core jobs to be done, which essentially are the things that we know we need to get right, lest we totally fail our users. But by developing a framework of consistency, we now have this lens through which we could structure and analyze our signals. It also means we developed a repeatable approach. So when we look at product market fit, we can compare how our product like Figma design is stacking up against a developer centric dev tool product. Lastly, creating this meaning layer allows us to bind open into data points like where are we feeling your expectations to the specific metrics in our framework, like initial impressions rating scale, to allow us to then create a much more robust and nuanced three sixty view. Okay. So at this point, we are getting above the waterline of the iceberg metaphor or what everyone sees despite the fact that the lion's share of work is hidden below it. The experience layer is what my colleagues actually interact with. It is the packaged end product that allows Figma employees to self serve insights from our continuous measurement programs. At Figma, we have one dashboard that is a cross product summary. It allows you to see metrics over the last thirty days broken out by products, And then you can click to be taken to a specific product hex dashboard, and that follows a prescriptive modular design system to make everything feel like a really cohesive product. Hex's ability to develop a notebook view and then place end visuals in an actual dashboard in a simplified app is what made the productization of all of this really possible. Every product specific dashboard starts with a summary and then allows for a metric deep dive where Figma employees can see how something's performing over time and then dive straight into the open ended data points. Now it's important to achieve stakeholder alignment about what this end dashboard should actually look like. And tools like FigJam, which is Figma's whiteboarding tool, or even Figma Make, which is our vibe coding tool, are really good instruments for this kind of prototyping and visualization exercise. You can bring people in, solicit feedback, make changes. And then when it comes to actually building the thing in hex, you have a go to direction and clarity about exactly what it is that you're driving for. And one more shout out to that meaning layer that we talked about, because now you'll be able to rest easy knowing that whatever people touch within the end experience is gonna be totally kosher. And finally, the extension layer. These are all of the moments, big and small, where information comes to you instead of waiting for you to come to it. It is the answer to provocations like, you might have the best data in the world, but your impact is gonna be stymied if people don't know about it. The job of the extension layer is to help answer important questions on behalf of the people who might have them. What metrics should I be worried about right now? Are there any noteworthy declines? What are the things that developers are saying about dev mode last week? Or how many people with negative initial impressions ended up churning? Outputs of this layer meet people where they're at, and they give them the answers without your Slack inbox going to ping town. So at Figma, we have started with weekly Slack summaries pushed into insight specific channels that provide an overall status update, metrics to watch out for, and a few recently captured direct verbatims associated with clear problem metrics. Over time, this is going to evolve into way more targeted pushes depending on how teams express a preference for receiving information. It also means we're gonna start creating more and more ancillary products like quote or trend navigators. And sooner rather than later, we're gonna start experimenting a lot more with Hex's threads feature, which is gonna let people who aren't technically data savvy ask questions of that information even if it's not an insight that we've already pre prepared for them. So we use Hex to execute on both the experience and extension layers of our continuous measurement pro programs. Earlier, I mentioned the value of a single slice versus trying to eat that entire cake, And I've just walked you through what it looks like in practice. Now in the future, when we do start to expand to additional signals, it's gonna be about iterating through each of these layers because we're standing on a firm foundation. And it's also the exact same foundation that makes leveraging tools like the, the notebook agent super useful because you're able to really provide that context and get it on the rails so it can really get out there and work for you. Okay. I am about to pass this mic. But before I do, I wanted to highlight one last user journey that we build in service of, and that is creating the paths for other researchers to stand on the shoulders of all of this work that we've done to ensure that within just a few hours, they can launch a survey, have results flow in, and then work with the agent to build a custom data rich dashboard even while I'm on PTO. So thank you all so much for listening. I'm gonna pass pass the mic on over to Nicole. Awesome. Thank you so much, Reid. Loved your cake metaphor, by the way, and just how you walked us through that multi layer approach all the way from signal to extension. I feel like it's a really great framework. I will now share screen and I will keep this pretty brief. But what I would just want to do is I have noticed there are some new folks here who are less familiar with HEX, and so I want to just give an overview of our platform. For those of you who aren't as familiar with what we do, HEX is a connected platform for using AI to work with data. Figma is one of over 1,700 companies using Hex today as really the place where the data team can do deep dive analysis, build data apps and reusable assets, and be able to really curate the trusted contexts that's needed to unlock self serve across the organization. All of that is done with AI and agents deeply integrated into your workflows so that you're able to really get to insights faster. At Hex, we've really built our platform to follow and supercharge the data analysis process. We think this approach is pretty unique. It really starts with enabling the data team. Enabling them to be able to really explore the new, the novel, and the gnarly faster than ever. So that questions that maybe used to take weeks to answer can now be answered in hours inside of our AI powered notebook environment. That really combines SQL, Python, and no code. That's the environment you'll see pretty shortly in Molly's demo. Then from there, the data team can actually canonize the output of that work and publish it in the form of data apps and semantic models. Ultimately, that's what's creating this trusted context that compounds over time the more that you build in HEX. Then it's that trusted context that then unlocks everyone else in the organization to be able to actually use AI and are very intuitive point and click explore interface to then be able to answer their own questions and explore data on their own. But inevitably, there are going to be times when these business users end up hitting a wall or asking a really novel question or potentially getting to an answer that they want the data team to be able to validate. That's where the cycle begins anew because they're able to then Volley that back to the data team, and the data team is actually able to investigate these assets that the agent has produced and be able to actually audit them and inspect them. We really see this as a virtuous cycle. When all these different pieces work together, answers are able to get better and better over time because every asset that the data team is building is actually context that makes humans and agents able to provide more accurate answers. How does this actually work? How do we enable this virtuous cycle and make it a reality? Well, we basically offer several key capabilities in order to really bring all of these workflows together into one integrated platform. For that deep analysis, we have that powerful notebook we've been hinting at and a notebook agent that's built into the notebook environment. Then for this middle piece of it, we have interactive data apps. Insights from the notebook can be published out as these data apps and reporting assets that then business stakeholders can interact with and self serve on top of. Then finally, on the self-service side of things, we have a conversational interface called threads. That's what really allows anyone and everyone to be able to chat with their data and get back trusted answers. That's how we are bringing the magic of AI to everybody. There are all these different modalities and so there really is something for everyone. What we really want to dig into today is the Notebook Agent. A few months ago, we launched a powerful Analysis Agent built right into our Notebook environment. Molly's demo is going to really focus on how you can use the Notebook agent for digging deeper and asking the tougher questions like why something happened, exploring potential what if scenarios, and really turning insights into knowledge that the whole business can use. With the agent, you can just start with any question in natural language. You'll see Molly momentarily prompts the agent in just plain English. Then the agent is able to leverage Cloud Sonnet 4.5, as well as powerful tools built into our notebook environment to really get you to a first draft of your analysis in minutes. Very much designed to work right alongside you, help you brainstorm ideas and approaches, generate code, visualize results so that you don't have to build charts from scratch. Lots of good stuff. I'm going to hand it over to Molly to actually show us what that looks in practice. I will stop screen sharing and hand it over to you for the demo. Perfect. Thank you so much. I will get that screen share going. Hello, everyone. Thanks for joining. My name is Molly. I'm a researcher at Figma. And in this short session, I'm gonna walk through how I go from raw user survey feedback that's been enriched with, data that we talked through and leveraging that entire pipeline. And I transform that into insights using Hex Agent, which is this AI assistant inside of Hex that writes the code and, thankfully for me, builds the charts from plain English prompts. A little bit about me, I have experience writing code, but I never really learned SQL. And I have experience working with data, especially qualitative data, but I really hate chart builders, and I am terrible at using them. So the first time I used this, it was like a light shown down from the heavens to tell me that I never had to do that again, which I was very excited about. So walk through what this looks like. Generally speaking, when I'm working with data, I'm connecting to that signal layer that we talked about that's been enriched thanks to Reece's pipeline to include a combination of feedback from users, behavioral data, and customer information like seat and tier, for example. For our demo today, I am using fake data, that I've created, but in real life, of course, this would pull directly, it from an enriched query built by Rees pipeline that connects directly into hex. And following good modular practices, we're always trying to reuse components as much as possible. So we have this reusable component here. Again, this is a fake one for the demo, but generally speaking, we try to reuse components as much as possible so I can kind of build on the shoulders of those expert data scientists and re who have created this foundation to let me work as quickly as possible. So today, I'm gonna I've already imported this reusable component. We're gonna explore the data, build a few charts in collaboration with the HEX agent, and then see what we can learn. So as you can see, I've already set up the environment, pulled in that existing component, and, this again lets me move fast. I don't have to worry about this boilerplate query. I don't have to write this from scratch every time. I'm reusing one that kind of already exists and I'm lucky that Rea has made the system so modular to allow me to move super fast. So starting at the beginning, I always wanna start and in general, you always wanna start when you're looking at data by just kind of grounding yourself in the data, getting some really basic questions, and making sure that the pipeline and the, data that's showing up is kind of matching your expectations. There's no bugs anywhere. So, for example, I'm just gonna quickly scroll through here. I'm looking at, I'm seeing that they're all the exact same date, which makes sense because I generated them. So but, normally, you might see a little bit more variation there. Then, of course, we see that the seat the tier that's showing up here, I have the three tiers that my fake data should have. I have the free tier, the premium tier, and the standard tier. I also see that the different seats are showing up. I see designer, PM, and developer. I can see that the question normally, we might have multiple questions. Here, I maybe only have one, and I can see that the responses are showing up in the range that I expect and they're, showing up as a number. And then here's the, user behavior that's been enriched in in Reece pipeline that's showing up here. So I can see there's a range of, activity and then open any question and open any answer. So that all looks pretty good. Great. We got kind of a a bit of a taste to sort of make sure that the basics are there. And now I'm gonna go ahead and talk to the HEX agents. I'm gonna post in, my question here. So I have added a prompt here, and I'm essentially asking the agent to help me build some really basic visualizations. I always wanna just start with kind of the basics. Again, it's kind of about you know, I scrolled through here, but now I just wanna see at scale a little bit more, confirm, again, that the data is kinda coming in and make sure that I have the basics about it. So I'm asking things like, how many people of each seat type completed the survey? How many people at each tier completed the survey? What was the distribution of satisfaction responses across all tiers? I'm not worrying about learnings or insights at this point. It's just kind of the basics. So here we go. It's created a chart, and we we give thanks to Hex AI agent for not letting me forcing me to click through and create a chart builder. Wonderful. I can just see right here that the numbers are kind of, popping up. I can see as expected because I know about our users that we have approximately equal distribution across these three seat types. Again, that's fake data, but just you would wanna leverage your knowledge about the context of your product and the context of the data that you're pulling in, and this is just letting you visualize it. So it's really a collaboration between you and the AI agent. So this looks about right. This is roughly what I was expecting. So I'm gonna go ahead and keep that chart. And then we see another one across tiers. Okay. This is great. I have in general, when we have launched surveys, I try to make sure that we have kind of representation across the tiers for whatever we need. Obviously, we've already done that, in this case, and so I can see that the representation across the different tiers is showing up again as I expect. So I'm gonna keep that chart as well. Fantastic. And then we see this distribution. I'm not really again worried about insights or learnings at this point. We're just making sure that the, that the data kind of looks reasonable, so we'll keep that. You'll notice that I'm keeping the chart that I like as I go through. This is one best practice that I recommend, because you always wanna make sure that as you're asking the next question, you kinda know what kind of already, charts were already built and what charts you already liked. And then if a chart changes, you can track and make sure that it's changing the correct chart, that it hasn't made an unintentional change that you didn't expect or didn't do something funky up higher. That almost never happens with the AI agent, but, again, this is something where you wanna be, the one who is kind of in charge and making sure that the insights and the way it's working are working the way as expected. Wanna feel confident in what it's building together with your help. And you also wanna see so right now, this all looks good. Kind of the data is as I expect, but this is a point where you'd wanna fix any issues with your, your initial query with your component up top. Fix them now upstream before you kind of build any additional charts. So this looks good. Again, I'm not really worried about, like, learnings or insights at this point. We're just kinda getting the basics. They all look fine. So let's go on and and ask something a little bit more confident, a little bit more comp more complex of a question. Okay. Next, best practice that I recommend is now that I'm kinda satisfied with this and I don't need to build any more basics, I'm gonna actually start a new chat thread. I don't wanna overwhelm the LLM with too much context. We kind of finished that conversation, and now I'm gonna start a new one. And that's typically how I like to work, and I recommend that as a best practice. So I'm gonna ask a more complicated question. Now we're gonna see how satisfaction varies by seat. So let's see what it's going to help me to investigate that question. Because you might imagine that different seat types are gonna feel differently about the product. And so we'll see if we can learn anything from this, from this data query. Just let it run. One thing that I like to do while, it's working, especially if it's a slightly more complex this isn't super complex. But if I'm doing something slightly more complex, I will often kick off a workflow and then transition my own work. So I'm kind of working in parallel, which helps me be more productive, and makes best use of the AI agent. Okay. So it's already creating, something that I'm noticing is exactly what I wanted is that it's creating new charts rather than existing, updating existing ones. That's perfect. So that's good. Alright. Showing the average satisfaction by seat type. Oh, and look at this. So another thing that the HEX agent does is it will give me, a few different charts, helping me answer the question a few different ways that can both help me wrap my own head around the data, and maybe introduce new visualizations that I didn't necessarily thought of or think of to build. And then it also will start giving me a little bit of a a hint into maybe some, like, additional learnings here. So it's saying that it's giving me a little bit of readout here. So developers are most satisfied, one designers are least satisfied, PMs fall in between. So that's kind of interesting, but, and and I appreciate the the input. But now what I really wanna see is actually, I'm wondering if, you know, it's not just seat type that matters. There's also different tiers, and our product looks different depending on, what tier you're on. So, again, I'm gonna keep all these changes. Those are fine. If I wanted to leave them later, I can, but I'm gonna start a new chat and ask a little bit more of a a deeper question. So I really wanna see a facet. I wanna split across satisfaction by seat type and split by tier because that again, the experience is quite different. And so let's see if we're gonna learn anything different about people's experience based on that information. Another piece that I'll talk through as this is generating is, I so far am just working in the notebook because, again, this is just kind of my basics for me, my own exploratory data analysis. At the end, maybe I will publish certain components of this and maybe I won't publish all of them so that it's a little bit more legible for me in the future, and also any collaborators that I might be working with. So let's see. It claims that it's finished a chart, but I'm not seeing where it put it. Oh, here we go. Okay. Perfect. Excellent. And by the way, if you wanna jump to it so if I was down here, I didn't know where it want I can jump to it that way, and it'll show me where the chart is. Okay. Perfect. I love how it's done this. So I see for each tier, it's separated, the, the average satisfaction across each tier. So free people are generally equally satisfied, but look at this interesting difference. So the premium tier, designers are much less satisfied than developers, and it's flipped at the standard tier. In fact, designers are much more satisfied than than developers. So there immediately we kind of already learned something about our data. Being able to really quickly generate, charts like this and really quickly go through, and parse your data and be able to quickly, like, collaborate with the AI tool is, for me, a total game changer in terms of building, learnings and insights, from our users. So this isn't necessarily something that, like, I I might take this and and now kick off a second survey. Like, well, let's dig into what this is. Maybe I even go a qualitative route. I'm not necessarily gonna answer all of my questions, just from this data, but I've within, I mean, this amount of time, I wanted this, like, ten minutes, we immediately got to a much deeper level of understanding of our data. This is something that we do fairly frequently, different types of data that we do, like, a lot of surveys, but, this is the type of work that I'm doing very regularly. Now let's take a little bit more of, let's go even further. So I'm gonna actually ask the AI agent to suggest a good way to visualize. For example, we have the data that we haven't dealt with yet, which is days of activity. So I'm gonna actually ask, that's kind of complex data. I don't really have an idea off top of my head how I wanna visualize that. So I'm gonna ask if the agent has any ideas about how, what are, like, good good ways for us to look at this data. I'm not really sure what is the best way. So we'll see we'll see what it says. And this is another sort of collaboration that you can do, with AI agent. I have found it to be more successful to ask a very open ended question rather than if I have a hypothesis and I wanna ask if that's a good way to do something. I think LLMs in general tend to be very encouraging, and they want you to, feel like you're doing the right thing. And so they will tend to just say yes, like, to kinda whatever you are doing. And so I tend to start with this, style more where I may maybe asking it for some specific suggestions. It's walking me through here a bunch of examples. So it suggested a scatter plot, a binned bar chart, a line chart over activity bins, and it's giving me some trade offs, maybe some of its own recommendations. Let's see. Did it create it? Did it create it? Oh, it did. Okay. Yeah. You can tell this is fake data. But, but yeah. Wow. Incredible. Look at that correlation. Wow. The more people use the product, the more they love it. Isn't that wonderful? Very expected, experience with I'm sure everyone has had that experience with their products that they build. So but there's an example. I I love that it I love that it built it already. Sometimes it'll ask me, you know actually, I'm surprised it didn't do that. I'm happy that it built it itself. And, of course, if I don't like what it built, I can always undo or delete it. You can delete it at any time. It's not, you know, it's not a big deal. But now at the end, if you do like it, you can actually, add it to the app builder. I will say we typically, like, I'll build the app I'll add things to the app builder as a way to kinda create a report more for myself in the future if I'm coming back to this or I have, like, additional questions later, and it will include sort of the basics or the charts that I thought were the most, interesting. It's like a mini report. But generally speaking, when we're doing readouts, I will go ahead and go into Figma slides and make everything, you know, much pretty. So I would I wouldn't worry about changing the colors here. I would go ahead and do and customize all that in a separate tool. So that is some examples of, some of the ways that we are using hex agent. Again, I use this very regularly, to build charts. I don't mind writing SQL. I do not like building charts. So it's magical. It takes that part away from me, and I get to focus on the fun and interesting parts of my work, which are more about how do we what do we action on with these lines? What do we do with them? What do we wanna learn next? So, that is essential that. The last piece that I'll briefly touch on is, this open ended response data. So this isn't something that I'm actively using, but this is something that we are actively exploring right now, how to do this. So there are other ways that we could we can actually just, try it. Let's see what, the HEX agent might do. And, again, following my best practice, I'm gonna open up a new oh, thank you. We'll keep that edit. Open up a new chat, and I'm gonna actually ask it to just kind of analyze this, pull up the top three themes, and then we'll see what it what it does so we can explore that together. While that's, thinking, Molly, I'm curious with this open ended data. How how is your team currently working with that kind of open ended response, type survey, Yeah. information. Yeah. Like, Yeah. I obviously, we're trying it out right now with the HEX agent, but I'm curious what you are all doing before before the agent. a lot depending on the context. So, for example, if I have a, an ad hoc survey where, maybe to fill out the open ended part, you know, you had to get through, three questions before that. So there's not that many, like, even if there's a 100, I'll probably just go through those by hand. I'm a qualitative researcher. I like reading qualitative, feedback, so that's not really a problem. But in terms of the like, Marie was talking earlier about our kind of regular dashboards that are kind of running on a constant basis asking, people the same questions over many months and eventually years, what we typically do with that is we have, also in HEX, an open ended navigator. And so and I can't show it, but we essentially have a way that we pipe all that data into, into HEX, and then I can essentially explore that data. Or we'll often pipe that also into Slack, via a HEX report, like a PDF report. And then on a weekly basis, we have, myself and the product team go through those open ended feedback and note any you know, if there's any bugs we're not expecting or things we wanna dig into, or if we're like another example is as we're thinking about, you know, road maps, we'll go and revisit that over a longer period of time and try to pull out qualitative information there. And so all of that is just, captured in sort of our hacks our our hex dashboards and our hex pipelines that allows us to sort of dig into that a little bit more. But we're looking at and I I think we talked about this a little bit about additional ways to, add that meaning and interpretation layer on top of the sort of raw signals so that we can, expand even more. There's essentially, I think, almost, infinite appetite for, being able to leverage user feedback. The biggest problem right now is not that we don't have user feedback. It's that it's very hard to, action on all of it, and so this can help us keep our connection to that feedback a little bit more and, and be able to action on it even better. Okay. So here we go. Looks like it did, I don't know what it did. I wrote some Python code. This is very interesting. Okay. Cool. It's pulling out some themes, performance and reliability. It's giving quotes. I mean, exactly what I asked for. So yeah. Yeah. There's, I think, a lot a lot, even more that we can build on in this space, but that's what we're that's what we're looking at. Amazing. Thank you so much. I I think we said this in the comments, but it really takes another level of bravery to to demo live. So really appreciate you doing that, Molly, and really you did such a wonderful job walking us through it. I think, Rhee, if you don't mind going back to screen sharing, I think we have kind of the last portion of the presentation that you'll kind of wrap us up with. Absolutely. Before I tell you kinda what's next, I wanted to just kinda coming at the end of of Molly's offer one more thing that I really love about the agent, which is, like, the notebook cleanup. So as you're doing more, robust analyses and it's kind of doing all of its thinking, it's gonna create cells that are kinda moving through, and that, you know, they have single purposes. They're doing particular things. That's really great from an education point of view because you can understand what exactly it's doing and how it's deriving information. But once you finally get to your end result, you can then ask the agent, what are the cells that I'm safe to delete Because I wanna keep my my dashboard and I or my notebook a nice clean space. And then it's gonna give you every single cell that you no longer need. You can just click on that cell, delete it, and then kinda keep moving forward. So that's a really good best practice in terms of maintenance and hygiene. But you might be wondering what is next for all of this work at Figma, and here is a quick look at where we plan to drive this continuous measurement program over time. First up, we're gonna be making information more and more and more and easily accessible through self-service workflows. I saw a question in the chat around whether or not you can pipe data into Teams. To give you a little bit of transparency into how we do this today, I have a, custom notebook in hex that formats the data pretty nicely, and then I have a Zapier workflow that then ultimately produces it in Slack. So I cannot confirm nor deny whether, Zapier has a connection to Teams, but assuming that it does, that would be a pretty kosher, workflow for you. Okay. And then beyond that, we're also going to work on drawing the relationship between this data and critical business metrics. And then finally, when we're ready, we're going to start to pursue the signal expansion path to really start to effectively solve this fragmented data problem and get to a place where I can have my whole cake and I can eat it too. But all in all, Hexis Agent is going to play a pretty big role in our future development, most critically when it comes to things like governed insights and building really impressive, efficient datasets. I really look forward to a lot more experimentation there, and I look forward to learning from other folks who are experimenting with it as well. And so with that, I think that we are ready to move into a q and a. Lovely. Wonderful. I've actually already seen some really great questions come in through the Q and A chat Definitely encourage folks, we've got about fifteen minutes left, so definitely plenty of time to get to everyone's question. Again, you can use that Q and A functionality, top right hand corner and keep submitting anything that comes to mind. So let's dive into it. I think, I'm seeing a question about just general sort of best practice around, sort of when, Molly, you choose to kind of create a new thread with the agent. And actually, I think we have a really great slide. If we can just click to the next slide, there's one more slide after this one where we have some best practices. I figure actually since we had a question from the audience, Molly, do you want to just voice over some of these best practices and especially when you think about starting a new chat thread with the agent? Yeah. Yeah. Sure. I think, part of this best practice comes from my experience with other LLMs, not necessarily the HECS agent specifically, but I've noticed that if the context gets a little bit too big, like, if I'm asking too much in there, it just becomes a little bit hard for, LLMs in general to kind of do the right thing. And so I'm generally speaking across all the tools that I AI tools that I use, in the habit of sort of once I have once I've, like, finished a specific conversation, I kinda start a new one and move on to the next one. And especially in so specifically how that manifests in hex is, once I have kind of a set of charts that I'm happy with and I don't feel like I need to ask more about those charts or that part of the data analysis, then I'll start a new, a new chat. And the other advantage that I like about that is that in hex on the right, there's a little history of your previous chats. And so you can go back and revisit, and you're like, oh, you know what? Actually, I wanna dig into this a little bit more. And so then you can you can easily jump to kind of that part of the conversation. That's kind of how I think about it. I will admit that I haven't really seen HEX struggle as much with those kinds of, like, running out of context. So, it's more a habit just generally from your across all AI tools, but that's how I typically think about it. Love that. That is super helpful. I think we're going to stay on the theme of just some additional follow-up questions from your demo because I am seeing a couple of different buckets of questions here. To keep the spotlight on you, Molly, could you share how you went about creating that fake data for this demo? I saw that was a question someone is just curious about. Yeah. Yeah. What's really funny about that is I created that fake data in hex using the hex AI agent. So very meta experience. And yeah. And I I told it essentially, like, the structure that I wanted to kind of align with how we typically do things, with a little, you know, slight differences. Obviously, it's all fake. I told the open ended question that I wanted to ask, and then I also told it what, secret, you know, data insight to, like, hide in the data so that we would, like, get to discover it together. So, yeah, I built that all with the HEX agent, and then it generated a CSV file, which I then uploaded back into HEX and used as the source for the for the data analysis. I love that. You would have thought that we had planted this question in the audience or something. The answer is so meta that you use Hex to create that fake data. I love it. I'm also seeing a question about whether or not Hex supports other data visualization libraries. I can take that question. The charts that you saw the agent create in Molly's demo, those were all Hex native visualizations, which we think are really beautiful and we think allow you to build pretty stunning charts. But if for some reason you do want to use alternative libraries, the answer is yes. Hex does definitely support that. I'm also seeing a question about the dashboard feature around, I think it might have been Ria actually who had mentioned that it sort of reruns automatically and kinda keeps that data fresh. So actually, Rhee, do you wanna jump in and maybe just share a little bit about how you think about how often that dashboard should be rerun, and what I guess in general what your experience has been with kind of the scheduled runs, feature in HEC. Yeah. For sure. For sure. Yeah. So Hexa allows a couple of different ways to run things. While you're actually working, you can just rerun on the dataset that's already there. If you need need to, you can rerun without cache if you think that there's, like, potentially new data kinda coming in. But then once you've actually built your dashboard and you want it kind of updating on a routine basis, they have a scheduling feature where you can determine, do I want this to happen weekly, daily, etcetera? For my program, because the very name of it is continuous measurement, it is all about the idea that we have the freshest data at any point in time. Granted, at the end of the day, a survey sample dataset is inevitably gonna be smaller than, you know, some of the media's biggest revenue oriented metrics in a business. So I think that there's kind of things to think about there. So, really, what's pretty cool is that HEX does have the flexibility depending on, like, how often you need it. When I think about the kind of extra dashboard that I made that has this weekly summary that then moves into Zapier and then moves into into Slack, Really, the kind of practice there is I don't wanna overburden my end users. So by having it set to a weekly cadence on, like, a Monday, we can just have that freshest information kinda come through, which is this roll up compared to the actual live dashboards themselves that will always have up to the up to that day's data. And if you wanna refresh it, it can be up to up to the hour. Awesome. Thank you. That was super helpful context. And since you kind of bring up again this, kind of hacks, Zapier, Slack integration you have going on, I do wanna share, just for folks who are maybe a little less familiar, we do have a direct integration, like a native hex to Slack integration that people can use. And that's great for, like, scheduling notifications, pushing out dashboards into Slack channels. I think, Ri, what you've set up is that you've gone even further into, like, really wanting to customize it. And so I think that's one great part about HEX is that there's just a lot of that flexibility. So if you wanna use a tool like Zapier, you're totally free to do so. And if you want more such as that, like, built in native Slack integration, we also offer that as well. So I did want to call that out. Cool. I'm also seeing a question about, just sort of how you go about making sure data is structured and maybe certified in a way that, you can actually trust folks to go ahead and, like, self serve off of that. I actually think maybe, Ri, you were mentioning that, like, this is part of the roadmap is actually building out semantic models, in Figma. So do you wanna, like, talk a little bit about how you're thinking about approaching that and how you're thinking about, like, self-service with with threads in general? Absolutely. In ten minutes' time, I'm going to be attending a meeting with my coworkers about how we're gonna be moving this forward, which is pretty exciting. But I think for I I kinda think about the meaning layer as this kind of, like, prototype of moving into more of that semantic layer and that semantic model. Right? I think, again, it's like you're you're kind of using that as a place where you are applying the structure and the intentionality. And so for what I'm doing right now is I'm working on building a set of rules of the road that are looking at a metric digest that then we're applying each of those metrics, how they apply or how they're what what product they're ultimately connected to. Then we're mapping the qual and the quant together to ultimately get to a place where, like, we will have a semantic model and we will have a semantic layer. And then once we do plug that into HEX, what we're gonna ensure is that when people are using the threads feature or the notebook agent, which are the places where people could potentially go off those rails, we'll ensure that we're connecting data sources. We're saying these are the governed data sources. Feel free to go wild with this. Right? I think that in the notebook agent space, there's a bit more flexibility because generally, the kinds of people that are in that space have a little bit more data fluency and they know kinda what to look for. But in the thread space, people just need to know that it works. And so, really, what it comes down to is taking all of your messy data, working to get to the place where it is well governed. And then once you get to that place where it's well governed, letting that be the thing that people interact with. I wanna just say generally that I appreciate that organizations at different levels of maturity, can end up with all different kinds of data schemas and structures. Figma just this year did a major update to our actual, like, data structures bringing it way more into a place where it could be governed. Right? So I I certainly appreciate that in some scrappy startup environments, you might be kinda like, oh, things are changing. Things are moving too fast. But I would encourage that if you can find one thing to hold on to, like, grab that grab that thread and be like, I'm gonna start pulling on this thread, and I'm gonna weave this thread into a tapestry. So you're kinda getting ahead of the curve, and being able to set a couple of things really, like, on those rails. Because as we all know, at the end of the day, inputs are what lead to good outputs, and so getting into a place where you can get two really solid inputs will be really important there. Love that answer. And I also think there's something that you said there I want to double click on, which is that you mentioned the notebook agent being a really great environment for folks who are a little bit more data fluent, data literate. And threads you're really thinking of as more for that maybe business end user who isn't as like data fluent, doesn't want to be in a notebook environment and really just wants to be able to ask the question and get a trusted answer. I think that was a great way, I think, of framing it because Molly has having tons of success clearly in the Notebook agent, but isn't necessarily a data scientist by training or by background. I think that's something that I've just noticed other customers also exploring. It's like, you know, who is a notebook agent a good fit for? And it's really not just exclusive to to members of the data team. So I really, really love that you'd called that out. And then also, I'm seeing a question about, just more about semantic models, what kind of semantic layers we integrate with. So since we're kind of on this topic, wanna, address that one. So, yes, we integrate with, dbt, dbt semantic layer. So if you're, like, building things in, in dbt and you're using metric flow, you can absolutely import those models and sync them into HEX. And really our thinking there is that we, wanna be able to, support folks who already have layers built out and other tools. We also integrate with Snowflake Symantec Views, and, we also integrate with Kube as well. That is always an option that you have, but we also do have a modeling workbench inside of HEX. If you haven't yet already built out your semantic layer, you're looking for a place where you can build out those semantic models, you can do it inside of HEX. We actually have an agent in our modeling workbench that helps you do that. The agent can actually also take existing HEX projects that you have and go about actually creating a model based on existing projects so that you don't have to start from scratch, which I think is really helpful feature. We just wanted to also answer that question. Let's see what else has come in. I love this fun question. I think it'll be great to get Ria and Molly's perspective on it. What do you think is organizationally true about Figma for an effort like this? These measurement programs you've been able to stand up that are clearly world class to be as successful as it has been. I can answer that. It's re. It's all Ree. No. I think there are a lot of things, but I think that Ree's, clarity of vision and ability to, like, build an internal conviction on where this could be and what this would enable, which we're now kinda finally seeing, like, come to fruition. I think re gets a lot of credit for that, working as well as it does. There are many other things, but that that's a big part of it. So yeah. And you've got the vision. Thank you very much. But do you and since we can't clone me into every organization and also clone my paycheck in relation to that, I will say that one thing that is really important is that Figma has, a value, which is run with it, and we are celebrated for that. I won't lie. Like, the ability to run with it doesn't necessarily mean that people will be running with you or that they will be asking, how can I help ensure that you don't get dehydrated while you're running? But just, like, the conviction and clarity there and, like, belief, I think, in perseverance. Like, one of my biggest values is perseverance because, you know, things like this take time and they're hard. And so being able to just, like, have that conviction and that clarity and to find your allies when everyone around you isn't there to support you yet. And you know people are kind of pushing back and they're like, I kinda see it, but I don't know. Find the people, the hand raisers. Molly was a huge hand raiser. And when she was like, I see this. I want this, I said, I'm gonna work with Molly. I don't need to go try to convince someone who doesn't see this yet. Right? And so I think that kind of that combination of the agency of the run with it, but the appreciation around finding that allyship is really important. I don't believe that any organization is ever by instinct just going to be the right foundation for you to do this. You kinda just gotta grab those reins and say, like, I believe in this. I believe in what we can do here, and so I'm gonna I'm gonna go forth. So I guess for whoever asked that question, I would say wake up in the morning, stand as tall as you can, maybe slap yourself in the face a couple of times, and, like, strap it and go. Yeah. The last piece that I'll add about organizational, like, culture at Figma is that, from top to bottom, the entire company, more than any place I've ever seen, is obsessed with hearing from users and making sure that we're actioning on user feedback, as quickly as possible. And so that, I think, we were able to tap into that of saying, like, oh, hey. Well, you wanna talk to users? Here's how we can combine that with all other kind of data sources and scale it up in a way that is, as you as you saw, kind of allows, people across the company to access the user feedback in a rich and detailed and nuanced way. And I think that is a key part of why it works so well at Figma. Beautiful answer. I feel like very well articulated. Figma from just working with the two of you, I feel like Figma is definitely a special place with a really great, great culture. But I think these are also pieces of advice I'm sure everyone can take back to their organizations. I think those are all the questions I saw in the Q and A and honestly great timing because we've got about two minutes left so we can wrap this up. What I've started to do is I started to put some links. I feel like this chat is formatting links in kind of a strange way, so apologies, bear with me. But basically what I've shared in the chat, a couple of different things. So one is additional resources. I'll actually kind of pin that. I think you guys can see pinned at the top of the chat. And so I've linked out to technical docs on the Notebook agent for if you want to kind of just get your hands on it and get started, as well as a guide for just how to set up your workspace so that, the agents really can be successful inside of Hacks. Those are two resources I've shared pinned at the top. We also do have upcoming in person events I'm super excited about coming up next month. These will be magical evenings with Hacks. We'll have talks from industry leaders, data leaders, really a sneak peek into Hacks' vision for what this next era of AgenTic Analytics is going to look like. There will be an actual magic show as well. It will actually literally be a magical evening and I will be attending the one in New York City. So would love to see folks there. And then lastly, through our LinkedIn profiles as well into the chat for folks who want to follow-up with us and connect with us. So that's all for today. Really appreciate everyone's questions. Huge, huge, huge thank you to Ria and Molly for joining us today. I hope you all get inspired to try the Notebook Agent if you haven't already. Thanks for coming and we'll see you all next time. Thanks, y'all. Thank you. Thank you.