Your AI Is on a Junk Food Diet: Let's Fix Its Data Nutrition

00:00:00 Welcome and logistics

Chris Bergh: Good morning, good afternoon, good evening everyone. we will start in about one minute and give a few people a few more seconds to join the meeting. All right, everyone. Welcome. My name is Chris Bergh. I am head chef of DataKitchen. and we are talking about AI and data and analytics today. And specifically, we're talking more about large language models than we are in general about AI, which I would have include, you know, other sorts of predictive analytics. And so that the what we're going to do just from a logistics standpoint is I'll we're recording this and and recording the slides and I'll set up a small website and share it with you today. there's a chat window on the right. So put your questions in and we'll answer them at the end of the session or if I have time I'll answer them during the session and we've targeted about 45 minutes. so sometimes I go long but it should take about that.

00:05:43 Agenda: Analysis-Palooza and the Vibe Coding Extravaganza

Chris Bergh: So what's our agenda today? So first is I'm going to give some perspective that maybe some of you have heard before but kind of that life is really an unproductive mess in data and analytic teams and and the fix are these principles of DataOps agile lean DevOps quality observability and then the meat of today's discussion is going to be talking about how AI specifically LLMs are awesome but they're going to make the mess that's in data and analytic teams much worse and they're going to make it in two particular ways. One I'm calling the Analysis-Palooza which is really a new use case which means that data is going to be entered into a large language model to give insight and you know it think of it as I can cut and paste a whole bunch of data into Claude or ChatGPT and ask it to analyze it and we're going to talk a lot about that and then the second case is I'm going to I'm going to call it the Vibe Coding Extravaganza and this is more people are creating code kind of Vibe data engineering, Vibe data science, Vibe and and how do we deal with that especially when they want to share it and get it into production and then we're going to go to the conclusion.

00:06:53 The crisis of productivity in data and analytics

Chris Bergh: So, first kind of a background on the perspective and then o we're going to overlay that perspective with the how LLMs are impacting data and analytic teams. And so as you know I I' I've spent about 15 years building software kind of an AI system at NASA MIT startups then I've been 20 years in data and kind of I've learned really three things. One is that stuff breaks and and and you're occasionally going to get crappy data and and your customers kind of don't know what they want until they see it and touch it and that you always have too much to do. And what that means is teams are are stressed and pretty unhappy. and that data analytics as a result has has what I call a crisis of productivity. People are spending a lot of time and not getting the right results. and and for that we've over the past dozen years have really focused on this idea of of DataOps and we've run this profitable independent business and and we've kind of had a bunch of businesses.

00:07:47 A system-level view: pipelines are factories

Chris Bergh: One is that we've done data engineering with our software based on DataOps practices. Another one is as you know we've done books and trainings and conferences and management consulting and writing and podcasts. And then we've tried to do for many years we tried to do sort of top down software sales to chief data officers and you know they last for about 18 months that didn't go so great. And then the last few years we've been really talking about our open source data quality and observability as a way to drive sort of kicking data off for people. And so our perspective on this is that everybody builds it's it's a system level perspective. So a process level perspective and and so a lot of people in data and analytics focus on the data and that's important but the data is only one part of a system. And we think of that system as a manufacturing line where data comes in one side. It's accessed and transformed and and perhaps modeled and visualized and reported. And and so we think of pipelines are factories that actually produce insight.

00:08:41 Changing the factory while it is running

Chris Bergh: And then the other thing that we think is that these pipelines actually need to change and that that process of changing a pipeline is is really about adding code or perhaps adding a new data set is very akin to what happens in software. that you've perhaps have turn heard the term CI and CD. and perhaps you've heard software terms like unit tests or integration tests or end-to-end tests. And so that process of moving things from the idea in your head into production and you know both of these things are honestly happening at the same time, right? You have your manufacturing line and you're changing your manufacturing line at the same time. And of course your customers want have 10 things for you to do and are very upset if the data is wrong. And so you're in this impossible situation and and and that's hard, right? because you don't want to learn about problems from your customers and you don't want to break production when you deploy changes. And so this idea that we've written about, we have software to help if you work on it, if you work on this system and process level view, it actually does increase productivity, sometimes five, sometimes 10, and gives you more time to work on stuff and you're not wasting time either building things that don't need or fixing things that are already broken.

00:09:48 LLMs arrive, and engineers start vibing

Chris Bergh: And and so this perspective on DataOps on agile and and has informed our view I think we've seen a lot of the world kind of adopting these principles over the last do dozen years you know there's about a dozen data observability companies there's lots of people who are talking these terms and maybe it isn't the word DataOps it's it's DevOps for data or code like a software you code data like a software engineer but there's it's really improved. So, we've got this new thing that's come along, you know, AI, specifically LLMs, and they're awesome, right? And how do we deal with that? Right? And and so, I'm going to talk specifically about LLMs. And maybe I'll slip the word AI, but when I use AI today, I'm going to meet LLMs. I'm not going to meet things like Kohonen and clustering or random forests. and so what's going on now is your analytic engineers, your data engineers, they're called vibing. So they're taking tools like Claude Code or cursor and what was happening is is your codebase is actually being sucked up into the model and it makes it easier for them to write code.

00:10:47 Confidently wrong, and stakeholders demanding AI magic

Chris Bergh: And so and then also people who are taking exports of your data and pushing it into things like Claude or a BI tool with an LLM back end and analyzing it. And you know, the thing with large language models, in fact, the thing with all AI is it's pretty confident on serving up things that are wrong. And sometimes it's amazing and sometimes it's wrong. And that's going to be a big theme of how we we talk today. And and yeah, you've done things that were traditionally AI, right? Random forests, predictive models, regression models, and those still have their challenges. And of course, you can't walk through an airport or watch TV or do anything nowadays and not hear the word AI. So it's it's AI craziness nowadays and and your stakeholders are screaming for AI magic. So it's not making the life the already stressed life of a data and analytic team. It it's in my mind it's not helping but it's going to make life more interesting. And so one of the biggest things is that LLMs or any model will never get 100% perfect results.

00:11:45 Air traffic control, and why no model reaches 100%

Chris Bergh: And so in the 90s I started my career working on an AI project to automate air traffic control. that is to improve the sequencing and spacing of aircraft. And when we first started, we thought, well, the the computer's perfect. the people are are are problematic. And what we learned is that we had to meld the model to fit with people. And so it because some percentage of the time it was inaccurate and then getting it from 82 to 85 to 86% just took forever. And so no model is ever going to be 100% accurate. and LLMs are anywhere from 50% to 10% inaccurate all the time and improving it is getting asymptotically harder. and so LLMs will always deliver imperfect results yet they've got this huge potential to assist users and make them more productive like sequencing and spacing. So, one attendee raised a hand. So if you've got a question just put it in the the chat window on the right and I'll try to answer it.

00:12:39 The LLM as a new interface to your data

Chris Bergh: So, it's the the the third button from the lower right is the chat. So, so again, I think there's different opinions. Some people believe AGI is going to happen and we're going to have this miraculous super intelligence. I don't believe that. you know, but if you do, I apologize for messing with your religion. But I think this is the state of the whether what happens in 10 years or not is up to to debate. I think this is the actual case now that that LLM hallucinate. They're inaccurate. They don't give good results yet people are using them increasingly more. And and so that makes this interesting situation, right? of really two cases that we're going to talk about. And the LLM is kind of think of it as an interface to your data. It's it's an interface for a customer, maybe a business analyst, maybe a VP, maybe a director, maybe someone on the business side. they're using that the data inside the LLM to make predictions, to understand, to chart, to graph, to analyze data.

00:13:39 Two shifts: more analysis, and more people writing code

Chris Bergh: So, it, you know, you used to just have BI tools in Excel in Excel. Now you've got BI tools and Excel and both of them have LLMs in and then you also have LLM that people are using directly like they open ChatGPT and and and put data in or they hook ChatGPT up to your database through something called Model Context Protocol and and query it that way. So LLMs are kind of increasingly analyzing data for us and making predictions. And then the second part is LLM is LLMs are expanding the pool of creators of people who are creating code. So more code is being put into production by many more people because it's easier. your product managers, maybe your analysts, people are starting to use Claude Code and apply it to your codebase and create tables and and create create mo create create Python code. And so this expansion of the of of the world of data and analytics is happening with LLMs. Now maybe it's bad, maybe it's not, but it's definitely happening and it definitely has this overriding challenges of things are sometimes wrong.

00:14:41 Analysis-Palooza: the hidden assistant in every tool

Chris Bergh: So what I'm going to talk about in the next section I'm calling the Analysis-Palooza. That means lots of people are analyzing data. And so how do you deal with this crazy case that that that LLMs are analyzing data and they're sometimes wrong? So let's just think about what this new use case is. So it's your data. So some internal data set maybe there's some metadata or context around the data that's being put into an LLM and then some insight. In one case is a conversational interface to data. Another case is the LLM's completely hidden and maybe it's you know there's there's a cutely named assistant that shows up in whatever tool you have. So almost every software tool has got an assistant now button on it somewhere. and it's helping you to and some of those are helping you navigate the tool, but a lot of them are helping you analyze data. And so that's that's the situation, right? And so what's the challenge with that?

00:15:36 An 80% model on 80% data is a 64% answer

Chris Bergh: Well, I think the first thing is it's just driving more data usage, right? And it's and it's people have are voracious to analyze data and they can start analyzing data they maybe never have before because they can just throw it in an LLM and see what happens. And so what that means is that bad data compounds the error of the LLM. So if we look at this middle column here and we look at a model let's say a model is 80% accurate or 20% wrong on making predictions on data and that's a little bit high or a little bit low depending upon the use case but let's say your data is only 80% correct and how you judge that well let's we can talk about that but if you look at the combination of 80 and 80% you get a 64%. So really the you know the data is is driving a a lower expected value in terms of the quality of your insight. And so that that's the challenge because I don't think the model itself is going to increase in accuracy.

00:16:32 Shift left, and a bad report that reached 5,000 sales reps

Chris Bergh: The data quality and the metadata or the context can help, but it's it's again you're you're always you're going here from if you had perfect data, you still have 80% accuracy. And so the way that we think about this in in our world is that we we think of it as shifting left. and and when you have bad data that's being put into a report or an LLM, finding that problem, it's better to find it sooner rather than later, right? It's it's always the best perhaps to fix it at the source, right? So it never happens again or to patch it in the raw data layer or fix it in the integrated data. So every time you let bad data sort of squeak into delivery, it's a hundred times more expensive as a rule of thumb than fixing it at the beginning. And so this sort of 10 you know 1:10:100 rule I think is yeah it's old and it's it's been applied in different things and is it one or 10 or 100 well I think this is a metaphor right finding problems sooner is better than finding them later and and having to you know look back and this happened to me 20 years ago you know having bad data in a report that ran out to 5,000 sales reps at Johnson & Johnson and then having having saying yeah this data has been bad for three months and yeah it's our faults and that that is you know I still have shivers at night about

00:17:49 LLMs increase the surface area of data quality

Chris Bergh: That and so you know the other part is that these LLMs kind of create kind of increase the surface area of data quality and let me talk about what that means so users are putting more and more data into LLMs for insight generation because it's easy because they can go and query data and put it in So they're using more data and the models are analyzing every field. They're basically taking an extract of a table and putting it into an LLM and saying analyze this or their tool is analyzing it. So every table and every column is being used or at least looked at by the model and and it's this is something that happened in the old data science era, right? Where data scientists would give me the raw data, I want the process data. But this is you could talk to a data scientist and say okay you've got to fix it and patch it. But ordinary business users kind of don't care. So in some ways the surface area of attack on the quality of a data has has increased because of the ability of LLMs to get insight very broadly across lots and lots of columns of data.

00:18:48 A thousand data quality dashboards nobody looks at

Chris Bergh: And so that's a challenge for data teams, right? Because I think all of us know that when people get poor insights, people will blame the data and and hence the data team. And and I think hopefully or unfortunately I think all of us have experienced that. And so when you have this increased surface area of people using the data, it means you need more test coverage. you need to be able to increase the cover of that surface with data quality tests. And so we believe that that can't be manual. You can't do it by looking and extracting and running running manual checks. You can't look at it by looking at dashboards because they go out of date and no one follows them. like yesterday we talked to a large eastern kind of insurance health insurance company and they've got a thousand dashboards for data quality but they still have enormous data quality problems because no one looks at them and they're not automated. So we think automating test coverage by putting a layer of data quality tests across all your data on every table and on every column and on every metric.

00:19:52 What test coverage means, table by table and column by column

Chris Bergh: And then of course being able to monitor every tool that uses the data is also important. And so what does it mean to have test coverage? Well, every table should have a test. you should look at kind of I think of the delivery or supply chain tests like volume freshness schema you know am I getting things that are out of bounds and then when you look at every co column you're really consistency is there some change in the volume of a column or is is there some drift in the data quality of a column and then every business metric where you're combining things like I'm doing the 12 month sales sum of my major product and has that changed? and all these tests need to happen as well as looking kind of at the operational part of of of logs and errors and metrics because all of these have challenges. And so in every part of your database, you should have minimum two or three tests per table and a minimum of two tests per column and also a minimum of one custom test per metric.

00:20:49 Counting the tests a medallion architecture needs

Chris Bergh: And so if we look at that from an example standpoint and look at a medallion architecture here. And so here's a case where there's three layers L1, L2, L3, which is a common case in medallion architecture. It's called gold, silver, bronze. And then there's some tools acting upon it. So let's say there's a 100 tables in L1, 100 tables in L2, and 10 tables in L3 because that's the sort of processed fact tables. So I've got 10 columns each. So I've got 200 table tests and 2,000 column tests in L1. Similar amount in L2. And then in L3 I've got 620 tests and plus let's say a handful of domain specific business tests. Now that's a lot, right? That's a lot of tests. So I don't want to code those. That would take time. And so if you look at it, let's say I have 2500 tests to write. And so how do I do that? Right? I could either there there's a couple of ways, right?

00:21:40 Seven months of hand-written tests, or AI to fight AI

Chris Bergh: You could say I'm just going to have someone write the SQL or I'm going to use a domain specific language like Soda SQL or I'm going to configure it in my you know click and click and click through a sort of a a UI tool. even all those if you say let's say that process takes 30 minutes and that's fast you still end up with seven months of work to actually code all those. And so what happens is what AI is doing is driving the surface area. And so what if you're going to have people analyze data, you're going to have to have data quality tests across all the data. And so do your data engineers know what to write? And so if you're going to get this expansion of surface area, you're going to need some AI to help fight the AI of the attack service on your data, so to speak. And so that's partly why we've built our our open source software. Now, this is a bit of an ad.

00:22:28 Open source that builds the coverage automatically

Chris Bergh: But you know, we've noticed this problem and we came at it from a different way. back in the pre-AI days, we noticed that no data engineer was writing enough tests. They just never had enough time. And, and if even if they did, it took a while for them to understand exactly what they needed to test. And so we built this tool that kind of think of it at will scan your database, scan every table and build that test coverage automatically across all your tables. And so the benefit here is whether you're using AI or not AI or the your data is going in an LLM or not or even if you don't know it's LLM, you're getting you know thousands of tests created with a click of a button that you can then manage through an a UI and gives you that time to build we don't believe that we can ever get to 100% test coverage because there's a whole bunch of business specific rules and also we have a UI to do that and so ending the ad.

00:23:19 Recap, and on to the Vibe Coding Extravaganza

Chris Bergh: But that's that's u partly why we think test coverage is so important for AI and why people are just you know they're they're this sort of Analysis-Palooza is driving the need to have things like automated data quality tests. So any questions so far? So we're about 23 minutes minutes into it. I've talked about the idea of DataOps. I talked about that people are using large language models to analyze data whether they know it or not which increases the surface area which drives the need for automated data quality testing and and test coverage. Now let's talk about another case this Vibe Coding Extravaganza and this term has come in the last six months or so and and what it means is that I'm using an a large language model to produce code. So let's say I take my dbt codebase and I put Claude code on it and it can actually start writing tables. It can write SQL for me and it's not bad. It's not perfect but it's not bad.

00:24:21 Every ETL tool now has an agent writing code

Chris Bergh: And so this is again happening sometimes whether you know it or not. every ETL tool now has got a cute named agent that can help you write code on its platform. And that code may be YAML, it may be JSON, it may be SQL, it may be Python. However, their internal execution is is done. they're building a model that can actually build it for you based on English language. And so I can say, I want a table that has three columns that look like this. Can you create it? And boom, three seconds, it's there. And so what that means is is that's really great, right? But it has that same aspect of LLM. It's fast, impressive, but not perfect. And so, how do you deal with that? And so, I'm going to talk about that. But first, I'm going to talk about what this means, right? That more code is being put into production by many people. And so, to do that, I have to talk sort of about the history of coding.

00:25:09 The book and man page era of coding

Chris Bergh: And so, I have a little gray hair, so hopefully you're not going to freak out at my my view of the world, but I learned to code kind of in the 90s, the 80s and 90s. And to me, I think of it as the sort of the book and man page era and and that was actually on top of the original era of coding in the 50s when it was sort of women's work. which is as crazy as it seems. but the the decade I learned in and I don't know much about that. I've just sort of read some blog articles on it so I won't talk about it. But I do know a lot about sort of the the the deep nerd coding era. And in that case, I learned to code by looking at other people's code on the project and going to bookstores. And so if you in 19 in the early 90s, if you were out on a Friday night, there was two bookstores here in Boston. There was Quantum Books and SoftPro that were full of technical books.

00:25:55 The search and Stack Overflow era

Chris Bergh: And there was another one in Palo Alto called Stacey's. And you'd find sort of scruffy looking, mainly men looking at it, some women. and that's how you figured out stuff. You had a pile of books in your office. you had some man pages where you do it, you look at other people's code and you sort of have to puzzle it out. and and that was fun and that's how how basically I learned to code. And then in the past 20 years, we we kind of went into this search and Stack Overflow era where I could Google search and I could find examples of code. I could find blogs that people wrote. I could find questions that people answered on Stack Overflow. And I think that actually increased by far the number of people doing coding. And so, you know, I I I call it the tech brocoding era. And and so for where just there's just lots of people who, you know, heard about Facebook, thought they could get rich, and there's just millions of more people coding all around the world.

00:26:44 The vibe and prompt era: CFOs and sales VPs writing SQL

Chris Bergh: And it's just much easier because, yeah, you still have books that you can learn from, and you still have man pages, but you also have all this example code in all these blogs. and they're sort of they're fantastic, right? because you can code by analogy by looking at it. And so what happened in the past couple years or just two years ago really is that all that stuff got hoovered up by large language models. And so they created this sort of search index that writes things. And so I think of it as sort of the vibe and prompt era. And so everybody codes now, right? It doesn't even have to be a tech bro. It could be you just don't know it. You you type some English language things in and it'll write code for you. and that's incredible, right? And so what that means is more people and more types of people and people you wouldn't expect, right? It could be your sales VP or your sales director.

Chris Bergh: It could be your chief financial officer. All of these people are kind of creating things like new tables or new transformations and new models and you know and and probably a lot of SQL. And so I think of this era as kind of a Vibe Coding Extravaganza like a lot of more people are going to be kind of come into the fold and create code. And some of that code's going to be one-off, but some of it's going to actually be put into production. And so, how do we deal with that? So, you know, like for instance, just to kind of belabor the point of of how people, so I I've been using Claude Desktop and you can point it at a ETL project and help it write SQL and some of our data engineers have been doing that. I know Eric Estabrooks has been doing it against our our DataOps automation tool and helping it write recipes in SQL. you know, and as I said, the AI assistant in your favorite ETL tool or orchestrators is helping you write the YAML configuration or the JSON configuration.

00:28:18 How accurate is LLM-written SQL?

Chris Bergh: And you could obviously use Claude Code to write some Python code and and your customers are doing all kinds of these actions today. And that means there's just a lot more people coding and and there's more coming. and so think of it as a tidal wave of code that's coming at you that people are going to want to use, want it, want to put into production, and that you as a data team are going to have to. And it's sort of self-service plus+, is another way to think of it. and so the challenge again is accuracy, right? And so like if we look at a benchmark of how LLMs write SQL, well, they're increasing, right? they're getting up to 50 60%. there's a benchmark called Spider that does it. you know GPT-5 is good at math but struggles with databases and and accuracy is is not great and and actually database specific accuracy is really poor. like if I want to write something specific to Snowflake versus Postgres versus SQL Server.

00:29:10 Context helps, but it is not a miracle cure

Chris Bergh: And so even if you look at this chart looking at sort of 56 submissions over the past couple years, you can start seeing how it's getting asymptotically harder for them to improve. And what that means is people are using it and they're creating SQL that is both imperfect or perhaps not correct and or not performant. And so you've got this human judgment on it that happens. And so the the next thing is that people say well that's great but you need better context right you can't just write a sentence and expect it you need context you need metadata and so AI may generate the code so you can add to it add context to it or metadata so you can put things like data profiling or your data catalog or your data lineage or your test results but again context is not a miracle cure it can help increase it but it's never going to get close to perfection so you're always going to have a significant p percentage of the time where people are using large language models to code in this case SQL or YAML where it's incorrect and you know they're playing around they're running with it maybe they've fixed it maybe they haven't maybe it's performant maybe it's not and so how do you handle this right especially when maybe they've used this for their personal thing they like it and they say can we I've been running this manually can you you on the data anal analytics team fix it.

00:30:23 Shift down: put vibed code through a tested process

Chris Bergh: So there's another concept here called shift down and that means being able to take that code that's been vibed up and actually put it as part of a process, a deployment and tested process to be able to go in and make sure that that code can run in a performant way that it's covered with data quality tests that is part of a system. And so that system again means that you want to be able to run a whole bunch of tests. And this diagram shows a development environment at the bottom, shows a production environment at the top. And basically what you want is a red light, green light, like I've made some change. I've vibed up some new SQL. I've put it part of my ETL process. and I've got a little some transformation code. And great, I've run it and I've run data quality tests or data integration tests and I know that the tests ran here, but also it didn't break the report. and so checking the data transformation, seeing the impact on the report and that's your sort of red light, green light here.

00:31:16 Regression testing across tools, not just tables

Chris Bergh: And so you want to have that and if it's green, then you should be able to go push it to production and ex and not expect a regression. So that means sort of shifting down to be able to test the data and test the tools and the impact of it on on the data. And so we think of that as a technical way to accomplish that is called regression. And that means building a separate environment that has kind of sort of today's code and perhaps yesterday's data and then running a a copy of your production environment laced with data quality tests laced with a whole bunch of tool monitoring as well to see if things work. and that's where we have kind of here's another ad. we have another tool that helps because the challenge of doing regression testing is being able to see the impact on other tools. And so if I make a change to a table or I vibe code a new table or I vibe code a modification to an existing table, how do I know that hasn't broken a report or broken some Python-based predictive model that that's there?

00:32:10 One pane of glass where every tool reports in

Chris Bergh: Or it won't break some other use of the data. And so we have another open source tool that where all the tools like your ETL tool, your you know your your Airflow, your visualization tool, your models can connect to as well as our our our data quality tool. Think of it as a single pane of glass. And that single pane of glass gives you sort of a process lineage across your tools. And what it enables is this sort of full regression testing in development across all your tools and data. And it also includes production monitoring and and so you can actually stop production when you break a quality gate which I think is also a really interesting use case. So ending the ad I want to get to the conclusion here in the last few minutes. So let's talk about this. So large language models are great. you know I I use them a lot to help me write. I take a lot of outlines.

00:33:03 Never ship what the model writes without review

Chris Bergh: I create them and I edit. So I think it's great. they they are my assistant in writing but I never ship what they write without several layers of review because it can get things wrong and make stuff that sounds stupid and so what that means is people are analyzing more data in a new way and creating more code that's being in in into production and so that actually makes our life kind of worse right because if you look back and reflect on the challenge as I said is that you're getting crappy data things are breaking your customers are act asking for for too much. and so that stress is those stressors are still there and kind of being supercharged by the Analysis-Palooza and the Vibe Coding Extravaganza and so how do we think about this right so let's take it case by case just to summarize so more people more roles are taking data to analyze in LLMs in more tools and so that generally means increases the the the the surface area of data that people are being confusing.

00:34:00 Case one: more analysis means more test coverage

Chris Bergh: It also means that you've got to have better data quality because people are analyzing more data and it also means that you need to have better test coverage across all your different tables. And so hence that means the need for automated data quality testing. And so whether you create them manually, whether you do it this increase in surface area of data, increase in use of the data is a is a is a problem for is a problem for your data team. And so being able to lay a layer of data quality automated data quality tests across your data is is super important. It always was, but it's helps. You know, it's just making things worse or better depending on your perspective. So the second case, the Vibe Coding Extravaganza. Well, there's more people creating more code, right? And and and more roles. And so what does that mean? Well, they've got more code, some of which are going to want to be put into production. That means you're going to have to be able to do regressions.

00:34:49 Case two: regression testing and the cycle time tornado

Chris Bergh: And people are going to create code faster and change code faster. And so they're going to be able to create things on an hourly basis and they're going to want those changes put into production. So regression or impact testing of the data and the tools that are using the data becomes super critical. because you know it's you know I thought about when I was doing this about saying it's sort of you've got this Analysis-Palooza the Vibe Coding Extravaganza and some ways the cycle time tornado right because people are doing things faster you're going to have more impact on your career where people are going to say look this data is wrong can you fix it and what they mean can you fix it in the next hour because I want to anal analyze it again in in ChatGPT or I've got this new I've vibe coded up a whole bunch of dbt stuff I want to get in production can you make that happen for me in the next hour and so I I debated about putting this in but I think that's implied here is that you've got this cycle time tornado coming your way because of these two factors and so the extravaganza vibe coding and the Analysis-Palooza all of which means that you need to apply these DataOps principles more because the use and the creation of code is increasing because of the facilities that that LLMs have.

00:36:05 Shift left, shift down, and AI to fight AI

Chris Bergh: And so, you know, we talked in this about sort of shifting left and shifting down with the an Analysis-Palooza and the Vibe Coding Extravaganza. And I think these are really the principles. And again, you could use our open source or not, but the principles still apply. you've got to have a lot of data quality tests. you've got to look at understanding all how all the tools are acting upon the data, being able to give good regression tests and those things are I think are just continually true. and LLMs make it much more imperative. And so the last thing is you know just want to plug our open source again. Think of it as AI to fight AI. We have our own AI engine in it that kind of takes all your data, looks at it, and applies the appropriate tests to the data. It actually does much better than an LLM. We've sort of compared it to what LLMs create and and they're great.

00:36:54 Pricing, free resources, and where to start

Chris Bergh: They're like good ideas. but, you know, we've taken it it's not nearly as good enough. And so, we'll be talking about that in a future case. It creates that test and tool coverage that you need. And and we've been very good at kind of helping our open source users, helping our enterprise users with kind of a white glove experience. And and yeah, we are a business. So we have a full-featured open source for one user, but we have a very reasonable fixed fee pricing that's user and connection-based for our enterprise versions of both products. So give it a try and and I've included a bunch of links on on this as well as to the books that we've written that are free and and the trainings that we've written. So, I want to stop there. and I've talked a lot about, AI and and how it impacts. I want to see if there's any questions I can answer before before I end up. So, let me see.

00:37:46 Closing thoughts

Chris Bergh: Don't see any questions yet. so just to you know I think this is an exciting time right these are are cool tools but the it doesn't change the fundamental situation that our data teams are in which is their their their things are breaking and they have too much to do and and and it's just going to kind of make it a little worse. And so we're hoping that these principles of DataOps and our and our our tools can aid in that in that process. So again, I'll I'll send out these slides. I'll send out the recording. I'll put it up in a little website today and and email it to you all. And Attendee, we didn't get a chance to answer your question, so I don't know if you want to put it in the chat window or or turn off your mic and ask it. I'm I'm certainly happy to do that. All right. Well, that's it. I u Thanks again everyone. have a great rest of your afternoon.