I want to understand about tokens. When do you need them and how do you use them?

40:57So it can be a full word, part of word, syllable, punctuation mark, or a single character, whatever that creates a meaning to the model. So that counts as a token. So every model consumes that in a different way and how their architecture is. and those are the billables that when you start paying for using AA models. Now, most models start out where there's a free tier down there that you can kind of learn on,

41:30and then you eventually get up to the point where you have to spend tokens to do a large-scale project. Am I understanding that correctly? No, you're not spending tokens. You're not spending tokens. Okay. So those are just the units of work? Tokens are like a unit of compute effort to get something done. You will buy tokens and consume tokens, but you don't purchase tokens necessarily.

41:55Well, I'm confused about that. You said you'll buy them, but you won't consume them. You'll buy the ability to use them. Ah, okay. Okay. And you'll consume them while you're performing tasks. It's a little abstract. Yes. It sounds like carnival tickets. You have to go to the big shop in the front and buy certain amounts, and then you can get into certain things and do certain things, but you have to spend up your tickets to get in?

42:20No. Is that a good or bad metaphor? It's the tokenization of information. Ah. That's how the AI computes. And obviously, with all we're hearing about the desperate desire from many big, big, big players to build these huge computing environments because they believe this is going to continue to accelerate and grow and grow and grow and grow. Somehow you have to pay for all of that.

42:47And is this tokenization process a piece of that puzzle as to how the debt gets retired from all this construction? So really what they're doing, what training, right, is the big cost. And then they're trying to get big enough data centers where the tokens per second are fast enough for everyone who's trying to use the software. So you don't get the message, please wait.

43:11Oh, okay, so it's capacity and speed as well. That's fascinating. Yeah. Yeah, so- That's how you measure how effective a machine is, is how many tokens per second. So if you have a Mac mini, you'll do 20 tokens per second. If you have a, you know, like the M5 or something, and M1 will do 10 tokens per second, which essentially is like a word, 10 words per every second.

43:32So if you're trying to write, you know, if it gets slow enough, it's actually slower than writing by hand. And Jackson and I were talking about the fact that you can have two software solutions, one of which takes a lot of tokens to accomplish the task. A better coded version takes far less tokens. tokens? Yeah, if you if you ask the agent to drive, drive your entire development or

43:57entire software, it actually does most of the job job by itself that cost you more tokens. So if you have like a framework that the agents, the agent depart is only get gets used to wherever it's needed. That's where you save tokens. So it doesn't do do everything by itself it's just does a job and does it take less of this tokenization to grab an api from that it

44:27already exists or a version of it rather than having the thing create something new from scratch or am i off base there yeah if you if you're creating something uh from scratch yeah yeah you will it will consume more tokens because it has to think think through all of the the full plan what what are the things that it's going to build, the components and things like that.

44:51And if you're doing something like a small change, it actually consumes less. It used to consume more, bigger, but now we have like a caching and it consumes less now. Yeah, I was thinking in terms of things like I remember when I learned about core video, you don't have to write code to reflect something on a pond. you can just grab the API that does reflections and bolt it into your code.