-
What is a token in AI ? Prompt examples
What is a token in AI ? – A piece of text, usually a word that we send to the LLM. The same goes for when we get a response ( usually number of words = number of tokens, roughly). How to use it ? Since we pay for it , rather cautiously. We want to send the minimum and get the maximum out of every request. Pretty much the basics of economics. Below are couple of prompts i used to analyze my tokens usage. Something are obsiously hallucinations but on the other hand we get a prety decent breakdown of all the data i did send for that coding…
-
The Golden Byte. Most valuable data
The Golden Byte. Most valuable data In data engineering, every byte has a cost but not all bytes are made to be equal ( read Animal Farm by George Orwell). We collect terabytes of data in the form of logs, metrics, cookies, text, pictorues and transactions. Yet only a small portion of this information is truly crucial and drives business outcomes. That fraction is what can call the Golden Byte, single most valuable unit of data that fuels strategic insight and decision-making. Data tiers architecture The Golden Byte embodies the essence of a gold layer in modern data architecture: raw ,curated , aggregated, and business-ready information. It is the outcome…
-
Know Your data. Cost per byte vs value per byte
Cost per byte vs. Value per byte: Rethinking Data Efficiency We are living in an era where nothing gets erased (just archived). Let us dwell on cost per byte vs value per byte of such data. Every byte you store, move, or process has a cost. We focus on cost saving. Data engineering isn’t just about hoarding everything, it’s a calculated risk about understanding whether those bytes are worth to store them. Pro hint – do not fall into the trap of ‘let us grab everything and think about it later’. It does make sense until you figure out what is what but then remember to delete it ? Oh…
