AI and knowledge

 


We are covering the topic of AI in a series of blog posts. The first two posts were looking at the matter at a more practical level, this one will be more theoretical / analytical.

It is universally acknowledged that AI solutions are becoming more and more clever. However, this word “clever” is very abstract. What about something that is a bit better defined: are AI models becoming more knowledgeable? What does that mean in the first place? Let’s analyse.

There are multiple definitions of knowledge available, but most of them revolve around the four elements listed as an introduction in Wikipedia: “Knowledge is a form of familiarity, awareness, understanding, or acquaintance”. While three of these elements are easily attributable to the existing AI agents, understanding is standing out as an outlier in this aspect.

What we have come to believe in Qualifast is that the understanding aspect of knowledge is probably the most valuable. It shows good analytical skills and versatility, qualities of significant impact, more important than experience or plain remembering of facts. That is why when we are interviewing new people to join our team we especially try to measure their level of understanding in their field of operation. We have devised a system that helps us measure this. It can be briefly broken down into four tiers:

  1. The “zero”, nothing known about the matter
  2. The “how”: people that can practically execute a particular task following patterns that they have seen or have read about. At this level you are able to solve repetitive highly similar tasks.
  3. The “why”: people actually can explain why they are doing the particular thing and reason on its effects. At this level you are able to adapt the solution to fit more unconventional cases and are usually able to cover all cases and specifics correctly. This is the lowest level that can claim understanding of the topic.
  4. The “when”. People that have deeply understood the implications of a particular practice are now able to judge when it is a good fit for a particular problem and when alternative solutions should be pursued. At this level you are able to use the solution when it is really needed and not try to kill a mosquito with a bazooka or empty the ocean with a spoon.
The more topics a candidate can achieve a higher tier, the more valuable we assess them to be. It seems creators of AI models follow similar reasoning. If we apply the same valuation scheme to the current AI agents they will score a “how” tier on insanely many topics: far more than any human being. In recent months reasoning AI agents are gaining popularity and the focus of the community. What they focus on is to explain their rationale on the route to their conclusion. Probably one might think they want to cover the second tier. However, remember, our tiers are measuring understanding. Although the AI reasoning models answer a “why” question it is not from an understanding perspective. We seek for people that are able to abstract away higher order patterns and rules from more specific tasks. All this in a succinct and communicable way. Something that AI models still do not cover. The current state of the art models will fail our test for tier 2.

And here we want to bring one more aspect. Knowledge is the greatest value humanity builds up on our planet. Ever since the earliest days people have been making observations and experiments, making inductions and deductions and learning new things. These were thought to the next generations and they had not needed to start off from scratch making the next step in scientific progress. The most important act of this process has always been the knowledge transfer. In order to have this really happening you need to be able to explain what you know in a succinct and understandable way. Something that we mentioned that current AI models are not good at. This leads us to the conclusion that they are not good at making knowledge transfers as of now, something that we have already intuitively felt when interacting with them. With this they can not be a step of the constant knowledge increase for humanity yet.

The step to higher order abstraction and explainable pattern extrapolation is a very important one and impending, but it is hard to tell how much time it will take to cover (if ever). For the moment we need to recognize the proper way to interact with AI models regarding knowledge: we can use it as a storage of indexed and browsable facts and advice on how to solve tasks in domains we have not enough knowledge in. Some extreme adopters are almost giving up the process of thinking, exporting all decision making to AI assistants even nowadays. This should not be done, we should not export that much responsibility to them. We need to recognize what they are good at and use their capabilities to help us deliver better to the very purpose why we are here: to build more and more value and knowledge, and prepare our legacy for the future generations. Maybe someday AI will be more central in this process, but not yet.