How I turned what I learned in AI-Devs-4 into decision memory for myself and my AI assistants

I finished AI-Devs-4 with something that cannot be measured by the number of lessons completed, tasks submitted, or commits pushed.

From the beginning of the course, I had one additional rule for myself: while going through the material, I would build a technical knowledge base for myself and for my AI assistants. Not an archive of notes. More like a graph of concepts, tools, examples, and decisions that could later support real work on real projects.

By "vault" I mean a folder of .md notes opened in Obsidian. Technically, these are just text files. In practice, it is my private wiki, where individual notes connect into a knowledge graph.

That assumption mattered to me because the AI tooling landscape changes faster than any course, documentation page, or single model. When a new framework, library, or service appears, I do not want to simply ask a model: "Is this good?" I want to add it to my own wiki as a concept or tool, connect it with my existing standards, and ask an assistant to evaluate it in the context of a specific project: my stack, my constraints, my technical criteria, and my way of working.

The problem is not that the model does not know the concepts.

The problem is that it does not know which of those concepts belong to my way of working.

A model's base knowledge tells it what exists. Web search tells it what is current. RAG tells it where something was written down. But none of these automatically tells the assistant how I weigh decisions: when I prefer a simple wrapper, when I need an intermediate layer, when a tool should become part of the stack, and when it is just an interesting thing from the internet.

This matters even more because I am not a software engineer by the classical career path. I do not have years of experience in software houses, maintaining large production systems, and making hundreds of technical decisions before LLMs appeared. My path into coding is different: more architectural, more design-oriented, and strongly supported by AI agents.

That is exactly why this system matters to me.

When an expert sees a new tool, they often place it on the map immediately. They know what it competes with. They know whether the problem is real or mostly marketing. They can sense maintenance cost, lock-in, security risk, or plain overengineering.

I cannot pretend to have that experience.

But I can build a system that helps me acquire it gradually.

When I encounter a new concept, framework, or service, I do not want an assistant to explain it from thin air using a generic web search. I want it to add the concept to my wiki based on sources I have already collected and synthesized. I want it to compare the new thing with what already exists in my graph. I want it to show me differences, tensions, and decisions in a language that fits my profile: an architect learning agentic engineering by building.

That changes the way I take notes.

A normal note answers: "What was in the material?" A good note for an AI assistant should also answer: when to use this, what it connects to, what decisions follow from it, where the risks are, and what should not be done automatically.

This is not a cosmetic difference in note-taking. It changes a note from a record of content into part of a decision environment.

Here is an editorial translation of one such note from my AI-Devs-4 vault:

---
created: 2026-05-04
updated: 2026-05-04
tags:
  - tool
  - tool/sandbox
aliases:
  - daytona.io
  - Daytona sandbox
  - sandbox
  - dev-sandbox
  - Code Execution Sandbox
  - Daytona-Sandbox
category: sandbox
path: Tools/Daytona
url: https://www.daytona.io/
---
 
# Daytona
 
## Definition
 
Daytona is a tool for creating isolated development environments and code execution sandboxes.
 
## Why it matters
 
It can be used when an agent needs a controlled place to run code, inspect files,
execute commands, or test changes without touching the user's main environment.
 
## Discussed in
 
- AI-Devs-4 course material
 
## Related concepts
 
- Sandbox
- Code execution
- Agent harness
- Tool permissions
 
## Sources
 
- Official Daytona documentation
- AI-Devs-4 notes

This is a single unit of memory. Daytona is not just defined as "a sandbox for running code." It is placed in the graph: it has aliases, a category, related concepts, a place in the course material, and sources an agent can return to.

For a human, this is order.

For an AI assistant, it is context.

This becomes especially important when agents start suggesting tools.

Increasingly, an agent does not only help write code. It also suggests what stack to use. It chooses a database, a UI framework, a queue system, a deployment platform, or an auth pattern. Sometimes it does this well. Sometimes it follows popularity in training data. Sometimes it recommends something outdated, misunderstands the tool's current status, or builds from scratch what a sensible engineer should not build from scratch.

This is not a small problem. If an agent chooses a tool, it effectively chooses an architectural direction. And if it does not know my criteria, the decision may only be statistically correct.

This is where a knowledge graph starts to matter.

The point is not to give the assistant more definitions. The point is to give it access to relationships between definitions. When I add a new tool to the wiki, I do not want to save only "what it is." I want to save what problems it fits, what it competes with, what it costs to maintain, what risks it introduces, when it becomes overengineering, and in what types of projects it can become a default choice.

Then the new concept is not a loose note.

It becomes part of my decision stack.

Here is a concrete example: I find Peter Pang's text about AI-first engineering.

The shallow use of AI would be to ask an assistant for a summary. I would get a list of tools, a few lines about CI/CD, observability, evals, review gates, and permissions. Correct, but not very useful.

The more interesting move is different.

I ask the agent to treat the text as an external thesis and compare it with the knowledge already stored in AI-Devs-4 and Brain-Vault. I am not asking only: "What did the author say?" I am asking: what is the essence of this text, which concepts from my graph help me understand it, what is missing, and what should be added?

Here is an editorial translation of that exchange:

Mateusz

Read this article: https://x.com/intuitiveml/status/2043545596699750791

Treat it as an external thesis about AI-first engineering and compare it with the knowledge I already have stored in AI-Devs-4 and Brain-Vault.

Goal: help me understand the essence of the article and place it inside my existing knowledge.

Structure your response:

1. Essence of the article Briefly explain what the author is really saying. Do not summarize the list of tools.

2. What I already recognize from the vault Show which concepts from AI-Devs-4 / Brain-Vault help me understand this text.

3. What is new or worth adding Point out 2-3 things I should add or deepen.

4. The most important lesson One sentence.

Conditions:

• maximum 1200 characters
• technical mentor tone
• no marketing
• maximum 5 wikilinks
• use the vaults as comparative context, not as the topic of the answer

Chadli

1. Essence of the article

The author is saying that [[AI-first engineering]] is not "Cursor plus faster coding", but a redesign of the whole production line around agents. The key shift: the human stops being the author of every brick and becomes the architect of the process, technical acceptance, and risk control.

2. What you already recognize from the vault

From [[Agent-Harness]], you already have the mental model that an agent needs an environment, tools, boundaries, memory, and observability. From [[Agent Behavior Evaluation]], you recognize that quality should be measured, not judged intuitively. From [[CI-CD]] and [[Langfuse]], you have a bridge to production: gates, traces, regressions, and a feedback loop.

3. What is new or worth adding

I would add three things: the difference between AI-assisted and AI-first as an organizational decision, not a tooling decision; the architect/operator role split as a new division of work; and release controls such as feature flags, kill switches, and review gates as trust infrastructure for agents.

4. The most important lesson

Agentic engineering starts when you design a work system for agents, not when you simply ask a model to write code.

That is the moment where new knowledge from the internet starts to work.

The agent does not replace my thinking. It helps me see the structure of the problem faster: the essence of the text, the concepts I already know from the course, the gaps in my wiki, and one lesson worth carrying into my own workflow.

At that point the agent is no longer a summarizer.

It becomes a sparring partner that uses the graph as context for understanding new material.

Without the vault, I would get a summary. With the vault, I get a comparison.

And that is much closer to what I want. Every new article, tool, or concept can be added to the system not as a loose link, but as part of a map: what it says, what it agrees with, what it conflicts with, what decisions it changes, and what I should deepen in my own understanding.

A second example is observability.

Before the course, I knew the word mostly in a general sense. I knew that systems should have logs, monitoring, and ways to debug problems. But while working through AI-Devs-4, I started to understand that with agents, observability is not a technical add-on. It is a condition for meaningful autonomy.

If an agent makes decisions, calls tools, and acts in multiple steps, "it worked" or "it failed" is no longer enough. You need to see what the agent saw, what decision it made, which tool it used, what result it got, where the error appeared, and whether that error came from the prompt, the data, the tool, the permissions, or the model itself.

The important part is that this knowledge did not remain only in my head.

It went into the vault as concepts, examples, links, and project criteria. Because of that, when I design a new agentic system, I do not have to rely only on the assistant's base knowledge. I can say: go through my graph around observability, evals, and permissions, find the related examples from the course, and then propose a monitoring approach for this specific workflow.

That changes the quality of the conversation.

Without the vault, the assistant can give a correct but generic answer: "add logs, tracing, metrics, and a dashboard." With the vault, it can refer to the material I have already worked through and ask questions that are closer to actual design: which agent decisions should be logged, where an execution trace is needed, which actions require confirmation, which failures should become eval cases, and what must be visible before I increase the agent's autonomy.

This is still technical knowledge I am learning. The difference is that I am learning it in a form that can later work with me. A note records a concept. A graph lets an agent use that concept in a project decision.

And if a new observability tool appears, or a new eval framework, or a new way to trace agent work, I can add it to the same wiki. Not as a loose link. As another element in the map: what it is for, what it connects to, when it makes sense, what problems it solves, and how it compares with the standards I have already extracted from the course.

In that sense, the vault is not where I put knowledge "for later."

It is one of the ways this knowledge changes how I will work in the future.

This is the most interesting consequence of the whole experiment for me. When notes are written only for humans, we naturally compress them into hints, quotes, and reminders. It is enough that I understand them a few months later. But when I assume that an agent will also use these notes, I have to write differently. I have to add context, relations, reasons, constraints, and examples of use.

Not because the agent is stupid.

Because the agent was not there when I learned the thing.

It does not know which sentence was a breakthrough for me. It does not know which example changed my understanding of the problem. It does not know that a concept connected in my head with a specific project, decision, or risk. If I do not write that down, only text remains. If I write it down and connect it, memory starts to form.

That is a different way of thinking about notes.

A note is no longer the end of learning. It becomes an interface between what I have learned and how I will later make decisions with AI.

That is why I increasingly think that "knowledge management" is not enough when working with AI agents. The point is not only to find information. The point is to let a future agent enter the context of my decisions without starting from zero.

That requires a different kind of material than ordinary summaries.

It requires notes that say:

• what this is,
• when to use it,
• what to compare it with,
• what the risks are,
• what decisions follow from it,
• where the examples are,
• when a human should return to the loop.

For me, AI-Devs-4 was a very good source material for this experiment because the course did not only give disconnected concepts. It gave problems, tools, examples, and situations where something had to be designed or made to work. That is exactly the kind of material that can be turned into decision memory.

Not only for me.

Also for any assistant that later gets access to this graph.

Finally, I want to thank Adam Gospodarczyk for preparing the technical part of AI-Devs-4. The course gave me more than lessons to complete. It gave me a language I can keep using to think about agents, tools, architecture, autonomy, constraints, and my own way of working with AI.