The eval section is the one I keep. A demo with vibes attached is exactly it.
We wrote a mutation audit that breaks each of our safety checks on purpose and re-runs it. Three of ten guards stayed green after I broke the thing they were guarding.
The dashboard would have called that healthy. The test that could have proven us wrong did not.
A green tick on a check you never tried to kill is a decoration.
The 2015 paper reference lands. The glue-code problem in MLE shows up in agent systems as a different shape but same disease: every workaround you add to handle a flaky tool becomes a thing the next run has to read and reason about.
For my setup the worst version of this was correction prompts accumulating in the system message. I had to build an explicit consolidation pass to drain ad-hoc instructions into stable rules, otherwise context bloat compounded faster than capability. Looking forward to the rest of the issue.
The eval section is the one I keep. A demo with vibes attached is exactly it.
We wrote a mutation audit that breaks each of our safety checks on purpose and re-runs it. Three of ten guards stayed green after I broke the thing they were guarding.
The dashboard would have called that healthy. The test that could have proven us wrong did not.
A green tick on a check you never tried to kill is a decoration.
I am impressed on how good this article is. Thanks Miguel!
thanks to you for the support!
I think after reading this article I will take the class
Amazing article, this is worth an office hour Miguel!! Really good stuff, lots to learn and think about.
will do :)
Mapping to an old article, wow. Thanks.
A true agentic system that has all the components mentioned clearly.
I like this comparison! The original paper was written by my team and it’s amazing how relevant it continues to be.
Amazing, easy to understand thanks Miguel for a good article 🔥🔥🔥
Glad you like it!!
The 2015 paper reference lands. The glue-code problem in MLE shows up in agent systems as a different shape but same disease: every workaround you add to handle a flaky tool becomes a thing the next run has to read and reason about.
For my setup the worst version of this was correction prompts accumulating in the system message. I had to build an explicit consolidation pass to drain ad-hoc instructions into stable rules, otherwise context bloat compounded faster than capability. Looking forward to the rest of the issue.