← Back Philosophy // Engineering

In Pursuit of Simplicity

November 23, 2025

I have been interested in starting my own blog for a while now - partly as a way to share my thoughts and learnings with the world, but also as a way to bring more clarity to my own thinking. Writing has always felt like a disciplined way to understand what I actually believe. Yet, I have been putting this off for far too long, because I could not decide where to begin.

Initially, I gravitated toward some of the programming concepts or technical subjects I have been working on, but it didn’t feel right as the starting point of something broader. So I began listing a set of aspirational human ideals that I personally relate to - curiosity, excellence, simplicity, empathy, enthusiasm, balance, elegance, exploration, abstraction, altruism, craftsmanship, agency, harmony. At some point, I would like to explore all these concepts in depth - learning about the philosophical origins and broad thinking around them.

For this post though, I have decided to focus on “simplicity”. A large part of that choice comes from the nature of work I do: spend time wrestling with complex systems, where every design choice seems to introduce more moving parts, interactions, unintended consequences. In the middle of that, I have found myself craving simplicity, as a guiding principle for making sense of complexity and building solutions that can stand the test of time. Exploring simplicity seems like a natural place to begin with.

The Definition

I recently read an article from the Stanford Encyclopedia on the philosophy of simplicity, which helped me understand the concept in more formal terms. It helped me with the definition beyond the casual Occam's Razor use.

Simpler theories are better.

“The grand aim of all science – is to cover the greatest possible number of empirical facts by logical deductions from the smallest possible number of hypotheses or axioms.” — Einstein

“We are to admit no more causes of natural things than such as are both true and sufficient to explain their appearances.” — Newton

These quotes carry a shared intuition: science progresses when we explain more with less. But this immediately raises a deeper question—how do we actually measure “simplicity”? It turns out there are two well-known forms:

People often blur these together, but they behave differently, and sometimes even work against each other. A classic example is the prediction of Neptune. Before it was visible to any instrument, scientists introduced one additional planetary entity to account for the strange perturbations in Uranus’s orbit.

Ontologically, the theory became less parsimonious (one more entity), but the explanation became more elegant: celestial mechanics no longer needed complex adjustments or ad hoc fixes. By adding a new entity, the theory as a whole became simpler.

Rationalism vs. Empiricism

This leads to a broader philosophical question: why should simplicity be valued at all? There are two extreme justifications:

A-priori rationalism — simplicity is taken as a primitive, self-evident principle; a virtue of thought itself.

Naturalized empiricism — simplicity is justified because, historically, simpler theories tend to survive empirical testing more often.

But neither of these justifications are equipped to answer how to balance simplicity against empirical adequacy. “Simple but wildly inaccurate theories are not hard to come up with. Nor are accurate theories which are highly complex. So how much accuracy should be sacrificed for a gain in simplicity?”

This philosophical tension mirrors the practical tension in real world system design as well. There is no perfect level or form of simplicity. Which is exactly what makes the pursuit of “reasonable abstractions” so fascinating across science, engineering, and AI.

Machine Learning: Parsimony vs Elegance

In machine learning, I’ve observed model design as a trade-off between the different forms of simplicity. With Ontological parsimony, I refer to the number of parameters a model needs to represent the world. Too few parameters and the model underfits—it simply doesn’t have enough expressive capacity. Too many and it memorizes noise, behaving like a lookup table rather than a system that generalizes. But parameter count alone doesn’t tell the story.

The deeper issue is syntactic elegance—the architecture that determines how those parameters interact. Earlier architectures like LSTMs, GRUs, and autoencoders were limited not because they lacked parameters, but because they lacked the right structural abstractions to represent language and long-range dependencies cleanly. Simply scaling them didn’t help; the abstraction itself was the bottleneck.

Transformers broke this barrier. The introduction of self-attention, positional encodings, layer normalization etc created a structure that could model relationships across text with both precision and flexibility. Suddenly, increasing parameters made sense—the architecture had the elegance to support richer reasoning, long-context understanding, and more sophisticated patterns.

Software Engineering: Abstraction as a Craft

Abstractions in Code

In software engineering systems design, I’ve always been drawn to thinking about abstractions—how they’re chosen, how they shape the architecture, and how they influence the generalizability, maintainability, and readability of code. One of the forcing functions for me has always been pretty simple: can I understand what the code is supposed to do just by reading it? If an abstraction hides more than it reveals, it usually causes friction later. And on the other side, I try to avoid repeating the same snippets of logic because duplication often becomes a maintenance problem as the system evolves.

Early in my career, Object Oriented Programming felt like the perfect mental model. Everything could be organized neatly into classes and hierarchies, and the abstraction boundaries felt clean. But over time I realized that pushing OOP principles too far often led to the opposite effect—deep inheritance trees, unnecessary layers, and code that theoretically embodied “good design” but was harder to understand in practice. There’s always a fine balance between how many layers you introduce and how complex the underlying problem actually is.

These days, my rule of thumb is simple: I create a class or module only when I’m confident the functionality will genuinely be reused, or when it meaningfully reduces the cognitive load of reading the code. Abstractions should make the system easier to understand, not harder.

Abstractions in System Design

More recently, I have been thinking about abstractions for translating business requirements into the right architecture for AI agentic workflows. What should the technical architecture look like for a system of action platform as opposed to a system of record platform? The SaaS-era MVC architecture felt like a great fit when the core job was to store data records in a database and provide different views on that data. But when the core premise of the application is to deliver value by taking a series of actions or jobs, the old mental model starts to feel incomplete.

In a system of action, the primary interface isn’t obvious. Should it still be a snapshot of the current state, or more like a timeline of actions that shows how the system arrived here? And should that design follow backend implementation or user expectations? The truth is, users themselves may not yet know the ideal way to interact with this new kind of system, because we don’t really have established patterns for it.

These questions show up very concretely when I think about modeling real-world business operations into automation. In accounting workflows, for example, a lot of the work today involves stitching together context across emails, spreadsheets, ERPs, vendor records, approvals, and reconciliations. If we want to automate this end to end, we can’t just automate isolated tasks—we have to decide what the core unit of work is, what boundaries we draw around it, and how to generalize it so the same system can adapt across companies with different tools and processes.

At that level, choosing the “right” abstraction feels less like following a fixed rulebook and more like a kind of craft, guided by the complexity of the domain and a desire to keep things as simple as they can be, but no simpler.

A Few Closing Reflections

As I reflect on simplicity across philosophy, machine learning, and software engineering, one idea keeps resurfacing: simplicity is not the absence of complexity, but a thoughtful relationship with it. The space I work in is inherently complex—distributed systems, evolving architectures, unpredictable interactions, emergent behaviors. The goal isn't to eliminate this complexity, but to shape it into structures that are easier to reason about, maintain, and extend.

Both elegance and parsimony play a role here. Sometimes adding a new component or layer makes the overall system more comprehensible. Other times, the most powerful move is to remove something—to avoid abstractions that don't earn their keep. Striking that balance is less about following rigid rules and more about cultivating judgment, taste, and an appreciation for the long-term arc of a system.

In the end, the pursuit of simplicity is also a pursuit of clarity—clarity in how we think, design, communicate, and build.