On Abstraction

LVIII

2024.08.04

Words Shipped / Words Written: 1,629 / 3,109

What long walks (lurks) in Cambridge taught me about writing code from the top down.

Abstraction

To leverage the power of abstraction, you assume that more atomic, lower level things are already taken care of so you can focus on creating the most elegant and robust top level as possible. 

When you function on a higher layer of abstraction, either the low level implementation is truly on auto pilot or you are acting as if it is already solved. When you are walking, you are not thinking about your muscles moving. Likewise, when you’re building a product that is only 10% a web crawler, you shouldn’t actually start with the web crawler; you should assume the web crawler is done and take it from the top.

Historically, I’ve conflated having a bias towards action with getting granular quickly. This week, I’ve made a significant step to uncouple those two ideas. 

To see what that looks like, first we’ll explore the Art of the Lurk.

Lurking

When I was staying at C House, a critical staple of the culture was the “lurk.” 

Oftentimes, you’d hear someone ask, “Down for a quick lurk?” I believe Mridul (you should sub to his blog) coined the term; likewise, the question often came from him. 

From an outside perspective, a lurk is indistinguishable from a “walk.” However, that is not always the case–all lurks are walks, but not all walks are lurks.

A lurk is a walk taken with the intention of following drifting ideas; when alone, you’re exploring thoughts. When with others, you’re deep in conversation. 

Just like a walk, when you go on a lurk, you’re not thinking about your gait. Am I walking by striking the balls of my feet first or the heels? That’s a critical part of walking, but in a way, it’s on autopilot. It’s one level of abstraction below the concept of walking or lurking.

Likewise, if you do think about your walking form, you’re probably not thinking about which muscles are twitching and contracting to create the locomotion. While the percentage contraction of your calf is critical to move in the direction you intend, as is the force exerted by your quad, you don’t typically pay much attention to that. 

Why would you? That part is on autopilot. The lower level mechanisms are, in this sense, abstracted away. The abstract concept of walking or lurking implies that your muscles will be doing what they need to do for you to have the gait you need to have for you to go in the direction you want to go. You don’t need to “manually” direct these things, but they are there.

Typically with a walk, you’re focused on where you’re going. With the lurk, you’ve assumed that that’s already taken care of as well and, resultantly, you abstract that away, too. It’s wandering that puts you at a high level of abstraction–ideas and thoughts. It implies flaneuring with the intention of focusing on ideas.

The actual mechanics of walking are irrelevant, as is the direction.

Goal Setting

While we first had to learn how to walk with an appropriate gait so that we could eventually abstract it away and stop thinking about it, you can also start at the most abstract level and work your way down.

One place I’ve done this for a while is in goal setting–I start with who the person I want to be sufficiently far in the future is, and then back solve for a yearly goal, and then select daily goals one layer below that. 

I actually think a shortcoming of mine is that I don’t always get rigorous enough about some of the intermediate steps and, resultantly, I find it more challenging to hold myself accountable to the longer term goals. Jumping down from a year to a day is quite extreme and certainly lacks some of the rigor you’d want to get where you’re going. How can you translate three or four hours of deep work a day directly into a financial goal? 

I’m jumping from a high level of abstraction, what I want to do in a year, to a very low level implementation of what I should do each day and how I should do it. Months, quarters, and weeks exist for a reason.

Funny enough, I’ve found that this mirrors a problem I have when building products; rightly so, I’ll start with a quite high level and abstract goal and direction. However, then I’ll jump right into implementing some little particular part of it without understanding what each piece of the chain looks like. 

I know that I want to walk, but then I jump to building a muscle without thinking of how it might function in concert with other muscles, or even the motion the leg should make when all of the different muscles are working together.

I wrongly assume that a bias towards actions implies jumping into the most granular action.

Too Deep in the Trenches

While my tendency to jump into building things has worked in some regards, I’ve already ran into a problem building BirdDog because of it. 

First, we built a rough outline for what we wanted our product to do technically. And when I say rough, I mean rough.  Then, I jumped right into coding one part of it, our WebCrawler & Scraper, Shelob. I spent probably 10-15 hours of focused coding work on it, including one rewrite. When I thought it was sufficient, I jumped into the next part of our pipeline, Haderach. 

Unfortunately, I quickly realized how many bad assumptions I baked into Shelob and the data structure that would support it, as well as just how much more rewriting this would imply. 

Thankfully, Gurish knocked some sense into me and told me that if I was rewriting that much of something that frequently, I probably wasn’t building it modular or abstract enough. 

So, I re-approached the problem from the top down rather than the bottom up. This time, I’m not skipping any steps, though.

The Code Lurk

In the past, if I needed to learn how to walk, I would pretend that I already knew how to walk. Then, I would pick the direction. After that, I would start figuring out how to control my muscles. 

This time around, after I picked the direction, I’m going one layer lower rather than jumping all the way down. Now, I’m designing the gait before the fine tuning of the muscles, and it’s working quite well.

In the past, I would do a rough outline of layer 1, and then jump down to the lowest layer. Now, I’ve coded the Access Point with fake functions from layer 1. Then, I started coding layer 1 with fake functions from layer 2. Then, I started making the layer 2 fake functions real. All of the information learned from the design of each layer will help me finalize the data structures underlying the whole thing. 

And really, you could argue that each layer has sub layers, too, given that there are one or two for each of Modules 1-3 in Layer 1 that are exposed to the access point. Each of those exposed functions are composed of functions defined in the classes of layer 1.

Lower Levels as Products

By designing from the top down, I am preemptively solving problems that I now know I otherwise would have ran into, which is an absolutely beautiful thing. Moreover, I’m able to reduce the total amount of code by designing each lower level of the system with the intention of using the same functions throughout multiple parts of the codebase. 

By exploring as much of the surface area of the top level of abstraction as possible, you’re able to identify patterns in what you’ll need from the next level down. That way, the lower levels can be as simple and modular as possible to save you time refactoring later.

In a way, constructing the lower level is like building a product for the higher level. When you’re building a product, you don’t build a widget and start looking for customers to use it. You get to know the customers and their problems as intimately as possible and validate the top levels of your proposed solution in the most lightweight manner that you can.

You change your gait because you want to walk faster, not because you designed the muscles in a certain way. The implementation is subservient to the goal. 

Likewise, you change the product design because it will better serve your client, not because it will be easier for you to write due to some poor lower level designs you’ve implemented.

While you need to play within your constraints, you shouldn’t give yourself more artificial constraints in exchange for a greater sense of motion and progress at the start.

Given that I’m very much a learn by doing sort of person, this was a good reframe in that it redefined execution for me. 

Execution does not mean that you need to immediately see the program do a thing; an architect does not start by digging a hole.

Live Deeply,