On Exponential Learning

XLVI

2024.05.12

After reading over some of my early journals this weekend, this post was going to be about how cool journaling is. There were a million and one insights I got into my younger self that were obscured by the sands of time that now I could write about.

However, I this post turned into a follow up on the last one, in which we talked about learning vs memorization. 13 year old me left me perfect example on the super linear nature of learning.

Memetics 101

When I was 13, near the start of my journals, I had this fixation on “memes” and group dynamics. Near the beginning of my eighth grade year, one of the first lines in my journal read as follows:

Then, I got into briefly mentioning the origin of the word meme and skimmed the surface of memetics. The strong start went south as I started insisting on the idea that you could test the hypothesis we were differentiated from apes by showing that we were able to resist the urge to say a meme. And, since I was quite religious, I got bonus points by stating that passing such a test would also be a proof for god.

Well, I went on to prove that God was real by catching myself before repeating some meme, and then from there, after failing at a few attempts to incubate a new meme, analyzed a couple other times when I or someone else did so in elementary school. This seems to have given me some new found resolve, because I went on to formulate a new, more robust experiment.

Non only was I on a mission to spread a meme amongst the other middle schoolers, but also amongst 12th graders, attendees of a pre-school day care, a group of college students, and “some sort of group of workplace adults.” See, I already had an intuition for needing to repeat the experiment in varying conditions to test its robustness. I even went as far as acknowledging that if the meme were to spread from one group to another before some incubation period was over, that would invalidate the experiment. After all, we were not testing if a meme could spread from adults to kids, we were testing if we could independently incubate it in each group.

Social Media is… alive!

Based on my journal entries, the grand experiment taper off without much fanfare. Years later, though, when I was 16, another entry indicates how I felt about social media:

I was obviously at the point where I though “MoRe bIgGeR wOrd” = “smarter,” but I did understand a very important point that is not immediately obvious—if you viewed the world from the eyes of social media, the goal was to continue to get people to use it. Our usage is the means by which the platform survives.

This idea, while not directly related to my 13 year old experiment, was very much down the same line of thinking about information and how it spreads.

Later, when I was 18, I wrote some short story and poems about my angst around how perfect social media was at rapidly testing what sort of content each user responded to and kept delivering that.

What bothered me most here was that any one piece of content you were delivered didn’t mattered; what mattered was that the content being created and delivered was perpetually honing in on the slowly moving target of what you and other people in some sub group found to be entertaining.

It was the illusion of individuality, being about “me,” when everyone was living the same way by using the same apps, receiving slightly different mixes of the same joke as someone else. It’s some strange scale up of personalization. Again, the important part is that this idea was in the same vein as the latter two.

The Selfish Gene

I read The Selfish Gene by Dawkins when I was 20. That gave me words for everything I’ve written above, colored it with nuance, and allowed me to actually understand memetics and biological evolution conceptually, two instance of roughly the same thing. In a way, it was a culmination of everything before it that also opened up new, non obvious doors.

You have these four sequential data points:

1. Meme experiment when 13

2. 3 years later, saw social media as optimizing for itself

3. 2.5 years later, understood the a/b testing involved with social media optimization

4. 2 years later, read the Selfish Gene

Chronologically, you can draw a line through these and say that the amount of things I knew here increased linearly, but I would disagree; each individual idea was more complex and less conceptual than the last and had a bigger impact on my life and world view.

After all, having some sort of understanding of memetics made it a lot easier to jump to understanding genetics relatively quickly. But, it gets better than that.

Genetic Algos

Last year, when I still was not very good at coding, I was exploring optimization functions, and one stuck out to me—genetic algorithms.

In general, if you have some sort of strategy with a lot of different variables that you can change, as well as possible random variables outside of your control impacting the environment, it is hard to pick the “best performing” strategy. However, that’s effectively the same challenge your genes face when trying to survive in the real world. The answer, of course, is that if the organism is more “fit,” it gets to propagate more, which is good for the genes.

Using this concept, the same way that you might make random changes on a website to see what color gets more clicks, you can change a lot variables in something like a trading strategy and simulate what is more “fit” by seeing what performs better across different market scenarios.

You then use the most “fit” strategies to influence the production of new strategies, allowing, of course, for some random variance. Then, you run the simulations again, and rinse and repeat a lot of times.

So, I coded up a genetic algo to optimize a trading strategy on whim 2 or 2.5 years after I read the Selfish Gene. This time gap matters because this is the same jump in time between when I understood that social media apps optimized for their own self interest and when I understood that a/b testing at scale was the bread and butter of how those apps worked. However, this more recent jump seems to me to have a higher impact on what I’m capable of doing in the world.

It’s hard to qualify the value of each of these jumps aswell as the comparative effort that went into each bit of new understanding, especially given the confounding variable that I got older and “smarter” along the way. In regards to the value of each jump, optimizing a trading strategy with genetic algos certainly sounds impressive and complicated compared to the other things, but it’s interesting in the sense that based on what I already understood, it wasn’t that complicated at all. In regards the point on me being older and knowing more, that should actually support my claim that learning is non linear.

Directional Knowledge Graph

To code a genetic algorithm for a stock market simulation, there were a number of other things I needed to understand, and each of those things required other things as well.

I needed to know how to simulate the stock market, but to do that, I needed to know how to program and at least one theory as to how the market worked. That would be made easier by understanding how stocks worked, both atomically and within a market system.

While many of these things are path dependent, the biggest take away of this graph is that certain pieces of information are the product of one or more pieces of information. In other words, for me, understanding how to code a genetic algo relatively easily was a product of understanding evolution, programming, simulating the stock market, and optimizing trading strategies. Understanding all of those things allowed me to understand this new, more specific thing, almost “for free.” I spent effort learning four things and got the fifth one at very little cost.

Something my graph entirely misses is ranking the comparative power of different ideas. I think there might be something about the centrality of an idea being correlated with its power. In other words, the more parts of the world an idea can help you understand, the more powerful the idea is.

And there’s something to narrowing into something to understand it’s nuance, and then broadening out to see how it applies to other things; meaning, I understood evolution easily because I had an understanding of memetics and how it mapped to genetics. And then, of course, both of these map over to the beautiful genetic algo optimization, which, again, practically came for free to me.

Keep Learning

The point of this post was not to remind you that I was weird and had few friends in middle school, the point is more so to reflect on the non linear nature of understanding.

Something else worth noting is that while coding the genetic algorithm project made me a better programmer, looking back on it, the code is now, by my standards, abysmal. I did the whole thing with classes, meaning I had a class for the strategy, each individual simulation, each collection of simulations, and the game as a whole.

Really, if I were to rewrite the code now, I would probably achieve most of it simply by making a more elegant data structure. I’m bringing this up, because this is a product of me being a better programmer with more experience, learning about lisp (where the data structure is the code), and understanding that in the real world, that is how things work, as well—our data structure, the DNA, is the program for building us. Read the Fabric of Reality for more on that.

The goal is to discover the powerful ideas and connect them in ways that make any field easier to understand and operate in. The idea that the data structure is the code seems to be an insanely powerful one, but it’s something only presenting itself as a product of literally everything else I have ever known.

Please excuse me if this was a little bit more windy than usual—we didn’t end up at all near the place I thought we would when I started writing.

But, thus is life. Another powerful heuristic to add to the center of the graph.

Live Deeply,