It’s a happy egg!

I was focusing on learning how to do a masking around the egg, and doing a colored wash background.

This was my first watercolor painting with “real” paints and brushes, and it’s much more fun with better paints. The pigment is can be stretched much further, and layering effects work out in an interesting way.

I’ve been on a watercolor kick for the last 1-2 months, and I think I’m actually starting to understand how to pigment behaves, and I’m starting to enjoy myself.

This was my first ambitious watercolor project, I think I spent ~10 hours on it total.

All ideas come from somewhere. All new ideas are just old ideas remixed into a new form. Sometimes when I’m remixing ideas, an attribute of one idea can amplify the impact of another concept. That’s how you know you’ve found a good idea.

I love dreaming up new ideas. I do it all the time; it’s probably my favorite hobby.

My strategy for brainstorming new ideas is to look at other ideas, bang them together, and see if something interesting happens. I tried to come up with an analogy for this, but the only visualization I could think of would be like a particle accelerator. In a particle accelerator, atoms get hurled at each other at high speed. Sometimes when the atoms collide, exciting things happen.

Ideas work the same way.

Smash a few ideas together at high speed, and you’ll start to pick up on patterns, and loose associations between distinctly different concepts. If you keep smashing ideas together, eventually, you’ll see a spark, and you know you’ve got something.

I love starting side projects, but I almost never finish them.

I have always been this way. I come up with an idea, convince myself that it’s a good idea, then do a deep dive into the nitty-gritty details of the concept, to determine if it could “be a thing.”

I’ve always felt guilty when I look back on the graveyard of half-built ideas that never saw the light of day. Thinking thoughts like “If I would’ve just done X, the project would’ve worked out.”.

But recently I’ve discovered a new perspective.

It’s not a graveyard of half-built ideas; it’s a bookshelf full of stories and hands-on, hard-earned experience. At any point in my future, I can pull an idea off the bookshelf, relive the experience (reload into my brain RAM), and use that experience to my advantage. (Like a Zettelkasten!)

I’m not making side projects, I’m just taking on side quests!

My day-to-day life is my main quest, but when I’m feeling bogged down and tired of the grind, I pick up a side quest to switch things up a little. I try only to take on side quests where I’ll learn a new skill, gain some unique insight, or to explore something I’m entirely unfamiliar with. That doesn’t always work out, sometimes I take a 20-minute Factorio side quest, and 12 hours later, I emerge with nothing gained.

When you think of it this way, the guilt melts away. I’m proud of my side quests, even if they never see the light of day.

Recently my team at work has been exploring the idea of using docker/kubernetes for deploying a disparate set of web applications. To date, we’ve always just deployed web applications directly to a VM/cloud instance, but that pattern is starting to feel antiquated as docker and Kubernetes have made considerable leaps in progress in recent history.

I’ve always been a fan of Docker on some level β€” I was actually a very early adopter/user of docker before it was even publicly released. I had built some early infrastructure for running data processing pipelines on top of OG docker, but we eventually killed it for a more stable and less complex platform. It turns out; building production-grade systems atop an API that changes and gets completely rewritten every other week is hard 😬

It seems like Kubernetes has made major leaps and bounds towards making containers usable for deploying web applications that require multiple components (like databases, redis, etc). To the point that deploying applications with Kubernetes seems EASIER in some ways that manually maintaining virtual machines, keeping the kernels up to date/secure, and monitoring your entire stack. With Kubernetes you build tooling to maintain your cluster as a whole, but then all of the complexity of application-layer stuff gets defined inside the Dockerfile and podfile yml configurations. So in some ways, it might actually be easier to deploy web apps in Kubernetes than to have to worry about the entire stack of software that you don’t use which lies on the host operating system.

Anyway, I’m still a noob at Kubernetes, and I’m actively trying to build some intuition around it, to see whether it would actually be a good fit for what I’m looking for (run 20-50 small web apps with minimal maintenance and effort).

Hardware > Virtual Machines

I’ve found that I often build a more visceral intuition for how distributed systems are working if I have a physical understanding of how everything is laid out. So while it’s 100% possible to use virtual machines, and likely the best (cost to benefit ratio) strategy for building a production Kubernetes cluster β€” I wanted to build a physical cluster.

So I started poking around at the idea of making a little Raspberry Pi Kubernetes cluster with a minimal form factor.

The idea is to make an all-in-one lab environment that I can grab from the closet, plug in a single ethernet port, and a single power cord, and have everything up and running β€” and ready for experimentation!

The best resource I found on this stuff is http://www.pidramble.com/ by Jeff Geerling. On the pidramble.com website, he put together a bunch of parts lists and general thoughts on how he organized his Raspberry Pi cluster.

The best part about his cluster setup, in my opinion, is the form factor. If you include a Raspberry Pi PoE hat, you can pull power directly from the ethernet connection which makes things significantly more simple. It checks all of my boxes in terms of a self-contained cluster that’s easy to set aside, and easy to bust out for experimentation.

Check this out:

To Raspberry Pi or Not???

I shopped around a bunch and floated this idea to some of my friends. The most critical piece of feedback I received was that Raspberry Pis are underpowered; and that there may actually be better options.

One such option is the ODROID-MC1. The ODROID-MC1 is a compact 4-host compute cluster with a total of 32 CPU cores and 8GB ram. At first, this seemed like a huge win, as the compute per dollar capability is roughly double that of a Raspberry Pi 4B cluster; which weighs in at 16CPU cores and 8GB ram.

The downside, however, is the form factor and effort required to set things up. At first glance, it looks amazing. But if you want to run all of the nodes off of a single power supply, you need to purchase a cheapo whitelabel Chinese power supply, and manually wire up all of the 5.5mm power connectors like the image below. At first, I thought this sounded fun, but I have a long history of destroying electronics by making stupid mistakes with electricity β€” so this approach comes with an elevated risk.

The other option is to purchase 4x 5V/10A power bricks, which means I would also need to add a power strip to the build, which further hurts the form factor.

Learning LED Indicators (future improvement)

Another super interesting addition to this cluster project would be to hook up some indicator LEDs like the BlinkStick Nano. You could write some code that uses these USB LED indicators to change colors based on different behavior/states of each host. This would be super cool because it would help you develop a clear intuition for how a high availability Kubernetes cluster works.

img_3487

Small additions like this make this an even more interesting learning environment for distributed systems. But it takes a lot more work to write the monitoring code to trigger the LEDs, so this would definitely be an in-the-future project addition.

An example scenario for how these LEDs could work would be something like:

  1. Disconnect ethernet
  2. Disconnected host LED turns orange – to indicate degraded state.
  3. Containers get automagically re-provisioned (scheduled) to other hosts.
  4. LEDs on other hosts start blinking yellow to indicate that new pods are scheduled, but have not yet been started.
  5. After a minute, all hosts but the unplugged host turn green, while the degraded host remains orange.

So what’s the game plan?

So I’ve decided to roll with the Raspberry Pi cluster, for the form factor alone. Though I also suspect that the software support and resources are significantly better for the Raspberry Pi, given projects HypriotOS (a container operating system setup for running docker).

Below you’ll find a parts list including links and breakdowns of both the Raspberry Pi and ODROID-MC1 options.

At the time of writing, the Raspberry Pi 4B cluster weighs in at ~$385, while the ODROID-MC1 setup costs ~$355.

The Pi cluster is a little more expensive but provides a better form factor for being a learning tool.

I’m purchasing everything today, so sometime in the next month or so I may have a followup post for how this all goes.

Raspberry Pi 4B 2GB Cluster

Specs:

  • Cost: ~$385
  • 16 Cores (Broadcom BCM2711)
    • 4 cores per host @ 1.5GHz
  • 8GB RAM
    • 2GB per host

Raspberry 4B 2GB Parts List (~$385)

ItemQty.PriceCostNotes
Raspberry Pi 4B 2GB4$35.00140https://www.canakit.com/raspberry-pi-4-2gb.html
PoE Pi Hat4$20.0080https://www.canakit.com/raspberry-pi-poe-hat.html
128GB Samsung Evo MicroSD4$22.4189.64https://www.amazon.com/gp/product/B07BS3HLY9
Netgear GS305P 5port PoE Switch1$50.0050https://www.amazon.com/NETGEAR-Gigabit-Ethernet-Unmanaged-Desktop/dp/B01MRO4M73
Stackable Raspberry Pi Chasis1$25.0025https://www.amazon.com/GeeekPi-Raspberry-Cluster-Cooling-Heatsink/dp/B07MW3GM1T

ODROID-MC1

Specs:

  • Cost: ~$355
  • 32 Cores (Exynos 5 Octa 5422)
    • 4x per host @ 2.1GHz
    • 4x per host @ 1.4GHz
  • 8GB RAM
    • 2GB per host

ODROID-MC1 Parts List (~$355)

ItemQty.PriceCostNotes
ODROID-MC11$220.00220https://www.hardkernel.com/shop/odroid-mc1-my-cluster-one-with-32-cpu-cores-and-8gb-dram/
5v/20A Power Supply1$20.0020https://www.amazon.com/gp/product/B01IMP3V4Q
128GB MicroSD card4$22.4189.64https://www.amazon.com/gp/product/B07BS3HLY9
12x 5.5mm pigtail power plugs1$7.007https://www.amazon.com/gp/product/B07199792W
NEMA 5-15P to Unterminated1$17.0017https://www.amazon.com/Parts-Express-Power-Cord-Black/dp/B072YNYPZR

Spent some time this evening sketch doodling in Procreate. I ended up making a liquified comet / asteroid thing.

These thoughts stem from reflecting on my current mental state and trying to figure out how I got to where I am now: depressed, angsty, depleted, and burnt out. The last few months, and especially the last few weeks, have been incredibly taxing for me on my psyche.

I went from “living the dream”, being self-employed, and running my own business to working for “the man”, in the blink of an eye. I went from blissfully following COVID19 stay-at-home orders, and being my best hermit-self; to having my heart and soul completely crushed as I watched our global cultural awareness begin to come to grips with the reality of racial injustice and police brutality.

On some level, I think that my mental state has a lot to do with how I’ve been consuming information, and how that strategy tends to drown out ideas and inspiration.


Information

When you are a conduit for information: you consume, internalize, and regurgitate the interesting bits.

Consuming information can be largely mindless; and if you let the information hijack your brain, you feel the information more viscerally, and begin to unwittingly overheat and burn out.

Ideas

When you are a conduit for ideas: you consume, become inspired, and create new and interesting things.

Consuming ideas can be mindless too, but when you consume an idea, it becomes illuminated with all of the other tangentially related connections floating around in your mind. In some cases, this new illumination can trigger a eureka moment that changes your life and the world.


This is an interesting mental model that I’ve started to see in myself. In many ways it feels like the more “information” I consume, the more burnt out I become. Just keeping up-to-date with what’s happening on Twitter, Facebook, and the news has become wholly exhausting. The more “news” and “social media” I consume, the further downward I spiral.

Ideas feel different, though. When I consume a new idea, I fall into a natural optimism where I can’t help but be motivated to remix the idea into something new.

For me, creatively remixing thoughts and ideas is the nuclear fusion of optimism and motivation.

I guess what I’m saying is: It feels like it’s time to stop mindlessly consuming information about how fucked we all are, and time to start collectively thinking our way into a better future, together.

At the end of march I returned full-time to working at Planet due to all of the uncertainty caused by the COVID19 pandemic. Overnight I went from using video chat once every 2-3 months to multiple times a day.

I hadn’t done much painting in the last few months, and I wanted something simple to put up on my wall behind me, as my zoom video conference background. I had a few 6×6″ thick canvases that were stashed away waiting to be painted.

In my office I have a dual workspace. One desk for working+gaming, and another desk for art (where I can splatter paint and get messy). My webcam for video conferences faces my messy desk (which isn’t actually that messy… just artsy!). So I had the thought that it might be cool to fill my background with vibrant colorful art, which would serve the dual purpose of having a cool background for video calls, and some bright inspiration to have in my workspace. I spend like 8-12hr/day in that room, so might as well make it artsy/nice.

To get my art wall started, I decided to paint a paint 4 6×6″ canvases with simple color schemes that make me happy.

I remembered Gal Shir’s old medium post about the creation of ColorHunt, and how he spec’d out what the sizes/percentages were for color bars on ColorHunt, and I decided to give that a shot. Here’s a photo that he shared in his medium post that inspired my idea.

stolen from Gal Shir @ https://medium.com/@galshir/color-hunt-behind-the-scenes-3e32573f7752

I browsed through colorhunt to find a few color palettes that made me happy, and came up with a quick mockup like this:

Then I started painting! Below are the photos that I took along the way. I still haven’t taken the time to hang them on the wall, but I’m super happy with how they turned out.

A few months ago I went on a multi-night binge spree of watching Adam Savage’s One Day Builds. If you haven’t watched them, I highly recommend setting aside a few hours and watching 2-5 of them at 2x speed, it’s sooooo satisfying!

I really love these videos because they give you a glimpse into how Adam Savage thinks about building physical objects. He has a huge amount of experience and just watching him work teaches you so much about how to think about building things, using/making tools, and how to sketch in real-time with physical objects.

What I mean by “sketch with physical objects” is that there’s a creative process happening, and instead of drawing something out on paper, it starts to take form as you slowly assemble the physical pieces. To me, that feels a lot like sketching on the iPad or on paper. It’s the same mindset, the same feeling, with a different medium — and to me, that’s super cool!

Shortly after this binge spree, I went to an art store in Denver. This particular art store is my favorite because it’s massive and they have a huge selection of paints as well as printmaking stuff. It’s a super cool store, it kind of reminds of the Flax art supply store that used to live on Market Street in San Francisco.

As I was slowly browsing the shop, I found some super cheap sheets of basswood, and I suddenly had an idea!

During that time I was drawing super frequently on the iPad with Procreate, but I was having a hard time positioning the iPad at the perfect angle for drawing — so I was kind of constantly uncomfortable. The closest thing to perfection was when I put my phone underneath the iPad, which created a ~45ΒΊ angle, which felt perfect.

So I channeled my inner Adam Savage, and decided that I was going to buy some of this basswood, and make a little stand that propped up the iPad at the perfect angle. I had no idea how to do what I wanted, but I was excited to dive in.

Below are photos I took throughout the build that kind of demonstrate my thinking.

It’s not perfect, but it is exactly what I wanted. I learned how to work with a new material (basswood), and I also learned that it warps in weird ways when you blast it with a heat gun to dry paint because you’re impatient 😬

Any day that you build a new tool that fits the task at hand is a good day.