Bridgetown2021-11-13T20:32:27-08:00/feed.xmlCamertron’s BlogMy thoughts on programming, mostly using Ruby and Rails.Some Thoughts on Technical Debt2021-11-09T20:53:00-08:002021-11-09T20:53:00-08:00/2021/11/09/some-thoughts-on-technical-debt<p>My thoughts on a <a href="https://www.redhat.com/en/compiler-podcast/what-is-technical-debt">“recent episode of the Compiler podcast”</a> entitled “Do We Want a World Without Technical Debt?”</p>
<hr />
<p>I’m a big fan of pretty much everything Saron Yitbarek does, and recently one of those things has been the excellent <a href="https://www.redhat.com/en/command-line-heroes">Command Line Heroes</a> podcast. The most recent show was actually promoting another podcast called Compiler, specifically their fourth episode entitled <a href="https://www.redhat.com/en/compiler-podcast/what-is-technical-debt">“Do We Want a World Without Technical Debt?”</a>.</p>
<p>This was a really thought-provoking episode. I found myself trying to add my opinion out loud while on my daily walk with my daughter. Considering she’s only two and couldn’t hear the audio, the conversation was fairly one-sided. Fortunately at the end of the episode, hosts Angela Andrews and Brent Simoneaux ask their audience to weigh in. This post is an attempt to write down my thoughts.</p>
<h2 id="what-is-technical-debt">What is Technical Debt?</h2>
<p>In my experience, technical debt is the cost of choosing a faster, easier, or partially complete solution over a more thoughtful, time-consuming, or correct one. Most of the time these decisions are made because of time constraints. “We need this feature yesterday!”</p>
<p>Anyone who’s worked for a few years in the software industry has likely encountered these opposing forces of time and correctness. Given infinite time, theoretically a software development team could produce a completely correct and consistent system. The problem of course is that infinite time - or even sufficient time - is often not available. In these situations, software development teams frequently choose to incur technical debt, producing a system that works <em>well enough</em> for the given set of requirements. The idea is that, at some future time, they’ll revisit the problem and implement a more robust solution.</p>
<p>Sounds like a fair tradeoff, right?</p>
<h2 id="when-tech-debt-accumulates">When Tech Debt Accumulates</h2>
<p>Unfortunately in my experience, paying down tech debt only rarely happens in practice. It’s not difficult to understand why. Many teams find it challenging to justify revisiting a feature that’s already working for customers. In addition, software development teams are always under pressure to fix bugs and ship new features, which means they often prioritize those things over less glamorous tasks like reducing debt.</p>
<p>As you might imagine, ignoring tech debt and over again can lead to a large accumulation of it. Software is built on other software, meaning that hastily written code not only adds tech debt of its own but has the potential to affect any of the code written on top of it as well. Even in small systems, failure to address tech debt in a timely manner can gradually and perniciously slow down development, leaving the team with increasingly less capacity even though the size of the team itself hasn’t changed.</p>
<h2 id="so-is-debt-bad-then">So is Debt Bad Then?</h2>
<p>I realize I just made tech debt sound like a bad thing.</p>
<p>As you might have noticed, conversations about debt - tech debt included - are usually framed in the negative as if debt is this horrible thing we should avoid. The truth is, debt is an extremely important and useful tool. Let’s examine how it works in its native environment: the financial world.</p>
<p>The general public is probably most familiar with the sort of debt that comes with using a credit card or buying a house or car. It’s probably not a stretch to say that most people don’t have $25k lying around at any given point to drop on a car. Instead, they finance the purchase by getting an auto loan. They make monthly payments on that loan until the lender has received back the full amount. In this scenario, everybody wins. The lender makes money by charging interest, and the buyer gets to drive away in a new car even though they can’t afford the full purchase price up front. Such a transaction would have been impossible without debt.</p>
<p>Financial institutions, governments, and businesses use debt all the time. The US government for example issues bonds, which are sort of “reverse” loans where individual people loan money to the government. Venture capital firms loan money to start-ups they hope will take off and make them large returns. You can even see debt at work between whole countries.</p>
<h2 id="tech-debt-through-the-financial-lens">Tech Debt Through the Financial Lens</h2>
<p>For whatever reason, I haven’t heard very many programmers talk about why “technical debt” contains the word “debt.” It’s a curious choice of words, but really very appropriate.</p>
<p>Imagine for a minute that every decision you make as a programmer is a financial transaction. Our currency is time. Decisions that don’t require trading large amounts of time for correctness are simple, direct transactions. Decisions in which you sacrifice correctness in the interest of time are <em>loans</em>. The loan represents time borrowed from the bank - time you promise to repay later. At the risk of stretching the metaphor a little too far, we might even say the bank collects interest since it always takes more time to dive back into code you wrote a while ago.</p>
<p>Again, everybody wins. With their borrowed time the team can deliver systems and features faster. The bank will eventually get its time back plus interest.</p>
<p>I like this parallel to the financial world because it makes it more obvious why it’s dangerous to ignore tech debt. Taking on too much financial debt can lead to monthly payments you can’t afford. Miss too many payments, and the bank will repossess (take back) your car. Taking on too much tech debt can have a similar effect. Instead of working on a new feature, you might be forced to spend all your available time dealing with flaky test suites, awkward database relationships, and production emergencies.</p>
<p>If tech debt can cause such headaches, let’s ask the obvious next question: is it possible to get rid of it entirely?</p>
<h2 id="tech-debt-is-unavoidable">Tech Debt is Unavoidable</h2>
<p>Unfortunately, no.</p>
<p>The most successful teams I’ve seen that manage their tech debt effectively do so by including it in their planning. They track it in their software management system and discuss how much time should be allocated to reducing it every sprint. Sometimes, an entire sprint is dedicated to refactoring a certain part of the code to make it easier to implement a new feature the <em>following</em> sprint.</p>
<p>However, tech debt accumulates even in systems nobody’s working on.</p>
<p>If you’ve ever tried to dust off and boot up an old app, you’ve likely run into the phenomenon of <em>bitrot</em>. Code (almost literally) <em>rots</em> as it ages. Bitrot happens because our apps are made up of so much more than just our own code. They depend on a bunch of other software packages, build tooling, CI pipelines, 3rd-party APIs, and external systems. A software package that built successfully on your laptop with Clang 3.5 in 2014 now dumps out a whole mess of compiler warnings and an esoteric error message. The version of the 3rd party API your app relies on was deprecated in 2015 and removed in 2016. The latest version of MySQL no longer supports a specific type of column your app relies on. The list goes on and on.</p>
<p>While it’s true that, in actively maintained systems bitrot is less likely, such systems are still susceptible to forces outside of your control. The canonical example is probably upgrading to the next version of your app’s web framework. You want all the new features, bug fixes, and security patches in the new version, but the upgrade isn’t going to happen without a serious investment of time and effort. In my experience these upgrades can sometimes take years to pull off.</p>
<p>New framework or library versions are an example of passive tech debt. Similar to bitrot, passive debt accumulates by no fault of your own. New versions signal that the framework and community are moving on without you. In fact, there will always be new versions of this lib or that, new technologies and new approaches for solving problems. Unless you’re upgrading on a regular schedule, this sort of issue can become a real thorn in your side. For example, it can be hard to attract talent to work with outdated technologies and tech stacks. Older libraries often don’t receive security patches either, meaning you’ll be on the hook for fixing them if vulnerabilities are found.</p>
<h2 id="embracing-tech-debt">Embracing Tech Debt</h2>
<p>It’s not all bad news though. Tech debt might be unavoidable, but that doesn’t mean it can’t be effectively managed. In fact, I would argue that any team making good progress on a software project must be taking on debt.</p>
<p>That’s because software is almost never written correctly the first time around. It often takes a number of refactorings, real-world usages, and mental model shifts before a piece of software settles into its final shape. Considering this, there’s no reason to spend time worrying about the correctness of your code, at least at first. It’s going to change, perhaps drastically, during the next iteration anyway. Rather than spending lots of precious time worrying about code correctness, focus on getting something out the door.</p>
<p>Frame the tech debt conversation in terms of the inevitability of bad code. In my opinion, there’s no such thing as “bad” code running in production anyway. If it’s making you money, it probably isn’t bad. With that mindset, debt becomes the sort of tool it is in the financial sector. Let it free you from worrying about writing flawless code, and from the negative connotations associated with the word “debt.”</p>
<p>Happy hacking :)</p>Cameron DutroMy thoughts on a “recent episode of the Compiler podcast” entitled “Do We Want a World Without Technical Debt?”Responsible Monkeypatching2021-08-24T12:00:00-07:002021-08-24T12:00:00-07:00/2021/08/24/responsible-monkeypatching<p>This is a <a href="https://blog.appsignal.com/2021/08/24/responsible-monkeypatching-in-ruby.html">post</a> I wrote for the AppSignal blog about how to monkeypatch without making a mess :)</p>
<hr />
<link rel="canonical" href="https://blog.appsignal.com/2021/08/24/responsible-monkeypatching-in-ruby.html" />
<p>When I first started writing Ruby code professionally back in 2011, one of the things that impressed me the most about the language was its flexibility. It felt as though with Ruby, everything was possible. Compared to the rigidity of languages like C# and Java, Ruby programs almost seemed like they were <em>alive</em>.</p>
<p>Consider how many incredible things you can do in a Ruby program. You can define and delete methods at will. You can call methods that don’t exist. You can conjure entire nameless classes out of thin air. It’s absolutely wild.</p>
<p>But that’s not where the story ends. While you can apply these techniques inside your own code, Ruby also lets you apply them to anything loaded into the virtual machine. In other words, you can mess with other people’s code as easily as you can your own.</p>
<h2 id="what-are-monkeypatches">What Are Monkeypatches?</h2>
<p>Enter the <em>monkeypatch</em>.</p>
<p>In short, monkeypatches “monkey with” existing code. The existing code is often code you don’t have direct access to, like code from a gem or from the Ruby standard library. Patches are usually designed to alter the original code’s behavior to fix a bug, improve performance, etc.</p>
<p>The most unsophisticated monkeypatches reopen ruby classes and modify behavior by adding or overriding methods.</p>
<p>This reopening idea is core to Ruby’s object model. Whereas in Java classes can only be defined once, Ruby classes (and modules for that matter) can be defined multiple times. When we define a class a second, a third, a fourth time, etc, we say that we’re <em>reopening</em> it. Any new methods we define are added to the existing class definition, and can be called on instances of that class.</p>
<p>This short example illustrates the class reopening concept:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">class</span> <span class="nc">Sounds</span>
<span class="k">def</span> <span class="nf">honk</span>
<span class="s2">"Honk!"</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">class</span> <span class="nc">Sounds</span>
<span class="k">def</span> <span class="nf">squeak</span>
<span class="s2">"Squeak!"</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="n">sounds</span> <span class="o">=</span> <span class="no">Sounds</span><span class="p">.</span><span class="nf">new</span>
<span class="n">sounds</span><span class="p">.</span><span class="nf">honk</span> <span class="c1"># => "Honk!"</span>
<span class="n">sounds</span><span class="p">.</span><span class="nf">squeak</span> <span class="c1"># => "Squeak!"</span>
</code></pre></div></div>
<p>Notice that both the <code class="highlighter-rouge">#honk</code> and <code class="highlighter-rouge">#squeak</code> methods are available on the <code class="highlighter-rouge">Sounds</code> class through the magic of reopening.</p>
<p>Monkeypatching is essentially the act of reopening classes in 3rd-party code.</p>
<h2 id="is-monkeypatching-dangerous">Is Monkeypatching Dangerous?</h2>
<p>If the previous sentence scared you, that’s probably a good thing. Monkeypatching, especially when done carelessly, can cause real chaos.</p>
<p>Consider for a moment what would happen if we were to redefine <code class="highlighter-rouge">Array#<<</code>:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">class</span> <span class="nc">Array</span>
<span class="k">def</span> <span class="nf"><<</span><span class="p">(</span><span class="o">*</span><span class="n">args</span><span class="p">)</span>
<span class="c1"># do nothing 😈</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>
<p>With these four lines of code, every single array instance in the entire program is now broken.</p>
<p>What’s more, the original implementation of <code class="highlighter-rouge">#<<</code> is gone. Aside from restarting the Ruby process, there’s no way to get it back.</p>
<h2 id="when-monkeypatching-goes-horribly-wrong">When Monkeypatching Goes Horribly Wrong</h2>
<p>Back in 2011, I worked for a prominent social networking company. At the time, the codebase was a massive Rails monolith running on Ruby 1.8.7. Several hundred engineers contributed to the codebase on a daily basis, and the pace of development was very fast.</p>
<p>At one point my team decided to monkeypatch <code class="highlighter-rouge">String#%</code> to make writing plurals easier for internationalization purposes. Here’s an example of what our patch could do:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">replacements</span> <span class="o">=</span> <span class="p">{</span>
<span class="ss">horse_count: </span><span class="mi">3</span><span class="p">,</span>
<span class="ss">horses: </span><span class="p">{</span>
<span class="ss">one: </span><span class="s2">"is 1 horse"</span><span class="p">,</span>
<span class="ss">other: </span><span class="s2">"are %{horse_count} horses"</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="c1"># "there are 3 horses in the barn"</span>
<span class="s2">"there %{horse_count:horses} in the barn"</span> <span class="o">%</span> <span class="n">replacements</span>
</code></pre></div></div>
<p>We wrote up the patch and eventually got it deployed into production…only to find that it didn’t work. Our users were seeing strings with literal <code class="highlighter-rouge">%{...}</code> characters instead of nicely pluralized text. It didn’t make sense. The patch had worked perfectly well in the development environment on my laptop. Why wasn’t it working in production?</p>
<p>Initially, we thought we’d found a bug in Ruby itself, only to later find that a production Rails console produced a different result than a Rails console in development. Since both consoles ran on the same Ruby version, we could rule out a bug in the Ruby standard library. Something else was going on.</p>
<p>After several days of head-scratching, a co-worker was able to track down a Rails initializer that added <em>another</em> implementation of <code class="highlighter-rouge">String#%</code> that none of us had seen before. To further complicate things, this earlier implementation also contained a bug, so the results we saw in the production console differed from Ruby’s official documentation.</p>
<p>That’s not the end of the story though. In tracking down the earlier monkeypatch, we also found no less than three more, <em>all patching the same method.</em> We looked at each other in horror. How did this ever work??</p>
<p>We eventually chalked the inconsistent behavior up to Rails’ eager loading. In development, Rails lazy loads Ruby files, i.e., only loads them when they are <code class="highlighter-rouge">require</code>d. In production, however, Rails loads all of the app’s Ruby files at initialization. This can throw a big monkey wrench into monkeypatching.</p>
<h2 id="consequences-of-reopening-a-class">Consequences of Reopening a Class</h2>
<p>In this case, each of the monkeypatches reopened the <code class="highlighter-rouge">String</code> class and effectively replaced the existing version of the <code class="highlighter-rouge">#%</code> method with another one. There are several major pitfalls to this approach:</p>
<ol>
<li>The last patch applied “wins”, meaning behavior is dependent on load order.</li>
<li>There’s no way to access the original implementation.</li>
<li>Patches leave almost no audit trail, which makes them very difficult to find later.</li>
</ol>
<p>Not surprisingly, perhaps, we ran into all of these.</p>
<p>At first, we didn’t even know there were other monkeypatches at play. Because of the bug in the winning method, it appeared the original implementation was broken. When we discovered the other competing patches, it was impossible to tell which won without adding copious <code class="highlighter-rouge">puts</code> statements.</p>
<p>Finally, even when we did discover which method won in development, a different one would win in production. It was also programmatically difficult to tell which patch had been applied last since Ruby 1.8 didn’t have the wonderful <code class="highlighter-rouge">Method#source_location</code> method we have now.</p>
<p>I spent at least a week trying to figure out what was going on, time I essentially wasted chasing an entirely avoidable problem.</p>
<p>Eventually, we decided to introduce the <code class="highlighter-rouge">LocalizedString</code> wrapper class with an accompanying <code class="highlighter-rouge">#%</code> method. Our <code class="highlighter-rouge">String</code> monkeypatch then simply became this:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">class</span> <span class="nc">String</span>
<span class="k">def</span> <span class="nf">localize</span>
<span class="no">LocalizedString</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="nb">self</span><span class="p">)</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>
<h2 id="when-monkeypatching-fails">When Monkeypatching Fails</h2>
<p>In my experience, monkeypatches often fail for one of two reasons:</p>
<ol>
<li><strong>The patch itself is broken.</strong> In the codebase I mentioned above, not only were there several competing implementations of the same method, but the method that “won” didn’t work.</li>
<li><strong>Assumptions are invalid.</strong> The host code has been updated and the patch no longer applies as written.</li>
</ol>
<p>Let’s look at the second bullet point in more detail.</p>
<h2 id="even-the-best-laid-plans">Even the Best-Laid Plans…</h2>
<p>Monkeypatching often fails for the same reason you reached for it in the first place - because you don’t have access to the original code. For precisely that reason, the original code can change out from under you.</p>
<p>Consider this example in a gem your app depends on:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">class</span> <span class="nc">Sale</span>
<span class="k">def</span> <span class="nf">initialize</span><span class="p">(</span><span class="n">amount</span><span class="p">,</span> <span class="n">discount_pct</span><span class="p">,</span> <span class="n">tax_rate</span> <span class="o">=</span> <span class="kp">nil</span><span class="p">)</span>
<span class="vi">@amount</span> <span class="o">=</span> <span class="n">amount</span>
<span class="vi">@discount_pct</span> <span class="o">=</span> <span class="n">discount_pct</span>
<span class="vi">@tax_rate</span> <span class="o">=</span> <span class="n">tax_rate</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">total</span>
<span class="n">discounted_amount</span> <span class="o">+</span> <span class="n">sales_tax</span>
<span class="k">end</span>
<span class="kp">private</span>
<span class="k">def</span> <span class="nf">discounted_amount</span>
<span class="vi">@amount</span> <span class="o">*</span> <span class="p">(</span><span class="mi">1</span> <span class="o">-</span> <span class="vi">@discount_pct</span><span class="p">)</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">sales_tax</span>
<span class="k">if</span> <span class="vi">@tax_rate</span>
<span class="n">discounted_amount</span> <span class="o">*</span> <span class="vi">@tax_rate</span>
<span class="k">else</span>
<span class="mi">0</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>
<p>Wait, that’s not right. Sales tax should be applied to the full amount, not the discounted amount. You submit a pull request to the project. While you’re waiting for the maintainer to merge your PR, you add this monkeypatch to your app:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">class</span> <span class="nc">Sale</span>
<span class="kp">private</span>
<span class="k">def</span> <span class="nf">sales_tax</span>
<span class="k">if</span> <span class="vi">@tax_rate</span>
<span class="vi">@amount</span> <span class="o">*</span> <span class="vi">@tax_rate</span>
<span class="k">else</span>
<span class="mi">0</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>
<p>It works perfectly. You check it in and forget about it.</p>
<p>Everything is fine for a long time. Then one day the finance team sends you an email asking why the company hasn’t been collecting sales tax for a month.</p>
<p>Confused, you start digging into the issue and eventually notice one of your co-workers recently updated the gem that contains the <code class="highlighter-rouge">Sale</code> class. Here’s the updated code:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">class</span> <span class="nc">Sale</span>
<span class="k">def</span> <span class="nf">initialize</span><span class="p">(</span><span class="n">amount</span><span class="p">,</span> <span class="n">discount_pct</span><span class="p">,</span> <span class="n">sales_tax_rate</span> <span class="o">=</span> <span class="kp">nil</span><span class="p">)</span>
<span class="vi">@amount</span> <span class="o">=</span> <span class="n">amount</span>
<span class="vi">@discount_pct</span> <span class="o">=</span> <span class="n">discount_pct</span>
<span class="vi">@sales_tax_rate</span> <span class="o">=</span> <span class="n">sales_tax_rate</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">total</span>
<span class="n">discounted_amount</span> <span class="o">+</span> <span class="n">sales_tax</span>
<span class="k">end</span>
<span class="kp">private</span>
<span class="k">def</span> <span class="nf">discounted_amount</span>
<span class="vi">@amount</span> <span class="o">*</span> <span class="p">(</span><span class="mi">1</span> <span class="o">-</span> <span class="vi">@discount_pct</span><span class="p">)</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">sales_tax</span>
<span class="k">if</span> <span class="vi">@sales_tax_rate</span>
<span class="n">discounted_amount</span> <span class="o">*</span> <span class="vi">@sales_tax_rate</span>
<span class="k">else</span>
<span class="mi">0</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>
<p>Looks like one of the project maintainers renamed the <code class="highlighter-rouge">@tax_rate</code> instance variable to <code class="highlighter-rouge">@sales_tax_rate</code>. The monkeypatch checks the value of the old <code class="highlighter-rouge">@tax_rate</code> variable, which is always <code class="highlighter-rouge">nil</code>. Nobody noticed because no errors were ever raised. The app chugged along as if nothing had happened.</p>
<h2 id="why-monkeypatch">Why Monkeypatch?</h2>
<p>Given these examples, it might seem like monkeypatching just isn’t worth the potential headaches. So why do we do it? In my opinion, there are three major use-cases:</p>
<ol>
<li>To fix broken or incomplete 3rd-party code.</li>
<li>To quickly test a change or multiple changes in development.</li>
<li>To wrap existing functionality with instrumentation or annotation code.</li>
</ol>
<p>In some cases, the <em>only</em> viable way to address a bug or performance issue in 3rd-party code is to apply a monkeypatch.</p>
<p>But with great power comes great responsibility.</p>
<h2 id="monkeypatching-responsibly">Monkeypatching Responsibly</h2>
<p>I like to frame the monkeypatching conversation around responsibility instead of whether or not it’s good or bad. Sure, monkeypatching can cause chaos when done poorly. However, if done with some care and diligence, there’s no reason to avoid reaching for it when the situation warrants it.</p>
<p>Here’s the list of rules I try to follow:</p>
<ol>
<li>Wrap the patch in a module with an obvious name and use <code class="highlighter-rouge">Module#prepend</code> to apply it.</li>
<li>Make sure you’re patching the right thing.</li>
<li>Limit the patch’s surface area.</li>
<li>Give yourself escape hatches.</li>
<li>Over-communicate.</li>
</ol>
<p>For the remainder of this article, we’re going to use these rules to write up a monkeypatch for Rails’ <code class="highlighter-rouge">DateTimeSelector</code> so it optionally skips rendering discarded fields. This is a change I actually tried to make to Rails a few years ago. <a href="https://github.com/rails/rails/pull/31533">You can find the details here</a>.</p>
<p>You don’t have to know much about discarded fields to understand the monkeypatch, though. At the end of the day, all it does is replace a single method called <code class="highlighter-rouge">build_hidden</code> with one that effectively does nothing.</p>
<p>Let’s get started!</p>
<h3 id="use-moduleprepend">Use <code class="highlighter-rouge">Module#prepend</code></h3>
<p>In the codebase I encountered in my previous role, all the implementations of <code class="highlighter-rouge">String#%</code> were applied by reopening the <code class="highlighter-rouge">String</code> class. Here’s an augmented list of the drawbacks I mentioned earlier:</p>
<ol>
<li>Errors appear to have originated from the host class or module instead of from the patch code.</li>
<li>Any methods you define in the patch replace existing methods with the same name, meaning there’s no way to invoke the original implementation.</li>
<li>There’s no way to know which patches were applied and therefore which methods “won”.</li>
<li>Patches leave almost no audit trail, which makes them very difficult to find later.</li>
</ol>
<p>Instead, it’s much better to wrap your patch in a module and apply it using <code class="highlighter-rouge">Module#prepend</code>. Doing so leaves you free to call the original implementation, and a quick call to <code class="highlighter-rouge">Module#ancestors</code> will show the patch in the inheritance hierarchy so it’s easier to find if things go wrong.</p>
<p>Finally, a simple <code class="highlighter-rouge">prepend</code> statement is easy to comment out if you need to disable the patch for some reason.</p>
<p>Here are the beginnings of a module for our Rails monkeypatch:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">module</span> <span class="nn">RenderDiscardedMonkeypatch</span>
<span class="k">end</span>
<span class="no">ActionView</span><span class="o">::</span><span class="no">Helpers</span><span class="o">::</span><span class="no">DateTimeSelector</span><span class="p">.</span><span class="nf">prepend</span><span class="p">(</span>
<span class="no">RenderDiscardedMonkeypatch</span>
<span class="p">)</span>
</code></pre></div></div>
<h3 id="patch-the-right-thing">Patch the Right Thing</h3>
<p>If you take one thing away from this article, let it be this: don’t apply a monkeypatch unless you know you’re patching the right code. In most cases, it should be possible to verify programmatically that your assumptions still hold (this is Ruby after all). Here’s a checklist:</p>
<ol>
<li>Make sure the class or module you’re trying to patch exists.</li>
<li>Make sure methods exist and have the right arity.</li>
<li>If the code you’re patching lives in a gem, check the gem’s version.</li>
<li>Bail out with a helpful error message if assumptions don’t hold.</li>
</ol>
<p>Right off the bat, our patch code has made a pretty important assumption. It assumes a constant called <code class="highlighter-rouge">ActionView::Helpers::DateTimeSelector</code> exists and is a class or module.</p>
<h4 id="check-classmodule">Check Class/Module</h4>
<p>Let’s make sure that constant exists before trying to patch it:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">module</span> <span class="nn">RenderDiscardedMonkeypatch</span>
<span class="k">end</span>
<span class="n">const</span> <span class="o">=</span> <span class="k">begin</span>
<span class="no">Kernel</span><span class="p">.</span><span class="nf">const_get</span><span class="p">(</span><span class="s1">'ActionView::Helpers::DateTimeSelector'</span><span class="p">)</span>
<span class="k">rescue</span> <span class="no">NameError</span>
<span class="k">end</span>
<span class="k">if</span> <span class="n">const</span>
<span class="n">const</span><span class="p">.</span><span class="nf">prepend</span><span class="p">(</span><span class="no">RenderDiscardedMonkeypatch</span><span class="p">)</span>
<span class="k">end</span>
</code></pre></div></div>
<p>Great, but now we’ve leaked a local variable (<code class="highlighter-rouge">const</code>) into the global scope. Let’s fix that:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">module</span> <span class="nn">RenderDiscardedMonkeypatch</span>
<span class="k">def</span> <span class="nc">self</span><span class="o">.</span><span class="nf">apply_patch</span>
<span class="n">const</span> <span class="o">=</span> <span class="k">begin</span>
<span class="no">Kernel</span><span class="p">.</span><span class="nf">const_get</span><span class="p">(</span><span class="s1">'ActionView::Helpers::DateTimeSelector'</span><span class="p">)</span>
<span class="k">rescue</span> <span class="no">NameError</span>
<span class="k">end</span>
<span class="k">if</span> <span class="n">const</span>
<span class="n">const</span><span class="p">.</span><span class="nf">prepend</span><span class="p">(</span><span class="nb">self</span><span class="p">)</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="no">RenderDiscardedMonkeypatch</span><span class="p">.</span><span class="nf">apply_patch</span>
</code></pre></div></div>
<h4 id="check-methods">Check Methods</h4>
<p>Next, let’s introduce the patched <code class="highlighter-rouge">build_hidden</code> method. Let’s also add a check to make sure it exists and accepts the right number of arguments (i.e. has the right arity). If those assumptions don’t hold, something’s probably wrong:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">module</span> <span class="nn">RenderDiscardedMonkeypatch</span>
<span class="k">class</span> <span class="o"><<</span> <span class="nb">self</span>
<span class="k">def</span> <span class="nf">apply_patch</span>
<span class="n">const</span> <span class="o">=</span> <span class="n">find_const</span>
<span class="n">mtd</span> <span class="o">=</span> <span class="n">find_method</span><span class="p">(</span><span class="n">const</span><span class="p">)</span>
<span class="k">if</span> <span class="n">const</span> <span class="o">&&</span> <span class="n">mtd</span> <span class="o">&&</span> <span class="n">mtd</span><span class="p">.</span><span class="nf">arity</span> <span class="o">==</span> <span class="mi">2</span>
<span class="n">const</span><span class="p">.</span><span class="nf">prepend</span><span class="p">(</span><span class="nb">self</span><span class="p">)</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="kp">private</span>
<span class="k">def</span> <span class="nf">find_const</span>
<span class="no">Kernel</span><span class="p">.</span><span class="nf">const_get</span><span class="p">(</span><span class="s1">'ActionView::Helpers::DateTimeSelector'</span><span class="p">)</span>
<span class="k">rescue</span> <span class="no">NameError</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">find_method</span><span class="p">(</span><span class="n">const</span><span class="p">)</span>
<span class="k">return</span> <span class="k">unless</span> <span class="n">const</span>
<span class="n">const</span><span class="p">.</span><span class="nf">instance_method</span><span class="p">(</span><span class="ss">:build_hidden</span><span class="p">)</span>
<span class="k">rescue</span> <span class="no">NameError</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">build_hidden</span><span class="p">(</span><span class="n">type</span><span class="p">,</span> <span class="n">value</span><span class="p">)</span>
<span class="s1">''</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="no">RenderDiscardedMonkeypatch</span><span class="p">.</span><span class="nf">apply_patch</span>
</code></pre></div></div>
<h4 id="check-gem-versions">Check Gem Versions</h4>
<p>Finally, let’s check that we’re using the right version of Rails. If Rails gets upgraded, we might need to update the patch too (or get rid of it entirely).</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">module</span> <span class="nn">RenderDiscardedMonkeypatch</span>
<span class="k">class</span> <span class="o"><<</span> <span class="nb">self</span>
<span class="k">def</span> <span class="nf">apply_patch</span>
<span class="n">const</span> <span class="o">=</span> <span class="n">find_const</span>
<span class="n">mtd</span> <span class="o">=</span> <span class="n">find_method</span><span class="p">(</span><span class="n">const</span><span class="p">)</span>
<span class="k">if</span> <span class="n">const</span> <span class="o">&&</span> <span class="n">mtd</span> <span class="o">&&</span> <span class="n">mtd</span><span class="p">.</span><span class="nf">arity</span> <span class="o">==</span> <span class="mi">2</span> <span class="o">&&</span> <span class="n">rails_version_ok?</span>
<span class="n">const</span><span class="p">.</span><span class="nf">prepend</span><span class="p">(</span><span class="nb">self</span><span class="p">)</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="kp">private</span>
<span class="k">def</span> <span class="nf">find_const</span>
<span class="no">Kernel</span><span class="p">.</span><span class="nf">const_get</span><span class="p">(</span><span class="s1">'ActionView::Helpers::DateTimeSelector'</span><span class="p">)</span>
<span class="k">rescue</span> <span class="no">NameError</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">find_method</span><span class="p">(</span><span class="n">const</span><span class="p">)</span>
<span class="k">return</span> <span class="k">unless</span> <span class="n">const</span>
<span class="n">const</span><span class="p">.</span><span class="nf">instance_method</span><span class="p">(</span><span class="ss">:build_hidden</span><span class="p">)</span>
<span class="k">rescue</span> <span class="no">NameError</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">rails_version_ok?</span>
<span class="no">Rails</span><span class="o">::</span><span class="no">VERSION</span><span class="o">::</span><span class="no">MAJOR</span> <span class="o">==</span> <span class="mi">6</span> <span class="o">&&</span> <span class="no">Rails</span><span class="o">::</span><span class="no">VERSION</span><span class="o">::</span><span class="no">MINOR</span> <span class="o">==</span> <span class="mi">1</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">build_hidden</span><span class="p">(</span><span class="n">type</span><span class="p">,</span> <span class="n">value</span><span class="p">)</span>
<span class="s1">''</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="no">RenderDiscardedMonkeypatch</span><span class="p">.</span><span class="nf">apply_patch</span>
</code></pre></div></div>
<h4 id="bail-out-helpfully">Bail Out Helpfully</h4>
<p>If your verification code uncovers a discrepancy between expectations and reality, it’s a good idea to raise an error or at least print a helpful warning message. The idea here is to alert you and your co-workers when something seems amiss.</p>
<p>Here’s how we might modify our Rails patch:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">module</span> <span class="nn">RenderDiscardedMonkeypatch</span>
<span class="k">class</span> <span class="o"><<</span> <span class="nb">self</span>
<span class="k">def</span> <span class="nf">apply_patch</span>
<span class="n">const</span> <span class="o">=</span> <span class="n">find_const</span>
<span class="n">mtd</span> <span class="o">=</span> <span class="n">find_method</span><span class="p">(</span><span class="n">const</span><span class="p">)</span>
<span class="k">unless</span> <span class="n">const</span> <span class="o">&&</span> <span class="n">mtd</span> <span class="o">&&</span> <span class="n">mtd</span><span class="p">.</span><span class="nf">arity</span> <span class="o">==</span> <span class="mi">2</span>
<span class="k">raise</span> <span class="s2">"Could not find class or method when patching "</span><span class="p">\</span>
<span class="s2">"ActionView's date_select helper. Please investigate."</span>
<span class="k">end</span>
<span class="k">unless</span> <span class="n">rails_version_ok?</span>
<span class="nb">puts</span> <span class="s2">"WARNING: It looks like Rails has been upgraded since "</span><span class="p">\</span>
<span class="s2">"ActionView's date_select helper was monkeypatched in "</span><span class="p">\</span>
<span class="s2">"</span><span class="si">#{</span><span class="kp">__FILE__</span><span class="si">}</span><span class="s2">. Please reevaluate the patch."</span>
<span class="k">end</span>
<span class="n">const</span><span class="p">.</span><span class="nf">prepend</span><span class="p">(</span><span class="nb">self</span><span class="p">)</span>
<span class="k">end</span>
<span class="kp">private</span>
<span class="k">def</span> <span class="nf">find_const</span>
<span class="no">Kernel</span><span class="p">.</span><span class="nf">const_get</span><span class="p">(</span><span class="s1">'ActionView::Helpers::DateTimeSelector'</span><span class="p">)</span>
<span class="k">rescue</span> <span class="no">NameError</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">find_method</span><span class="p">(</span><span class="n">const</span><span class="p">)</span>
<span class="k">return</span> <span class="k">unless</span> <span class="n">const</span>
<span class="n">const</span><span class="p">.</span><span class="nf">instance_method</span><span class="p">(</span><span class="ss">:build_hidden</span><span class="p">)</span>
<span class="k">rescue</span> <span class="no">NameError</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">rails_version_ok?</span>
<span class="no">Rails</span><span class="o">::</span><span class="no">VERSION</span><span class="o">::</span><span class="no">MAJOR</span> <span class="o">==</span> <span class="mi">6</span> <span class="o">&&</span> <span class="no">Rails</span><span class="o">::</span><span class="no">VERSION</span><span class="o">::</span><span class="no">MINOR</span> <span class="o">==</span> <span class="mi">1</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">build_hidden</span><span class="p">(</span><span class="n">type</span><span class="p">,</span> <span class="n">value</span><span class="p">)</span>
<span class="s1">''</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="no">RenderDiscardedMonkeypatch</span><span class="p">.</span><span class="nf">apply_patch</span>
</code></pre></div></div>
<h3 id="limit-surface-area">Limit Surface Area</h3>
<p>While it may seem perfectly innocuous to define helper methods in a monkeypatch, remember that any methods defined via <code class="highlighter-rouge">Module#prepend</code> will override existing ones through the magic of inheritance. While it might seem as though a host class or module doesn’t define a particular method, it’s difficult to know for sure. For this reason, I try only to define methods I intend to patch.</p>
<p>Note that this also applies to methods defined in the object’s singleton class, i.e. methods defined inside <code class="highlighter-rouge">class << self</code>.</p>
<p>Here’s how to modify our Rails patch to only replace the one <code class="highlighter-rouge">#build_hidden</code> method:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">module</span> <span class="nn">RenderDiscardedMonkeypatch</span>
<span class="k">class</span> <span class="o"><<</span> <span class="nb">self</span>
<span class="k">def</span> <span class="nf">apply_patch</span>
<span class="n">const</span> <span class="o">=</span> <span class="n">find_const</span>
<span class="n">mtd</span> <span class="o">=</span> <span class="n">find_method</span><span class="p">(</span><span class="n">const</span><span class="p">)</span>
<span class="k">unless</span> <span class="n">const</span> <span class="o">&&</span> <span class="n">mtd</span> <span class="o">&&</span> <span class="n">mtd</span><span class="p">.</span><span class="nf">arity</span> <span class="o">==</span> <span class="mi">2</span>
<span class="k">raise</span> <span class="s2">"Could not find class or method when patching"</span><span class="p">\</span>
<span class="s2">"ActionView's date_select helper. Please investigate."</span>
<span class="k">end</span>
<span class="k">unless</span> <span class="n">rails_version_ok?</span>
<span class="nb">puts</span> <span class="s2">"WARNING: It looks like Rails has been upgraded since"</span><span class="p">\</span>
<span class="s2">"ActionView's date_selet helper was monkeypatched in "</span><span class="p">\</span>
<span class="s2">"</span><span class="si">#{</span><span class="kp">__FILE__</span><span class="si">}</span><span class="s2">. Please reevaluate the patch."</span>
<span class="k">end</span>
<span class="n">const</span><span class="p">.</span><span class="nf">prepend</span><span class="p">(</span><span class="no">InstanceMethods</span><span class="p">)</span>
<span class="k">end</span>
<span class="kp">private</span>
<span class="k">def</span> <span class="nf">find_const</span>
<span class="no">Kernel</span><span class="p">.</span><span class="nf">const_get</span><span class="p">(</span><span class="s1">'ActionView::Helpers::DateTimeSelector'</span><span class="p">)</span>
<span class="k">rescue</span> <span class="no">NameError</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">find_method</span><span class="p">(</span><span class="n">const</span><span class="p">)</span>
<span class="k">return</span> <span class="k">unless</span> <span class="n">const</span>
<span class="n">const</span><span class="p">.</span><span class="nf">instance_method</span><span class="p">(</span><span class="ss">:build_hidden</span><span class="p">)</span>
<span class="k">rescue</span> <span class="no">NameError</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">rails_version_ok?</span>
<span class="no">Rails</span><span class="o">::</span><span class="no">VERSION</span><span class="o">::</span><span class="no">MAJOR</span> <span class="o">==</span> <span class="mi">6</span> <span class="o">&&</span> <span class="no">Rails</span><span class="o">::</span><span class="no">VERSION</span><span class="o">::</span><span class="no">MINOR</span> <span class="o">==</span> <span class="mi">1</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">module</span> <span class="nn">InstanceMethods</span>
<span class="k">def</span> <span class="nf">build_hidden</span><span class="p">(</span><span class="n">type</span><span class="p">,</span> <span class="n">value</span><span class="p">)</span>
<span class="s1">''</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="no">RenderDiscardedMonkeypatch</span><span class="p">.</span><span class="nf">apply_patch</span>
</code></pre></div></div>
<h3 id="give-yourself-escape-hatches">Give Yourself Escape Hatches</h3>
<p>When possible, I like to make my monkeypatch’s functionality opt-in. That’s only really an option if you have control over where the patched code is invoked. In the case of our Rails patch, it’s doable via the <code class="highlighter-rouge">@options</code> hash in <code class="highlighter-rouge">DateTimeSelector</code>:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">module</span> <span class="nn">RenderDiscardedMonkeypatch</span>
<span class="k">class</span> <span class="o"><<</span> <span class="nb">self</span>
<span class="k">def</span> <span class="nf">apply_patch</span>
<span class="n">const</span> <span class="o">=</span> <span class="n">find_const</span>
<span class="n">mtd</span> <span class="o">=</span> <span class="n">find_method</span><span class="p">(</span><span class="n">const</span><span class="p">)</span>
<span class="k">unless</span> <span class="n">const</span> <span class="o">&&</span> <span class="n">mtd</span> <span class="o">&&</span> <span class="n">mtd</span><span class="p">.</span><span class="nf">arity</span> <span class="o">==</span> <span class="mi">2</span>
<span class="k">raise</span> <span class="s2">"Could not find class or method when patching"</span><span class="p">\</span>
<span class="s2">"ActionView's date_select helper. Please investigate."</span>
<span class="k">end</span>
<span class="k">unless</span> <span class="n">rails_version_ok?</span>
<span class="nb">puts</span> <span class="s2">"WARNING: It looks like Rails has been upgraded since"</span><span class="p">\</span>
<span class="s2">"ActionView's date_selet helper was monkeypatched in "</span><span class="p">\</span>
<span class="s2">"</span><span class="si">#{</span><span class="kp">__FILE__</span><span class="si">}</span><span class="s2">. Please reevaluate the patch."</span>
<span class="k">end</span>
<span class="n">const</span><span class="p">.</span><span class="nf">prepend</span><span class="p">(</span><span class="no">InstanceMethods</span><span class="p">)</span>
<span class="k">end</span>
<span class="kp">private</span>
<span class="k">def</span> <span class="nf">find_const</span>
<span class="no">Kernel</span><span class="p">.</span><span class="nf">const_get</span><span class="p">(</span><span class="s1">'ActionView::Helpers::DateTimeSelector'</span><span class="p">)</span>
<span class="k">rescue</span> <span class="no">NameError</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">find_method</span><span class="p">(</span><span class="n">const</span><span class="p">)</span>
<span class="k">return</span> <span class="k">unless</span> <span class="n">const</span>
<span class="n">const</span><span class="p">.</span><span class="nf">instance_method</span><span class="p">(</span><span class="ss">:build_hidden</span><span class="p">)</span>
<span class="k">rescue</span> <span class="no">NameError</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">rails_version_ok?</span>
<span class="no">Rails</span><span class="o">::</span><span class="no">VERSION</span><span class="o">::</span><span class="no">MAJOR</span> <span class="o">==</span> <span class="mi">6</span> <span class="o">&&</span> <span class="no">Rails</span><span class="o">::</span><span class="no">VERSION</span><span class="o">::</span><span class="no">MINOR</span> <span class="o">==</span> <span class="mi">1</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">module</span> <span class="nn">InstanceMethods</span>
<span class="k">def</span> <span class="nf">build_hidden</span><span class="p">(</span><span class="n">type</span><span class="p">,</span> <span class="n">value</span><span class="p">)</span>
<span class="k">if</span> <span class="vi">@options</span><span class="p">.</span><span class="nf">fetch</span><span class="p">(</span><span class="ss">:render_discarded</span><span class="p">,</span> <span class="kp">true</span><span class="p">)</span>
<span class="k">super</span>
<span class="k">else</span>
<span class="s1">''</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="no">RenderDiscardedMonkeypatch</span><span class="p">.</span><span class="nf">apply_patch</span>
</code></pre></div></div>
<p>Nice! Now callers can opt-in by calling the <code class="highlighter-rouge">date_select</code> helper with the new option. No other codepaths are affected:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">date_select</span><span class="p">(</span><span class="vi">@user</span><span class="p">,</span> <span class="ss">:date_of_birth</span><span class="p">,</span> <span class="p">{</span>
<span class="ss">order: </span><span class="p">[</span><span class="ss">:month</span><span class="p">,</span> <span class="ss">:day</span><span class="p">],</span>
<span class="ss">render_discarded: </span><span class="kp">false</span>
<span class="p">})</span>
</code></pre></div></div>
<h3 id="over-communicate">Over-Communicate</h3>
<p>The last piece of advice I have for you is perhaps the most important - communicating what your patch does and when it’s time to re-examine it. Your goal with monkeypatches should always be to eventually remove the patch altogether. To that end, a responsible monkeypatch includes comments that:</p>
<ol>
<li>Describe what the patch does.</li>
<li>Explain why the patch is necessary.</li>
<li>Outline the assumptions the patch makes.</li>
<li>Specify a date in the future when your team should reconsider alternative solutions, like pulling in an updated gem.</li>
<li>Include links to relevant pull requests, blog posts, StackOverflow answers, etc.</li>
</ol>
<p>You might even print a warning or fail a test on a predetermined date to urge the team to reconfirm the patch’s assumptions and consider whether or not it’s still necessary.</p>
<p>Here’s the final version of our Rails <code class="highlighter-rouge">date_select</code> patch, complete with comments and a date check:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1"># ActionView's date_select helper provides the option to "discard" certain</span>
<span class="c1"># fields. Discarded fields are (confusingly) still rendered to the page</span>
<span class="c1"># using hidden inputs, i.e. <input type="hidden" />. This patch adds an</span>
<span class="c1"># additional option to the date_select helper that allows the caller to</span>
<span class="c1"># skip rendering the chosen fields altogether. For example, to render all</span>
<span class="c1"># but the year field, you might have this in one of your views:</span>
<span class="c1">#</span>
<span class="c1"># date_select(:date_of_birth, order: [:month, :day])</span>
<span class="c1">#</span>
<span class="c1"># or, equivalently:</span>
<span class="c1">#</span>
<span class="c1"># date_select(:date_of_birth, discard_year: true)</span>
<span class="c1">#</span>
<span class="c1"># To avoid rendering the year field altogether, set :render_discarded to</span>
<span class="c1"># false:</span>
<span class="c1">#</span>
<span class="c1"># date_select(:date_of_birth, discard_year: true, render_discarded: false)</span>
<span class="c1">#</span>
<span class="c1"># This patch assumes the #build_hidden method exists on</span>
<span class="c1"># ActionView::Helpers::DateTimeSelector and accepts two arguments.</span>
<span class="c1">#</span>
<span class="k">module</span> <span class="nn">RenderDiscardedMonkeypatch</span>
<span class="k">class</span> <span class="o"><<</span> <span class="nb">self</span>
<span class="no">EXPIRATION_DATE</span> <span class="o">=</span> <span class="no">Date</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="mi">2021</span><span class="p">,</span> <span class="mi">8</span><span class="p">,</span> <span class="mi">15</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">apply_patch</span>
<span class="k">if</span> <span class="no">Date</span><span class="p">.</span><span class="nf">today</span> <span class="o">></span> <span class="no">EXPIRATION_DATE</span>
<span class="nb">puts</span> <span class="s2">"WARNING: Please re-evaluate whether or not the ActionView "</span><span class="p">\</span>
<span class="s2">"date_select patch present in </span><span class="si">#{</span><span class="kp">__FILE__</span><span class="si">}</span><span class="s2"> is still necessary."</span>
<span class="k">end</span>
<span class="n">const</span> <span class="o">=</span> <span class="n">find_const</span>
<span class="n">mtd</span> <span class="o">=</span> <span class="n">find_method</span><span class="p">(</span><span class="n">const</span><span class="p">)</span>
<span class="c1"># make sure the class we want to patch exists;</span>
<span class="c1"># make sure the #build_hidden method exists and accepts exactly</span>
<span class="c1"># two arguments</span>
<span class="k">unless</span> <span class="n">const</span> <span class="o">&&</span> <span class="n">mtd</span> <span class="o">&&</span> <span class="n">mtd</span><span class="p">.</span><span class="nf">arity</span> <span class="o">==</span> <span class="mi">2</span>
<span class="k">raise</span> <span class="s2">"Could not find class or method when patching "</span><span class="p">\</span>
<span class="s2">"ActionView's date_select helper. Please investigate."</span>
<span class="k">end</span>
<span class="c1"># if rails has been upgraded, make sure this patch is still</span>
<span class="c1"># necessary</span>
<span class="k">unless</span> <span class="n">rails_version_ok?</span>
<span class="nb">puts</span> <span class="s2">"WARNING: It looks like Rails has been upgraded since "</span><span class="p">\</span>
<span class="s2">"ActionView's date_select helper was monkeypatched in "</span><span class="p">\</span>
<span class="s2">"</span><span class="si">#{</span><span class="kp">__FILE__</span><span class="si">}</span><span class="s2">. Please re-evaluate the patch."</span>
<span class="k">end</span>
<span class="c1"># actually apply the patch</span>
<span class="n">const</span><span class="p">.</span><span class="nf">prepend</span><span class="p">(</span><span class="no">InstanceMethods</span><span class="p">)</span>
<span class="k">end</span>
<span class="kp">private</span>
<span class="k">def</span> <span class="nf">find_const</span>
<span class="no">Kernel</span><span class="p">.</span><span class="nf">const_get</span><span class="p">(</span><span class="s1">'ActionView::Helpers::DateTimeSelector'</span><span class="p">)</span>
<span class="k">rescue</span> <span class="no">NameError</span>
<span class="c1"># return nil if the constant doesn't exist</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">find_method</span><span class="p">(</span><span class="n">const</span><span class="p">)</span>
<span class="k">return</span> <span class="k">unless</span> <span class="n">const</span>
<span class="n">const</span><span class="p">.</span><span class="nf">instance_method</span><span class="p">(</span><span class="ss">:build_hidden</span><span class="p">)</span>
<span class="k">rescue</span> <span class="no">NameError</span>
<span class="c1"># return nil if the method doesn't exist</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">rails_version_ok?</span>
<span class="no">Rails</span><span class="o">::</span><span class="no">VERSION</span><span class="o">::</span><span class="no">MAJOR</span> <span class="o">==</span> <span class="mi">6</span> <span class="o">&&</span> <span class="no">Rails</span><span class="o">::</span><span class="no">VERSION</span><span class="o">::</span><span class="no">MINOR</span> <span class="o">==</span> <span class="mi">1</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">module</span> <span class="nn">InstanceMethods</span>
<span class="c1"># :render_discarded is an additional option you can pass to the</span>
<span class="c1"># date_select helper in your views. Use it to avoid rendering</span>
<span class="c1"># "discarded" fields, i.e. fields marked as discarded or simply</span>
<span class="c1"># not included in date_select's :order array. For example,</span>
<span class="c1"># specifying order: [:day, :month] will cause the helper to</span>
<span class="c1"># "discard" the :year field. Discarding a field renders it as a</span>
<span class="c1"># hidden input. Set :render_discarded to false to avoid rendering</span>
<span class="c1"># it altogether.</span>
<span class="k">def</span> <span class="nf">build_hidden</span><span class="p">(</span><span class="n">type</span><span class="p">,</span> <span class="n">value</span><span class="p">)</span>
<span class="k">if</span> <span class="vi">@options</span><span class="p">.</span><span class="nf">fetch</span><span class="p">(</span><span class="ss">:render_discarded</span><span class="p">,</span> <span class="kp">true</span><span class="p">)</span>
<span class="k">super</span>
<span class="k">else</span>
<span class="s1">''</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="no">RenderDiscardedMonkeypatch</span><span class="p">.</span><span class="nf">apply_patch</span>
</code></pre></div></div>
<h2 id="conclusion">Conclusion</h2>
<p>I totally get that some of the suggestions I’ve outlined above might seem like overkill. Our Rails patch contains way more defensive verification code than actual patch code!</p>
<p>Think of all that extra code as a sheath for your broadsword. It’s a lot easier to avoid getting cut if it’s enveloped in a layer of protection.</p>
<p><img src="https://media.giphy.com/media/KFPRdKqy96oaI4HWEn/giphy-downsized.gif" alt="Sword guitar" /></p>
<p>What really matters, though, is that I feel confident deploying responsible monkeypatches into production. Irresponsible ones are just time bombs waiting to cost you or your company time, money, and developer health.</p>Cameron DutroThis is a post I wrote for the AppSignal blog about how to monkeypatch without making a mess :)Installing 220 Gems in 40 Seconds2021-08-14T21:30:00-07:002021-08-14T21:30:00-07:00/2021/08/14/installing-220-gems-in-40-seconds<p>I gave a <a href="https://youtu.be/YMoa5JpjEtM?t=1315">lightning talk</a> at RubyConf in 2017 about a gem I was working on at the time called <a href="https://github.com/camertron/prebundler">prebundler</a>. I recently spent a bunch of time improving it, so I thought I’d write up a post.</p>
<hr />
<p>Back in 2017 I worked for Lumos Labs, the creators of Lumosity. We had recently transitioned from a custom Capistrano setup to Docker and Kubernetes for deploying our large Rails monolith. While we were pretty darn happy with it, the slowness of our Docker builds eventually became a major pain point. It would sometimes take over 30 minutes for CI to run, and while we used CI to run tests and a few other things as well, by far the most time-consuming part was building the Docker image. I decided to investigate.</p>
<p>Right off the bat, I identified two major sources of slowness:</p>
<ol>
<li>Building static assets.</li>
<li>Running <code class="highlighter-rouge">bundle install</code>.</li>
</ol>
<p>I’m going to focus on the second bullet point in this blog post.</p>
<h2 id="docker-vs-capistrano">Docker vs Capistrano</h2>
<p>In our Capistrano setup, installing gem dependencies was really fast, so we didn’t have to worry about it. That’s because Capistrano works by running commands on a remote machine over SSH. Every time you deploy, Capistrano runs <code class="highlighter-rouge">bundle install</code> for you. Most of the gems you need are already present on the machine from the previous deploy, so Bundler only has to fetch and install any new or upgraded ones.</p>
<p>In contrast, Docker images are built from scratch every time. In other words, Bundler has to fetch and install every gem every time you build a new container image. Depending on how many gems your app needs to run, this can take really long time. Considering that dependencies can themselves depend on other gems and so on, your app probably depends on a lot more than just what’s listed in your Gemfile. Furthermore, many popular gems contain native extensions - usually written in C - that need to be compiled during the installation process. Compilation time can add a significant amount of additional overhead.</p>
<h2 id="time-lost">Time Lost</h2>
<p>In April of 2017, the repo for lumosity.com (lumos_rails) contained 445 gem dependencies. That included both entries in the Gemfile and so-called transient dependencies, i.e. gems depended upon by other gems. It took our Travis CI builder job over six minutes to install them all inside the container image.</p>
<p>Six minutes might not <em>seem</em> like a lot of time, but compounded over a month, a week, or even a single day, those 6 minutes add up quickly. Our team ran about 30 builds per day which translated into spending 3 hours a day, 15 hours a week, 60 hours a month just waiting for <code class="highlighter-rouge">bundle install</code>.</p>
<h2 id="bundler-improvements">Bundler Improvements</h2>
<p>In the not so distant past, Bundler introduced some nice new features for speeding up installation. For example there’s the handy <code class="highlighter-rouge">--jobs n</code> flag, which will install gems in parallel using <code class="highlighter-rouge">n</code> threads. However only the I/O bound parts of installation are affected, since Ruby’s global VM lock (GVL) prevents multiple Ruby execution paths from running concurrently. Moreover, building native extensions is still a problem.</p>
<p>What more can we do to speed things up?</p>
<h2 id="enter-prebundler">Enter Prebundler</h2>
<p>Back in 2017 I started thinking about ways to address the native extensions problem. It seemed like a waste of time to recompile extensions for every Docker build, especially considering the resulting .so files can be cached. Caching would have to be done outside of Docker though, since Docker only caches entire layers (i.e. the full result of a <code class="highlighter-rouge">bundle install</code>) and not individual gems.</p>
<p>After some noodling, I came up with an idea: why don’t we stick all the gem’s files (including compiled native extensions, etc) into a TAR file and store it in some object storage system like S3? Installation would then be as simple as downloading the TAR file and expanding it onto the hard disk somewhere. All these operations are I/O bound, meaning the installation process can be highly parallelized.</p>
<p>I coded up a solution and integrated it into lumos_rails. Our team saw <code class="highlighter-rouge">bundle install</code> time decrease from 6 minutes 7 seconds to 43 seconds - that’s an 88% speed up!</p>
<h2 id="prebundling-discourse">Prebundling Discourse</h2>
<p>To demonstrate the sort of speed-ups prebundler can enable, let’s take a look at the Gemfile from Discourse, a large, open-source Rails app.</p>
<p>By the way, all the code for this example can be found in the <a href="https://github.com/camertron/prebundler_bench">prebundler_bench repo</a>.</p>
<p>At the time of this writing, Discourse has 220 direct and transient dependencies. Here’s a Dockerfile that installs all the gems from Discourse’s Gemfile without prebundler:</p>
<div class="language-dockerfile highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">FROM</span><span class="s"> ruby:2.7</span>
<span class="k">WORKDIR</span><span class="s"> /usr/src/app</span>
<span class="k">COPY</span><span class="s"> Gemfile* ./</span>
<span class="k">RUN </span>bundle <span class="nb">install</span> <span class="nt">--jobs</span> <span class="si">$(</span><span class="nb">nproc</span><span class="si">)</span>
</code></pre></div></div>
<p>Now, here’s a Dockerfile that uses prebundler instead:</p>
<div class="language-dockerfile highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">FROM</span><span class="s"> ruby:2.7</span>
<span class="k">ARG</span><span class="s"> PREBUNDLER_ACCESS_KEY</span>
<span class="k">ARG</span><span class="s"> PREBUNDLER_ACCESS_SECRET</span>
<span class="k">WORKDIR</span><span class="s"> /usr/src/app</span>
<span class="k">RUN </span>gem <span class="nb">install </span>prebundler
<span class="k">COPY</span><span class="s"> Gemfile* ./</span>
<span class="k">COPY</span><span class="s"> .prebundle_config ./</span>
<span class="k">RUN </span>prebundle <span class="nb">install</span> <span class="nt">--jobs</span> <span class="si">$(</span><span class="nb">nproc</span><span class="si">)</span>
</code></pre></div></div>
<p>Finally, here’s the contents of the .prebundle_config file:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="no">Prebundler</span><span class="p">.</span><span class="nf">configure</span> <span class="k">do</span> <span class="o">|</span><span class="n">config</span><span class="o">|</span>
<span class="n">config</span><span class="p">.</span><span class="nf">storage_backend</span> <span class="o">=</span> <span class="no">Prebundler</span><span class="o">::</span><span class="no">S3Backend</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span>
<span class="ss">client: </span><span class="no">Aws</span><span class="o">::</span><span class="no">S3</span><span class="o">::</span><span class="no">Client</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span>
<span class="ss">region: </span><span class="s1">'default'</span><span class="p">,</span>
<span class="ss">credentials: </span><span class="no">Aws</span><span class="o">::</span><span class="no">Credentials</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span>
<span class="no">ENV</span><span class="p">[</span><span class="s1">'PREBUNDLER_ACCESS_KEY'</span><span class="p">],</span>
<span class="no">ENV</span><span class="p">[</span><span class="s1">'PREBUNDLER_ACCESS_SECRET'</span><span class="p">]</span>
<span class="p">),</span>
<span class="ss">endpoint: </span><span class="s1">'https://us-east-1.linodeobjects.com'</span><span class="p">,</span>
<span class="ss">http_continue_timeout: </span><span class="mi">0</span>
<span class="p">),</span>
<span class="ss">bucket: </span><span class="s1">'prebundler'</span><span class="p">,</span>
<span class="ss">region: </span><span class="s1">'us-east-1'</span>
<span class="p">)</span>
<span class="k">end</span>
</code></pre></div></div>
<p>We can now build the images like so:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c"># regular installation using bundler</span>
docker build <span class="se">\</span>
<span class="nt">--no-cache</span> <span class="se">\</span>
<span class="nt">-f</span> Dockerfile <span class="se">\</span>
<span class="nt">-t</span> prebundler_test:latest <span class="nb">.</span>
<span class="c"># faster installation using prebundler</span>
docker build <span class="se">\</span>
<span class="nt">--no-cache</span> <span class="se">\</span>
<span class="nt">--build-arg</span> <span class="nv">PREBUNDLER_ACCESS_KEY</span><span class="o">=</span><span class="k">${</span><span class="nv">PREBUNDLER_ACCESS_KEY</span><span class="k">}</span> <span class="se">\</span>
<span class="nt">--build-arg</span> <span class="nv">PREBUNDLER_ACCESS_SECRET</span><span class="o">=</span><span class="k">${</span><span class="nv">PREBUNDLER_ACCESS_SECRET</span><span class="k">}</span> <span class="se">\</span>
<span class="nt">-f</span> Dockerfile-pre <span class="se">\</span>
<span class="nt">-t</span> prebundler_test:pre-latest <span class="nb">.</span>
</code></pre></div></div>
<p><strong>NOTE</strong>: don’t forget to populate <code class="highlighter-rouge">PREBUNDLER_ACCESS_KEY</code> and <code class="highlighter-rouge">PREBUNDLER_ACCESS_SECRET</code> with the contents of your S3 credentials when you run the script.</p>
<h2 id="the-results">The Results</h2>
<p>Building both images on my MacBook Pro produces the following output. The <code class="highlighter-rouge">docker build</code> command now helpfully times every operation, so we can see how long <code class="highlighter-rouge">bundle install</code> took vs <code class="highlighter-rouge">prebundle install</code>.</p>
<p>Here’s the output for regular installation using bundler:</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>[+] Building 185.7s (9/9) FINISHED
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 129B 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [internal] load metadata for ruby:2.7 1.2s
=> [internal] load build context 0.0s
=> => transferring context: 14.29kB 0.0s
=> [1/4] FROM docker.io/library/ruby:2.7 0.0s
=> CACHED [2/4] WORKDIR /usr/src/app 0.0s
=> [3/4] COPY Gemfile* ./ 0.0s
=> [4/4] RUN bundle install --jobs $(nproc) 179.2s
=> exporting to image 5.2s
=> => exporting layers 5.2s
=> => writing image 0.0s
=> => naming to prebundler_test:latest 0.0s
</code></pre></div></div>
<p>And here’s the output for installation using prebundler:</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>[+] Building 48.3s (11/11) FINISHED
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 191B 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [internal] load metadata for ruby:2.7 0.5s
=> [internal] load build context 0.0s
=> => transferring context: 533B 0.0s
=> [1/6] FROM docker.io/library/ruby:2.7 0.0s
=> CACHED [2/6] WORKDIR /usr/src/app 0.0s
=> [3/6] RUN gem install prebundler 3.3s
=> [4/6] COPY Gemfile* ./ 0.0s
=> [5/6] COPY .prebundle_config ./ 0.0s
=> [6/6] RUN prebundle install --jobs $(nproc) 39.9s
=> exporting to image 4.4s
=> => exporting layers 4.4s
=> => writing image 0.0s
=> => naming to prebundler_test:pre-latest 0.0s
</code></pre></div></div>
<p>As you can see, bundler took 179.2 seconds while prebundler took only 39.9. That’s a 78% speed increase! Note that because we had to install the prebundler gem in a separate build step, the overall speed increase is actually closer to 74%. Pretty good, I’d say!</p>
<h2 id="setting-up-prebundler-in-your-app">Setting up Prebundler in your App</h2>
<p>As you may have noticed, I used Linode’s object storage offering in the Discourse example above. As it happens, many of the object storage systems available from the various cloud providers are S3-compatible, requiring no changes to the example prebundler config I showed earlier.</p>
<p>Let’s walk through the necessary steps one by one.</p>
<ol>
<li>
<p>First, you’ll need a Linode account with object storage turned on. Linode charges a flat monthly fee under a certain storage limit. At the time of this writing, it costs $5/mo for 250gb of storage.</p>
</li>
<li>
<p>Create a bucket where all your TAR files will be stored. I called mine “prebundler.”</p>
</li>
</ol>
<p><img src="/images/220_gems/create_bucket.png" alt="" /></p>
<ol>
<li>Create the access key prebundler will use to store and retrieve TAR files. It’s a good idea to give it only the access it needs, i.e. read and write access to the prebundler bucket only.</li>
</ol>
<p><img src="/images/220_gems/create_access_key.png" alt="" /></p>
<ol>
<li>
<p>Make sure to store the access key and secret in a secure location like a password manager or other credentials store.</p>
</li>
<li>
<p>Install prebundler by running <code class="highlighter-rouge">gem install prebundler</code>. If you’re using a Ruby version manager like rbenv or asdf, don’t forget to run <code class="highlighter-rouge">rbenv rehash</code> or <code class="highlighter-rouge">asdf reshim ruby</code> to make the <code class="highlighter-rouge">prebundle</code> command available in your <code class="highlighter-rouge">PATH</code>.</p>
</li>
<li>
<p>Create a .prebundle_config file in the root of your project and copy/paste in the configuration given above. Make sure to change the endpoint and region if you created your bucket in a region other than us-east-1.</p>
</li>
<li>
<p>Add two <code class="highlighter-rouge">ARG</code>s to your Dockerfile, one each for the access key and secret key. Add a <code class="highlighter-rouge">COPY</code> directive to copy in the .prebundle_config, and finally <code class="highlighter-rouge">RUN prebundle install</code> instead of <code class="highlighter-rouge">bundle install</code></p>
</li>
<li>
<p>Last but not least, build your Docker image. Don’t forget to pass two <code class="highlighter-rouge">--build-arg</code> arguments containing the access key and secret.</p>
</li>
</ol>
<p>After building the image, you should see a bunch of TAR files in your object storage bucket:</p>
<p><img src="/images/220_gems/bucket.png" alt="" /></p>
<h2 id="conclusion">Conclusion</h2>
<p>If installing gem dependencies is slowing down your CI builds and blocking releases, consider giving prebundler a try. If you run into trouble, please don’t hesitate to file an issue or, better yet, submit a pull request :)</p>Cameron DutroI gave a lightning talk at RubyConf in 2017 about a gem I was working on at the time called prebundler. I recently spent a bunch of time improving it, so I thought I’d write up a post.Why Ruby’s Enumerable Module is Awesome2021-07-10T10:00:00-07:002021-07-10T10:00:00-07:00/2021/07/10/why-rubys-enumerable-module-is-awesome<p>This post was originally written in 2014 at the beginning of my tenure at Lumos Labs. At the time, I was a member of the Learning Team, an “extracurricular” group that met bi-weekly to discuss cool things we were learning about technology. We organized tech meetups in our office space, streamed live Google IO talks over the projector during lunch, and sent out a digest email to our colleagues every two weeks with links to various learning resources. I ended up writing a few longer-form articles for these email blasts. What follows is an embellished version of one of those articles.</p>
<hr />
<p>You’re probably familiar with the concept of “iteration” in computer programming. It’s the idea of examining - or iterating over - each of the things in a collection.</p>
<p>Perhaps the most obvious thing you can iterate over is an array. The elements of an array are accessed by their index, so iterating is pretty straightforward. Here’s an example in pseudocode:</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>array = [5, 3, 8]
for i = 0 to array.length
do something with array[i]
end
</code></pre></div></div>
<p>This code iterates over each item in the array. Inside the body of the loop, elements are accessed individually using the <code class="highlighter-rouge">[]</code> syntax.</p>
<p>We can do the same thing in Ruby using the <code class="highlighter-rouge">for</code> keyword:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">array</span> <span class="o">=</span> <span class="p">[</span><span class="mi">5</span><span class="p">,</span> <span class="mi">3</span><span class="p">,</span> <span class="mi">8</span><span class="p">]</span>
<span class="k">for</span> <span class="n">i</span> <span class="k">in</span> <span class="mi">0</span><span class="o">...</span><span class="mi">3</span>
<span class="c1"># do something with array[i]</span>
<span class="k">end</span>
</code></pre></div></div>
<h3 id="each"><code class="highlighter-rouge">#each</code></h3>
<p>The truth is though, in 11 years writing Ruby code, I’ve never, not even once, seen anyone use a <code class="highlighter-rouge">for</code> loop. Instead, Ruby programmers reach for the <code class="highlighter-rouge">#each</code> method. <code class="highlighter-rouge">#each</code> yields each element to the given block. Here’s a quick example that prints out each of the numbers in the array:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="p">[</span><span class="mi">5</span><span class="p">,</span> <span class="mi">3</span><span class="p">,</span> <span class="mi">8</span><span class="p">].</span><span class="nf">each</span> <span class="k">do</span> <span class="o">|</span><span class="n">number</span><span class="o">|</span>
<span class="nb">puts</span> <span class="n">number</span>
<span class="k">end</span>
</code></pre></div></div>
<p>Not only is the code easier to read with <code class="highlighter-rouge">#each</code>, it’s more obvious what it does. <code class="highlighter-rouge">#each</code> abstracts away the details of the iteration logic and lets the programmer focus on their goal: handling one element at a time.</p>
<h3 id="sum-of-integers">Sum of Integers</h3>
<p>Let’s get a little more adventurous and use Ruby to compute the sum of all the elements in our array.</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">sum</span> <span class="o">=</span> <span class="mi">0</span>
<span class="p">[</span><span class="mi">5</span><span class="p">,</span> <span class="mi">3</span><span class="p">,</span> <span class="mi">8</span><span class="p">].</span><span class="nf">each</span> <span class="k">do</span> <span class="o">|</span><span class="n">number</span><span class="o">|</span>
<span class="n">sum</span> <span class="o">+=</span> <span class="n">number</span>
<span class="k">end</span>
</code></pre></div></div>
<p>When <code class="highlighter-rouge">#each</code> returns, <code class="highlighter-rouge">sum</code> will contain 16.</p>
<h3 id="the-magic-of-inject">The Magic of <code class="highlighter-rouge">#inject</code></h3>
<p>It would be great if we could get rid of that extra local variable, <code class="highlighter-rouge">sum</code>. Fortunately, Ruby’s <code class="highlighter-rouge">#inject</code> method can help. Here’s how we might use it to sum up the elements in our array:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="p">[</span><span class="mi">5</span><span class="p">,</span> <span class="mi">3</span><span class="p">,</span> <span class="mi">8</span><span class="p">].</span><span class="nf">inject</span><span class="p">(</span><span class="mi">0</span><span class="p">)</span> <span class="k">do</span> <span class="o">|</span><span class="n">sum</span><span class="p">,</span> <span class="n">number</span><span class="o">|</span>
<span class="n">sum</span> <span class="o">+</span> <span class="n">number</span>
<span class="k">end</span>
</code></pre></div></div>
<p>Pretty cool, eh? The <code class="highlighter-rouge">#inject</code> method calls the block for each number, passing the <em>previous</em> result as the first argument and the next element from the array as the second argument (the previous result is simply the value returned by the block during the previous iteration).</p>
<p>I can hear some of you saying, “Whoa, slow down. What just happened?!” Ok, let’s break it down step-by-step.</p>
<ol>
<li>First iteration (<code class="highlighter-rouge">sum</code> is set to the initial value passed to <code class="highlighter-rouge">#inject</code>, which is <code class="highlighter-rouge">0</code>)
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="p">[</span><span class="mi">5</span><span class="p">,</span> <span class="mi">3</span><span class="p">,</span> <span class="mi">8</span><span class="p">].</span><span class="nf">inject</span><span class="p">(</span><span class="mi">0</span><span class="p">)</span> <span class="k">do</span> <span class="o">|</span><span class="n">sum</span><span class="p">,</span> <span class="n">number</span><span class="o">|</span>
<span class="c1"># sum = 0 (initial value passed to #inject above)</span>
<span class="c1"># number = 5 (first element of array)</span>
<span class="c1"># 0 + 5 = 5</span>
<span class="n">sum</span> <span class="o">+</span> <span class="n">number</span>
<span class="c1"># 5 becomes the return value of the block</span>
<span class="k">end</span>
</code></pre></div> </div>
</li>
<li>Second iteration
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="p">[</span><span class="mi">5</span><span class="p">,</span> <span class="mi">3</span><span class="p">,</span> <span class="mi">8</span><span class="p">].</span><span class="nf">inject</span><span class="p">(</span><span class="mi">0</span><span class="p">)</span> <span class="k">do</span> <span class="o">|</span><span class="n">sum</span><span class="p">,</span> <span class="n">number</span><span class="o">|</span>
<span class="c1"># sum = 5 (from previous iteration)</span>
<span class="c1"># number = 3 (second element of array)</span>
<span class="c1"># 5 + 3 = 8</span>
<span class="n">sum</span> <span class="o">+</span> <span class="n">number</span>
<span class="c1"># 8 becomes the return value of the block</span>
<span class="k">end</span>
</code></pre></div> </div>
</li>
<li>Third iteration
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="p">[</span><span class="mi">5</span><span class="p">,</span> <span class="mi">3</span><span class="p">,</span> <span class="mi">8</span><span class="p">].</span><span class="nf">inject</span><span class="p">(</span><span class="mi">0</span><span class="p">)</span> <span class="k">do</span> <span class="o">|</span><span class="n">sum</span><span class="p">,</span> <span class="n">number</span><span class="o">|</span>
<span class="c1"># sum = 8 (from previous iteration)</span>
<span class="c1"># number = 8 (third element of array)</span>
<span class="c1"># 8 + 8 = 16</span>
<span class="n">sum</span> <span class="o">+</span> <span class="n">number</span>
<span class="c1"># 16 becomes the return value of the block</span>
<span class="k">end</span>
</code></pre></div> </div>
</li>
</ol>
<p>Since there are only three elements in the array, iteration stops and the final sum of 16 is returned.</p>
<h3 id="even-more-magic">Even More Magic</h3>
<p>As it happens, there’s an even more succinct way to do this. <code class="highlighter-rouge">#inject</code> supports passing a symbol as the first argument. The symbol must be the name of a method that can be called on the elements of the array. Since we’re adding in this case, we can pass the <code class="highlighter-rouge">:+</code> symbol, which represents the <code class="highlighter-rouge">#+</code> method on <code class="highlighter-rouge">Integer</code>:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="p">[</span><span class="mi">5</span><span class="p">,</span> <span class="mi">3</span><span class="p">,</span> <span class="mi">8</span><span class="p">].</span><span class="nf">inject</span><span class="p">(:</span><span class="o">+</span><span class="p">)</span>
</code></pre></div></div>
<p>No block necessary! <code class="highlighter-rouge">#inject</code> automatically keeps track of the previous value and adds it to the next element on each iteration. As above, this code produces the value 16.</p>
<h3 id="the-enumerable-module">The <code class="highlighter-rouge">Enumerable</code> Module</h3>
<p>The <code class="highlighter-rouge">#inject</code> method is only one of the <strong>many</strong> methods provided by Ruby’s <code class="highlighter-rouge">Enumerable</code> module. <code class="highlighter-rouge">Enumerable</code> is included in <code class="highlighter-rouge">Array</code>, <code class="highlighter-rouge">Hash</code>, and other core classes, providing a uniform way to iterate over all the items in a collection.</p>
<p>This is where things get really interesting - <code class="highlighter-rouge">Enumerable</code> has a <em>ton</em> of cool methods. Need to process a collection in a specific or special way? Chances are there’s an <code class="highlighter-rouge">Enumerable</code> method (or methods) for it.</p>
<p>Accordingly, let’s take a look at a couple of the other useful tools in the <code class="highlighter-rouge">Enumerable</code> toolkit.</p>
<h3 id="enumerablemap"><code class="highlighter-rouge">Enumerable#map</code></h3>
<p><code class="highlighter-rouge">#map</code> is probably the next most commonly used <code class="highlighter-rouge">Enumerable</code> method after <code class="highlighter-rouge">#each</code>. It collects the results of the block into an array and returns it. For example, here’s how we might multiply every element in our array by 2:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">result</span> <span class="o">=</span> <span class="p">[</span><span class="mi">5</span><span class="p">,</span> <span class="mi">3</span><span class="p">,</span> <span class="mi">8</span><span class="p">].</span><span class="nf">map</span> <span class="k">do</span> <span class="o">|</span><span class="n">number</span><span class="o">|</span>
<span class="n">number</span> <span class="o">*</span> <span class="mi">2</span>
<span class="k">end</span>
</code></pre></div></div>
<p>After running this code, <code class="highlighter-rouge">result</code> will contain <code class="highlighter-rouge">[10, 6, 16]</code>.</p>
<h3 id="enumerableeach_slice"><code class="highlighter-rouge">Enumerable#each_slice</code></h3>
<p>Another great example of <code class="highlighter-rouge">Enumerable</code>’s utility is <code class="highlighter-rouge">each_slice</code>, which yields sub arrays of the given length to the block. For example, the following code turns this flat array of ingredients into a hash:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">recipe</span> <span class="o">=</span> <span class="p">{}.</span><span class="nf">tap</span> <span class="k">do</span> <span class="o">|</span><span class="n">result</span><span class="o">|</span>
<span class="p">[</span><span class="ss">:eggs</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="ss">:carrots</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="ss">:bell_peppers</span><span class="p">,</span> <span class="mi">3</span><span class="p">].</span><span class="nf">each_slice</span><span class="p">(</span><span class="mi">2</span><span class="p">)</span> <span class="k">do</span> <span class="o">|</span><span class="n">food</span><span class="p">,</span> <span class="n">amount</span><span class="o">|</span>
<span class="n">result</span><span class="p">[</span><span class="n">food</span><span class="p">]</span> <span class="o">=</span> <span class="n">amount</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>
<p>The desired length of each slice is passed as the first argument to <code class="highlighter-rouge">#each_slice</code>, eg. <code class="highlighter-rouge">each_slice(2)</code> as above.</p>
<p>After running this code, <code class="highlighter-rouge">recipe</code> will contain <code class="highlighter-rouge">{ eggs: 2, carrots: 1, bell_peppers: 3 }</code>.</p>
<p>As an aside, notice that you can also assign the elements of the sub-array to individual block parameters, eg. <code class="highlighter-rouge">food</code> and <code class="highlighter-rouge">amount</code>. If only one parameter is specified, it will contain an array with two elements.</p>
<h3 id="but-wait-theres-more">But Wait, There’s More!</h3>
<p>Check out the plethora of other <code class="highlighter-rouge">Enumerable</code> methods in Ruby’s <a href="https://ruby-doc.org/core-3.0.1/Enumerable.html">official documentation</a>.</p>
<h3 id="custom-enumerators">Custom Enumerators</h3>
<p>We’ve seen a few examples of <code class="highlighter-rouge">Enumerable</code>’s awesomeness so far, but in my opinion its real power can only be truly experienced in combination with custom enumerators.</p>
<p>Let’s say you’re writing a client that communicates with a search API. The API returns search results in pages (i.e. batches) of 50.</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">class</span> <span class="nc">SearchClient</span>
<span class="k">def</span> <span class="nf">search_for</span><span class="p">(</span><span class="n">keywords</span><span class="p">,</span> <span class="ss">page: </span><span class="mi">1</span><span class="p">)</span>
<span class="n">response</span> <span class="o">=</span> <span class="n">http_get</span><span class="p">(</span><span class="s1">'/search'</span><span class="p">,</span> <span class="ss">keywords: </span><span class="n">keywords</span><span class="p">,</span> <span class="ss">page: </span><span class="n">page</span><span class="p">)</span>
<span class="no">JSON</span><span class="p">.</span><span class="nf">parse</span><span class="p">(</span><span class="n">response</span><span class="p">.</span><span class="nf">body</span><span class="p">)</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">http_get</span><span class="p">(</span><span class="n">path</span><span class="p">,</span> <span class="o">**</span><span class="n">params</span><span class="p">)</span>
<span class="o">...</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>
<p>To fetch all the search results, the caller makes multiple calls to the <code class="highlighter-rouge">#search_for</code> method.</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">client</span> <span class="o">=</span> <span class="no">SearchClient</span><span class="p">.</span><span class="nf">new</span>
<span class="n">page</span> <span class="o">=</span> <span class="mi">1</span>
<span class="kp">loop</span> <span class="k">do</span>
<span class="n">results</span> <span class="o">=</span> <span class="n">client</span><span class="p">.</span><span class="nf">search_for</span><span class="p">(</span><span class="s1">'avocado'</span><span class="p">,</span> <span class="ss">page: </span><span class="n">page</span><span class="p">)</span>
<span class="k">break</span> <span class="k">if</span> <span class="n">results</span><span class="p">.</span><span class="nf">empty?</span>
<span class="n">results</span><span class="p">.</span><span class="nf">each</span> <span class="k">do</span> <span class="o">|</span><span class="n">result</span><span class="o">|</span>
<span class="c1"># do something with search result</span>
<span class="k">end</span>
<span class="n">page</span> <span class="o">+=</span> <span class="mi">1</span>
<span class="k">end</span>
</code></pre></div></div>
<p>This approach works great, but forces the caller to understand how the API works. Specifically it requires the caller to know that results are paginated and that an empty result set indicates all results have been retrieved.</p>
<p>Let’s move the pagination logic into a separate class.</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">class</span> <span class="nc">SearchClient</span>
<span class="k">def</span> <span class="nf">search_for</span><span class="p">(</span><span class="n">keywords</span><span class="p">)</span>
<span class="no">SearchResultSet</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="nb">self</span><span class="p">,</span> <span class="n">keywords</span><span class="p">)</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">http_get</span><span class="p">(</span><span class="n">path</span><span class="p">,</span> <span class="o">**</span><span class="n">params</span><span class="p">)</span>
<span class="o">...</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">class</span> <span class="nc">SearchResultSet</span>
<span class="nb">attr_reader</span> <span class="ss">:client</span><span class="p">,</span> <span class="ss">:keywords</span>
<span class="k">def</span> <span class="nf">initialize</span><span class="p">(</span><span class="n">client</span><span class="p">,</span> <span class="n">keywords</span><span class="p">)</span>
<span class="vi">@client</span> <span class="o">=</span> <span class="n">client</span>
<span class="vi">@keywords</span> <span class="o">=</span> <span class="n">keywords</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">each</span>
<span class="n">page</span> <span class="o">=</span> <span class="mi">1</span>
<span class="kp">loop</span> <span class="k">do</span>
<span class="n">results</span> <span class="o">=</span> <span class="n">client</span><span class="p">.</span><span class="nf">http_get</span><span class="p">(</span><span class="s1">'/search'</span><span class="p">,</span> <span class="ss">keywords: </span><span class="n">keywords</span><span class="p">,</span> <span class="ss">page: </span><span class="n">page</span><span class="p">)</span>
<span class="k">break</span> <span class="k">if</span> <span class="n">results</span><span class="p">.</span><span class="nf">empty?</span>
<span class="no">JSON</span><span class="p">.</span><span class="nf">parse</span><span class="p">(</span><span class="n">results</span><span class="p">).</span><span class="nf">each</span> <span class="k">do</span> <span class="o">|</span><span class="n">result</span><span class="o">|</span>
<span class="k">yield</span> <span class="n">result</span>
<span class="k">end</span>
<span class="n">page</span> <span class="o">+=</span> <span class="mi">1</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>
<p>Notice how our <code class="highlighter-rouge">SearchResultSet</code> class transparently encapsulates the API’s pagination behavior. The caller no longer has to know how the API works. Instead, callers simply fetch results and iterate over them using a mechanism they’re already familar with - <code class="highlighter-rouge">#each</code>.</p>
<p>Here’s an example.</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">client</span> <span class="o">=</span> <span class="no">SearchClient</span><span class="p">.</span><span class="nf">new</span>
<span class="n">results</span> <span class="o">=</span> <span class="n">client</span><span class="p">.</span><span class="nf">search_for</span><span class="p">(</span><span class="s1">'avocados'</span><span class="p">)</span>
<span class="n">results</span><span class="p">.</span><span class="nf">each</span> <span class="k">do</span> <span class="o">|</span><span class="n">result</span><span class="o">|</span>
<span class="nb">puts</span> <span class="n">result</span><span class="p">[</span><span class="s1">'id'</span><span class="p">]</span> <span class="c1"># or whatever</span>
<span class="k">end</span>
</code></pre></div></div>
<h3 id="mixing-in-enumerable">Mixing in <code class="highlighter-rouge">Enumerable</code></h3>
<p>Remember when I said a bunch of Ruby’s core classes like <code class="highlighter-rouge">Array</code> and <code class="highlighter-rouge">Hash</code> include <code class="highlighter-rouge">Enumerable</code>? I meant that they quite literally <code class="highlighter-rouge">include</code> the <code class="highlighter-rouge">Enumerable</code> module.</p>
<p>And because <code class="highlighter-rouge">Enumerable</code> is just a regular ‘ol Ruby module, <strong><em>you can include it too</em></strong>.</p>
<p>In fact, <code class="highlighter-rouge">Enumerable</code> was <em>designed</em> to be mixed into (i.e. <code class="highlighter-rouge">include</code>d) into any Ruby class. The only requirement is that the class defines an <code class="highlighter-rouge">#each</code> method.</p>
<p><strong><em>That’s because every other <code class="highlighter-rouge">Enumerable</code> method is implemented in terms of <code class="highlighter-rouge">#each</code></em></strong>.</p>
<p>Yes, that’s right. Simply defining an <code class="highlighter-rouge">#each</code> method and <code class="highlighter-rouge">include</code>ing the <code class="highlighter-rouge">Enumerable</code> module into your class gives you all the power of <code class="highlighter-rouge">Enumerable</code> <strong>FOR FREE</strong>. In other words, you get <code class="highlighter-rouge">#map</code>, <code class="highlighter-rouge">#each_slice</code>, and all the other <code class="highlighter-rouge">Enumerable</code> methods without having to lift a finger.</p>
<p>Let’s <code class="highlighter-rouge">include Enumerable</code> into our <code class="highlighter-rouge">SearchResultSet</code> class. With that very minimal effort, this is now possible:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">client</span> <span class="o">=</span> <span class="no">SearchClient</span><span class="p">.</span><span class="nf">new</span>
<span class="n">results</span> <span class="o">=</span> <span class="n">client</span><span class="p">.</span><span class="nf">search_for</span><span class="p">(</span><span class="s1">'avocados'</span><span class="p">)</span>
<span class="n">ids</span> <span class="o">=</span> <span class="n">results</span><span class="p">.</span><span class="nf">map</span> <span class="p">{</span> <span class="o">|</span><span class="n">result</span><span class="o">|</span> <span class="n">result</span><span class="p">[</span><span class="s1">'id'</span><span class="p">]</span> <span class="p">}</span>
</code></pre></div></div>
<p>Notice that we didn’t define <code class="highlighter-rouge">#map</code> on <code class="highlighter-rouge">SearchResultSet</code> directly - it came from <code class="highlighter-rouge">Enumerable</code>. By the same token, <code class="highlighter-rouge">#each_slice</code>, <code class="highlighter-rouge">#each_cons</code>, <code class="highlighter-rouge">#inject</code>, and many, many other useful methods are now available too. What’s more, they all Just Work. Not bad for a few lines of code.</p>
<h3 id="the-case-of-the-missing-block">The Case of the Missing Block</h3>
<p>There’s one last thing I’d like to talk about before wrapping up, and that’s lazy enumerators.</p>
<p>What happens if we call <code class="highlighter-rouge">SearchResultSet#each</code> without a block?</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">results</span><span class="p">.</span><span class="nf">each</span>
<span class="c1"># => LocalJumpError: yield called out of block</span>
</code></pre></div></div>
<p>Hmm, that’s weird. I don’t get an error if I try the same thing on an array:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="p">[</span><span class="mi">5</span><span class="p">,</span> <span class="mi">3</span><span class="p">,</span> <span class="mi">8</span><span class="p">].</span><span class="nf">each</span>
<span class="c1"># => #<Enumerator: [5, 3, 8]:each></span>
</code></pre></div></div>
<p>In Ruby, the <code class="highlighter-rouge">yield</code> keyword doesn’t check to make sure the caller passed a block. We can therefore avoid the <code class="highlighter-rouge">LocalJumpError</code> by checking for the block, and bailing out if one wasn’t passed.</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">def</span> <span class="nf">each</span>
<span class="k">return</span> <span class="k">unless</span> <span class="nb">block_given?</span>
<span class="n">page</span> <span class="o">=</span> <span class="mi">1</span>
<span class="o">...</span>
<span class="k">end</span>
</code></pre></div></div>
<p>Ok, let’s try that again:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">results</span><span class="p">.</span><span class="nf">each</span>
<span class="c1"># => nil</span>
</code></pre></div></div>
<p>Not quite what we wanted, but at least there’s no error. We need to figure out how to return the same kind of <code class="highlighter-rouge">Enumerator</code> object we got when calling a blockless <code class="highlighter-rouge">#each</code> on an array.</p>
<h3 id="kernelto_enum"><code class="highlighter-rouge">Kernel#to_enum</code></h3>
<p>Fortunately, there’s an easy way to convert any function into an <code class="highlighter-rouge">Enumerator</code> - Ruby’s <code class="highlighter-rouge">Kernel#to_enum</code>.</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">def</span> <span class="nf">each</span>
<span class="k">return</span> <span class="n">to_enum</span><span class="p">(</span><span class="ss">:each</span><span class="p">)</span> <span class="k">unless</span> <span class="nb">block_given?</span>
<span class="n">page</span> <span class="o">=</span> <span class="mi">1</span>
<span class="o">...</span>
<span class="k">end</span>
</code></pre></div></div>
<p>Now, calling <code class="highlighter-rouge">SearchResultSet#each</code> without a block will return an <code class="highlighter-rouge">Enumerator</code> object.</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">results</span><span class="p">.</span><span class="nf">each</span>
<span class="c1"># => #<Enumerator: #<SearchResultSet:0x00007fa715aaee70 @client=#<SearchClient:0x00007fa715aaeec0>, @keywords="avocado">:each></span>
</code></pre></div></div>
<h3 id="chaining-enumerators">Chaining Enumerators</h3>
<p>Ok, so why do this? While not particularly important for the <code class="highlighter-rouge">#each</code> method, returning an <code class="highlighter-rouge">Enumerator</code> when called without a block is the way all the other <code class="highlighter-rouge">Enumerable</code> methods work. I think it’s a good idea to be consistent.</p>
<p>Another less important reason is to enable chaining. <code class="highlighter-rouge">Enumerator</code>s respond to all the methods in <code class="highlighter-rouge">Enumerable</code>, meaning things like this are possible:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">enum</span> <span class="o">=</span> <span class="n">results</span><span class="p">.</span><span class="nf">each</span>
<span class="n">enum</span><span class="p">.</span><span class="nf">map</span><span class="p">.</span><span class="nf">with_index</span> <span class="k">do</span> <span class="o">|</span><span class="n">result</span><span class="p">,</span> <span class="n">idx</span><span class="o">|</span>
<span class="c1"># result is the search result, and idx is a counter automatically</span>
<span class="c1"># incremented on each iteration</span>
<span class="k">end</span>
</code></pre></div></div>
<h3 id="conclusion">Conclusion</h3>
<p>Enumerators and the <code class="highlighter-rouge">Enumerable</code> module are my all-time favorite Ruby features. They are what got me hooked on Ruby when I first started using it back in 2011. No other language I’ve used has been able to match the same level of expressiveness and flexibility.</p>
<p>I hope this post inspires you to use <code class="highlighter-rouge">Enumerable</code> in new and interesting ways!</p>Cameron DutroThis post was originally written in 2014 at the beginning of my tenure at Lumos Labs. At the time, I was a member of the Learning Team, an “extracurricular” group that met bi-weekly to discuss cool things we were learning about technology. We organized tech meetups in our office space, streamed live Google IO talks over the projector during lunch, and sent out a digest email to our colleagues every two weeks with links to various learning resources. I ended up writing a few longer-form articles for these email blasts. What follows is an embellished version of one of those articles.Encapsulation is a Lie2021-05-28T12:03:30-07:002021-05-28T12:03:30-07:00/2021/05/28/encapsulation-is-a-lie<p>In this post I respond to another of Jason Swett’s recent articles, <a href="https://www.codewithjason.com/dont-wrap-instance-variables-attr_reader-unless-necessary/">Don’t wrap instance variables in attr_reader unless necessary</a>. Jason, if you’re reading this please know this blog isn’t only about critiquing your writing, which I find insightful and thought-provoking. You’ve really gotten me thinking lately, and I’ve been meaning to start a blog for a long time anyway. Seemed like a good opportunity to finally get one going.</p>
<hr />
<h3 id="what-is-attr_reader">What is <code class="highlighter-rouge">attr_reader</code>?</h3>
<p>It’s common to see Ruby classes expose instance variables using a special class method called <code class="highlighter-rouge">attr_reader</code>, eg:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">class</span> <span class="nc">Email</span>
<span class="nb">attr_reader</span> <span class="ss">:subject</span><span class="p">,</span> <span class="ss">:body</span>
<span class="k">def</span> <span class="nf">initialize</span><span class="p">(</span><span class="n">subject</span><span class="p">,</span> <span class="n">body</span><span class="p">)</span>
<span class="vi">@subject</span> <span class="o">=</span> <span class="n">subject</span>
<span class="vi">@body</span> <span class="o">=</span> <span class="n">body</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="n">email</span> <span class="o">=</span> <span class="no">Email</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="s1">'Check this out'</span><span class="p">,</span> <span class="s1">'Ruby rocks'</span><span class="p">)</span>
<span class="nb">puts</span> <span class="n">email</span><span class="p">.</span><span class="nf">subject</span> <span class="c1"># => prints "Check this out"</span>
</code></pre></div></div>
<p>As you can see, <code class="highlighter-rouge">attr_reader :name</code> defines the <code class="highlighter-rouge">#name</code> method on our <code class="highlighter-rouge">Email</code> class. The <code class="highlighter-rouge">#name</code> method simply returns the value of the <code class="highlighter-rouge">@name</code> instance variable.</p>
<p>Ruby also features two other class methods, <code class="highlighter-rouge">attr_writer</code> and <code class="highlighter-rouge">attr_accessor</code>. The former defines a method for assigning a value to an instance variable (eg. <code class="highlighter-rouge">#name=</code>) while the latter defines both the getter and the setter (i.e. both <code class="highlighter-rouge">#name</code> <em>and</em> <code class="highlighter-rouge">#name=</code>).</p>
<p>Why would you ever do this? The basic premise is that instance variables are <em>private</em>, meaning nobody outside the class can get or set them. By wrapping ivars with <code class="highlighter-rouge">attr_reader</code> and friends, they are now available to the outside world.</p>
<h3 id="why-methods-are-almost-always-better">Why Methods are Almost Always Better</h3>
<p>In my opinion, by far the biggest benefit of <code class="highlighter-rouge">attr_reader</code> is that it exposes instance variables as methods. Why are methods better? In a word, <em>inheritance</em>.</p>
<p>There’s no way to override instance variables in Ruby. Consider an ivar-only version of our <code class="highlighter-rouge">Email</code> class from earlier (notice the addition of the <code class="highlighter-rouge">#deliver_to</code> method):</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">require</span> <span class="s1">'mail'</span>
<span class="k">class</span> <span class="nc">Email</span>
<span class="k">def</span> <span class="nf">initialize</span><span class="p">(</span><span class="n">subject</span><span class="p">,</span> <span class="n">body</span><span class="p">)</span>
<span class="vi">@subject</span> <span class="o">=</span> <span class="n">subject</span>
<span class="vi">@body</span> <span class="o">=</span> <span class="n">body</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">deliver_to</span><span class="p">(</span><span class="n">address</span><span class="p">)</span>
<span class="n">mail</span> <span class="o">=</span> <span class="no">Mail</span><span class="p">.</span><span class="nf">new</span>
<span class="n">mail</span><span class="p">[</span><span class="ss">:from</span><span class="p">]</span> <span class="o">=</span> <span class="s1">'no-reply@camerondutro.com'</span>
<span class="n">mail</span><span class="p">[</span><span class="ss">:to</span><span class="p">]</span> <span class="o">=</span> <span class="n">address</span>
<span class="n">mail</span><span class="p">[</span><span class="ss">:subject</span><span class="p">]</span> <span class="o">=</span> <span class="vi">@subject</span>
<span class="n">mail</span><span class="p">[</span><span class="ss">:body</span><span class="p">]</span> <span class="o">=</span> <span class="vi">@body</span>
<span class="n">mail</span><span class="p">.</span><span class="nf">deliver!</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>
<p>Let’s say I want to add a signature at the end of the email body. To do so, I’ll create a new <code class="highlighter-rouge">EmailWithSignature</code> class that inherits from <code class="highlighter-rouge">Email</code>:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">class</span> <span class="nc">EmailWithSignature</span> <span class="o"><</span> <span class="no">Email</span>
<span class="k">end</span>
</code></pre></div></div>
<p>Somewhere in my new class, I need to append the signature to the body. I only really have two options:</p>
<ol>
<li>Override <code class="highlighter-rouge">#initialize</code> and append the signature to <code class="highlighter-rouge">@body</code>.</li>
<li>Override <code class="highlighter-rouge">#deliver_to</code> so it appends the signature on send.</li>
</ol>
<p>Neither of these options feel particularly “clean.” Both <code class="highlighter-rouge">#initialize</code> and <code class="highlighter-rouge">#deliver_to</code> accept arguments, and those arguments can change over time. If I override one of them, I have to make sure my derived <code class="highlighter-rouge">EmailWithSignature</code> class changes whenever <code class="highlighter-rouge">Email</code> does.</p>
<p>If I choose to override <code class="highlighter-rouge">#deliver_to</code>, I have to copy/paste the logic into <code class="highlighter-rouge">EmailWithSignature</code> in order to change the content of the body, or potentially reassign <code class="highlighter-rouge">@body</code> before calling <code class="highlighter-rouge">super</code>. Yuck.</p>
<p>Instead, let’s wrap our instance variables in <code class="highlighter-rouge">attr_reader</code>s (see above). Now the derived class can simply override the <code class="highlighter-rouge">body</code> method, which will always accept zero arguments:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">class</span> <span class="nc">EmailWithSignature</span> <span class="o"><</span> <span class="no">Email</span>
<span class="k">def</span> <span class="nf">body</span>
<span class="k">super</span> <span class="o">+</span> <span class="s2">"</span><span class="se">\n\n</span><span class="s2">-Cameron Dutro</span><span class="se">\n</span><span class="s2">International Man of Mystery"</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>
<p>Much easier, much cleaner.</p>
<p>The limitations of the ivar approach have bitten me numerous times in my professional career. In most cases, requirements had changed and I found myself needing to extend an existing class. Using <code class="highlighter-rouge">attr_reader</code> would have given me the “hooks” I needed to non-intrusively modify the class’s behavior.</p>
<p>Remember that derived classes can call your private methods. If you’re worried about someone else messing with your encapsulated data, just make your <code class="highlighter-rouge">attr_reader</code>s private:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">class</span> <span class="nc">Email</span>
<span class="nb">attr_reader</span> <span class="ss">:subject</span><span class="p">,</span> <span class="ss">:body</span>
<span class="kp">private</span> <span class="ss">:subject</span><span class="p">,</span> <span class="ss">:body</span>
<span class="k">def</span> <span class="nf">initialize</span><span class="p">(</span><span class="n">subject</span><span class="p">,</span> <span class="n">body</span><span class="p">)</span>
<span class="vi">@subject</span> <span class="o">=</span> <span class="n">subject</span>
<span class="vi">@body</span> <span class="o">=</span> <span class="n">body</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="n">email</span> <span class="o">=</span> <span class="no">Email</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="s1">'Check this out'</span><span class="p">,</span> <span class="s1">'Ruby rocks'</span><span class="p">)</span>
<span class="nb">puts</span> <span class="n">email</span><span class="p">.</span><span class="nf">subject</span> <span class="c1"># => raises a NoMethodError</span>
</code></pre></div></div>
<p>However, I prefer to make all of mine public. That’s because I believe encapsulation <em>doesn’t exist</em>.</p>
<h3 id="the-problem-with-encapsulation">The Problem with Encapsulation</h3>
<p>If you’ve gone through any sort of formal computer science training, chances are you’ve been taught the four main pillars of object-oriented programming: <strong>inheritance</strong>, <strong>abstraction</strong>, <strong>polymorphism</strong>, and <strong>encapsulation</strong>.</p>
<p>As you probably already know, encapsulation prevents direct access to an object’s state. In Ruby, that means preventing direct access to an object’s instance variables. The idea is that the object and the object alone should be able to mutate its state, perhaps to maintain some invariants, etc.</p>
<p>However in the vast majority of object-oriented systems, much of what we think of as the internal, encapsulated state of an object is actually <em>shared</em> state. Let’s take a look at a class (granted, a very naïve one) that represents an email address:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">class</span> <span class="nc">EmailAddress</span>
<span class="k">def</span> <span class="nf">initialize</span><span class="p">(</span><span class="n">address</span><span class="p">)</span>
<span class="k">unless</span> <span class="n">address</span><span class="p">.</span><span class="nf">include?</span><span class="p">(</span><span class="s1">'@'</span><span class="p">)</span>
<span class="k">raise</span> <span class="no">InvalidAddressError</span><span class="p">,</span> <span class="s2">"the address '</span><span class="si">#{</span><span class="n">address</span><span class="si">}</span><span class="s2">' is invalid"</span>
<span class="k">end</span>
<span class="vi">@address</span> <span class="o">=</span> <span class="n">address</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">user</span>
<span class="vi">@address</span><span class="p">.</span><span class="nf">split</span><span class="p">(</span><span class="s1">'@'</span><span class="p">)[</span><span class="mi">0</span><span class="p">]</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">host</span>
<span class="vi">@address</span><span class="p">.</span><span class="nf">split</span><span class="p">(</span><span class="s1">'@'</span><span class="p">)[</span><span class="mi">1</span><span class="p">]</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>
<p>Our class nicely encapsulates <code class="highlighter-rouge">@address</code>. Nobody else should be able to mess with it right?</p>
<p>Wrong!</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">address_str</span> <span class="o">=</span> <span class="s1">'foo@bar.com'</span>
<span class="n">address</span> <span class="o">=</span> <span class="no">EmailAddress</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="n">address_str</span><span class="p">)</span>
<span class="n">address</span><span class="p">.</span><span class="nf">host</span> <span class="c1"># => "bar.com"</span>
<span class="n">address_str</span><span class="p">.</span><span class="nf">replace</span><span class="p">(</span><span class="s1">'woops!'</span><span class="p">)</span>
<span class="n">address</span><span class="p">.</span><span class="nf">host</span> <span class="c1"># => nil</span>
</code></pre></div></div>
<p>Woops! Since everything in Ruby is passed by reference, it’s entirely possible for the data given to an object to change without the object’s knowledge. In other words, <em>encapsulation can be easily bypassed</em>.</p>
<p>It doesn’t matter if we use ivars or make the <code class="highlighter-rouge">attr_reader</code> private. There’s always the chance someone else is holding on to a reference to our “private” data and can mutate it at will.</p>
<p>To me, that’s a pretty big deal. It means encapsulation is kind of a lie. If you assign ivars in your class’s constructor from passed-in data, you might as well expose them with <code class="highlighter-rouge">attr_reader</code>s. They’re basically public anyway.</p>
<h3 id="encapsulating-better">Encapsulating Better</h3>
<p>Hold on. If encapsulation doesn’t exist, then doesn’t that call all of object-oriented programming into question?</p>
<p>Maybe, but I’m not the right person to say one way or the other. I happen to really enjoy object-oriented programming. It fits the way my brain works.</p>
<p>But just like a number of aspects of software development, programming well requires <em>discipline</em>. I posit that large object-oriented systems stay afloat because programmers, mostly unconsciously, develop an understanding of the nuances of encapsulation and evolve habits to avoid the major pitfalls. That’s certainly been the case for me. It was only in thinking deeply about this article that I began to ask myself why, for example, reassigning instance variables feels wrong to me.</p>
<p>To that end, I like to follow these rules:</p>
<ol>
<li>Only set instance variables once. After they are set, treat them like constants.</li>
<li>Copy objects before mutating them.</li>
</ol>
<p>The goal here is to prevent our objects from changing except at well-known points in time.</p>
<h4 id="only-set-instance-variables-once">Only Set Instance Variables Once</h4>
<p>In most cases, I think reassigning instance variables is a code smell. If your object needs to change what data it references, <em>it should have asked for different data at initialization</em>.</p>
<p><strong>NOTE</strong>: Methods that use instance variables to memoize the result of a lazily evaluated expression are ok because, although the object didn’t ask for the data at initialization, it still only sets the instance variable once.</p>
<p>The less your object’s state changes after initialization, the less uncertainty you introduce into the system.</p>
<h4 id="copy-objects-before-mutating-them">Copy Objects Before Mutating Them</h4>
<p>Consider this classic example of pass-by-reference mayhem:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">def</span> <span class="nf">compliment</span><span class="p">(</span><span class="nb">name</span><span class="p">)</span>
<span class="nb">puts</span> <span class="nb">name</span> <span class="o"><<</span> <span class="s1">', you rock!'</span>
<span class="k">end</span>
<span class="nb">name</span> <span class="o">=</span> <span class="s1">'Cameron'</span>
<span class="n">compliment</span><span class="p">(</span><span class="nb">name</span><span class="p">)</span>
<span class="nb">name</span> <span class="c1"># => "Cameron, you rock!"</span>
</code></pre></div></div>
<p>The caller probably isn’t expecting <code class="highlighter-rouge">compliment</code> to mutate <code class="highlighter-rouge">name</code>.</p>
<p>The same principle applies to data referenced by an object’s instance variables. Mutating only copies of your data prevents these sorts of surprises.</p>
<h3 id="back-to-attr_reader">Back to <code class="highlighter-rouge">attr_reader</code></h3>
<p>Ok, but what does all this have to do with <code class="highlighter-rouge">attr_reader</code>?</p>
<p>In his usual erudite manner, Jason lays out the case against <code class="highlighter-rouge">attr_reader</code> with the following statement:</p>
<blockquote>
<p><strong>Adding a public <code class="highlighter-rouge">attr_reader</code> throws away the benefits of encapsulation</strong></p>
<p>Private instance variables are useful for the same reason as private methods: <strong>because you know they’re not depended on by outside clients</strong>.</p>
<p>If I have a class that has an instance variable called <code class="highlighter-rouge">@price</code>, I know that I can rename that instance variable to <code class="highlighter-rouge">@cost</code> or change it to <code class="highlighter-rouge">@price_cents</code> (changing the whole meaning of the value) or even kill <code class="highlighter-rouge">@price</code> altogether. What I want to do with <code class="highlighter-rouge">@price</code> is 100% my business. This is great.</p>
</blockquote>
<p>Hopefully I’ve shown in this article why encapsulation is more or less a myth. There’s no way to know who else may reference the same data as your instance variables, so it very well may not be “100% your business.” This is especially true if one of your methods returns it somehow, perhaps wrapped by another object.</p>
<p>I do agree that adding <code class="highlighter-rouge">attr_reader :price</code> changes the class’s public interface, making it more difficult to change in the future. Unless you’re developing library code or a gem however, I think the public interface argument is fairly weak. As Jason says a few paragraphs earlier, if a naming issue causes a problem in your application, your application probably isn’t tested well enough.</p>
<h3 id="conclusion">Conclusion</h3>
<p>My takeaway message is this: encapsulation doesn’t provide the guarantees we were taught it does. There’s no real benefit to “hiding” data inside an object, since that data may be referenced - privately or publicly - by any number of other objects. You might as well expose your instance variables with <code class="highlighter-rouge">attr_reader</code> for the inheritance benefits.</p>
<p>Disagree? <a href="https://twitter.com/camertron">Hit me up</a> on Twitter.</p>Cameron DutroIn this post I respond to another of Jason Swett’s recent articles, Don’t wrap instance variables in attr_reader unless necessary. Jason, if you’re reading this please know this blog isn’t only about critiquing your writing, which I find insightful and thought-provoking. You’ve really gotten me thinking lately, and I’ve been meaning to start a blog for a long time anyway. Seemed like a good opportunity to finally get one going.The Case for Service Objects2021-05-21T10:17:29-07:002021-05-21T10:17:29-07:00/2021/05/21/the-case-for-service-objects<p>This article is a response to Jason Swett’s <a href="https://www.codewithjason.com/rails-service-objects/">“Beware of ‘service objects’ in Rails”</a> blog post. In it, Jason warns of the dangers of letting service objects rob you of the benefits of object-oriented programming. I’ve read Jason’s post several times, and listened to a number of discussions he’s had about service objects on his podcast, <a href="https://www.codewithjason.com/rails-with-jason-podcast/">Rails with Jason</a>.</p>
<p><strong>By the way, both Jason’s blog and podcast are excellent. Go check them out right now 😊</strong></p>
<p>Just last week, Jason was invited onto the Remote Ruby podcast where he and the panelists <a href="https://remoteruby.transistor.fm/129">discussed service objects again</a>. Something about the converstaion struck a cord with me. I’ve listened to Jason talk about his distaste for service objects for a long time. I think he’s right, but also wrong. What follows are my thoughts on the humble, oft misunderstood, service object.</p>
<hr />
<h3 id="intro">Intro</h3>
<p>Rails has been around for a long time now. It feels weird to write this, but it’ll be Rails’ 20th birthday in just a few short years. For those of us who’ve used the framework for a long time, 20 years feels like a pretty incredible milestone.</p>
<p>I started using Rails ~11 years ago, pretty much straight out of college. I worked on Twitter’s International Engineering Team on the Twitter Translation Center, a Rails app that managed our large database of localized content and facilitated contributions from thousands of volunteer translators around the world. It was my first time using Ruby, and I absolutely fell in love with it. Ruby and Rails made CakePHP, the framework I was using at the time for my side projects, feel pretty clunky and outmoded. Ruby and Rails are still my favorite language and framework today, and I know many other devs who feel the same way.</p>
<p>Why has Rails had such staying power? I would argue there are two major reasons:</p>
<ol>
<li>The Ruby and Rails communities are nonpareil in the software world, and</li>
<li>Rails keeps evolving.</li>
</ol>
<p>Hotwire is just the latest example of the evolution Rails devs have enjoyed for the last two decades. I invite you to look back on the asset pipeline, action cable, and turbolinks as a few examples from the past that also changed the game.</p>
<h3 id="weve-evolved-too">We’ve Evolved Too</h3>
<p>While the framework has changed, so have we as Rails developers. A few years ago much noise was made over the “fat model, skinny controller” concept (in case you’re not familiar, the idea is to keep your controller code to a minimum and put all your domain logic into the model layer).</p>
<p>In fact, I would posit that a number of the changes in thinking we’ve gone through as a community have been related to code organization. Where do you put that odd piece of code that doesn’t seem to fit in any of Rails’ predefined slots?</p>
<p>One of the answers is to give Rails <em>new</em> slots:</p>
<ol>
<li>The <a href="https://github.com/drapergem/draper">draper gem</a> adds the app/decorators directory for “view models,” i.e. view presenters.</li>
<li>The <a href="https://www.codementor.io/@victor_hazbun/complex-form-objects-in-rails-qval6b8kt">form objects</a> design pattern adds the app/forms directory for handling complex forms.</li>
<li>The <a href="https://github.com/varvet/pundit">pundit gem</a> adds the app/policies directory for specifying authorization rules.</li>
<li>The <a href="https://github.com/github/view_component">view_component gem</a> adds the app/components directory for components that encapsulate view code.</li>
<li>Etc, etc.</li>
</ol>
<p>Of course Rails itself also adds new slots from time to time:</p>
<ol>
<li>Rails 3.0 introduced <a href="https://api.rubyonrails.org/v6.1.3.1/classes/ActiveSupport/Concern.html">concerns</a> and the app/models/concerns folder for augmenting models.</li>
<li>Rails 4.2 introduced <a href="https://guides.rubyonrails.org/active_job_basics.html">active job</a> and the app/jobs folder for background jobs.</li>
<li>Also see app/assets, app/channels, etc.</li>
</ol>
<p>While all these also added awesome new features to Rails, let’s not overlook how significant it is that they introduced a bunch of additional slots to help us organize our code better. In fact, you’d probably agree that a <em>lot</em> of Rails’ power comes from its predefined folder structure (just ask your favorite React dev 😏).</p>
<h3 id="service-objects">Service Objects</h3>
<p>Service objects are yet another slot for organizing our Ruby code, albeit a fairly misunderstood one. For instance, although the community has produced a number of gems for creating service objects, we haven’t really coalesced around any one of them. That’s probably because service objects in their purest form are just Ruby classes with a fancy name.</p>
<p>For example, here’s a service object that creates a user:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">class</span> <span class="nc">CreateUser</span>
<span class="k">def</span> <span class="nc">self</span><span class="o">.</span><span class="nf">call</span><span class="p">(</span><span class="n">params</span><span class="p">)</span>
<span class="no">User</span><span class="p">.</span><span class="nf">create</span><span class="p">(</span><span class="n">params</span><span class="p">)</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>
<p>Now, I hear you saying, “Wait wait, that’s not what service objects are!” and you’re right. The term “service object” means different things to different people. In my opinion however, here’s what a “service object” is:</p>
<ol>
<li>A plain ‘ol Ruby object with a <code class="highlighter-rouge">call</code> method.</li>
</ol>
<p>That’s literally it.</p>
<p>Ok there is one other thing. You may have noticed that the name of the example service object above, <code class="highlighter-rouge">CreateUser</code>, sounds more like the name of a method than a class. That’s intentional.</p>
<p>I like to think of service objects as representing <em>actions</em>.</p>
<h3 id="skinny-controllers">Skinny Controllers</h3>
<p>“Hey,” I hear you saying. “Actions… like in controllers?”</p>
<p>Yes! In the applications I’ve worked on, <em><strong>service objects were extracted exclusively from controller actions</strong></em>.</p>
<p>This is the major point on which Jason and I differ. Whereas he writes about service objects as being part of the <em>models</em> directory, I think of them as being part of the <em>controllers</em> directory. In my mind, “services” are just miniature web applications. Service <em>objects</em> therefore should aid in responding to web requests.</p>
<p>Consider again our humble <code class="highlighter-rouge">CreateUser</code> service object. We can easily imagine how the <code class="highlighter-rouge">User.create</code> call inside it could have once been part of a controller action:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">class</span> <span class="nc">UsersController</span> <span class="o"><</span> <span class="no">ApplicationController</span>
<span class="k">def</span> <span class="nf">create</span>
<span class="vi">@user</span> <span class="o">=</span> <span class="no">User</span><span class="p">.</span><span class="nf">create</span><span class="p">(</span><span class="n">user_params</span><span class="p">)</span>
<span class="k">if</span> <span class="vi">@user</span><span class="p">.</span><span class="nf">valid?</span>
<span class="no">UserMailer</span><span class="p">.</span><span class="nf">with</span><span class="p">(</span><span class="ss">user: </span><span class="vi">@user</span><span class="p">).</span><span class="nf">welcome_email</span><span class="p">.</span><span class="nf">deliver_later</span>
<span class="n">redirect_to</span> <span class="n">dashboard_path</span><span class="p">,</span> <span class="ss">notice: </span><span class="s1">'Welcome aboard!'</span>
<span class="k">else</span>
<span class="n">render</span> <span class="ss">:new</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="kp">private</span>
<span class="k">def</span> <span class="nf">user_params</span>
<span class="n">params</span><span class="p">.</span><span class="nf">require</span><span class="p">(</span><span class="ss">:user</span><span class="p">).</span><span class="nf">permit</span><span class="p">(</span><span class="ss">:first_name</span><span class="p">,</span> <span class="ss">:last_name</span><span class="p">,</span> <span class="ss">:email_address</span><span class="p">)</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>
<p>Notice that the <code class="highlighter-rouge">#create</code> action creates the user, but also sends a welcome email.</p>
<p>As your app grows, so do your controller actions. Maybe you decide you want to register the user with your 3rd-party email/marketing system when they sign up. A few months later you decide to A/B test sending a free trial email at signup instead of the traditional welcome email:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">class</span> <span class="nc">UsersController</span> <span class="o"><</span> <span class="no">ApplicationController</span>
<span class="k">def</span> <span class="nf">create</span>
<span class="vi">@user</span> <span class="o">=</span> <span class="no">User</span><span class="p">.</span><span class="nf">create</span><span class="p">(</span><span class="n">user_params</span><span class="p">)</span>
<span class="k">if</span> <span class="vi">@user</span><span class="p">.</span><span class="nf">valid?</span>
<span class="no">BrazeClient</span><span class="p">.</span><span class="nf">new</span><span class="p">.</span><span class="nf">register_email</span><span class="p">(</span><span class="vi">@user</span><span class="p">.</span><span class="nf">email_address</span><span class="p">)</span>
<span class="k">if</span> <span class="no">Flipper</span><span class="p">.</span><span class="nf">enabled?</span><span class="p">(</span><span class="ss">:free_trial_email_at_signup</span><span class="p">,</span> <span class="vi">@user</span><span class="p">)</span>
<span class="no">UserMailer</span><span class="p">.</span><span class="nf">with</span><span class="p">(</span><span class="ss">user: </span><span class="vi">@user</span><span class="p">).</span><span class="nf">free_trial_email</span><span class="p">.</span><span class="nf">deliver_later</span>
<span class="k">else</span>
<span class="no">UserMailer</span><span class="p">.</span><span class="nf">with</span><span class="p">(</span><span class="ss">user: </span><span class="vi">@user</span><span class="p">).</span><span class="nf">welcome_email</span><span class="p">.</span><span class="nf">deliver_later</span>
<span class="k">end</span>
<span class="n">redirect_to</span> <span class="n">dashboard_path</span><span class="p">,</span> <span class="ss">notice: </span><span class="s1">'Welcome aboard!'</span>
<span class="k">else</span>
<span class="n">render</span> <span class="ss">:new</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="kp">private</span>
<span class="k">def</span> <span class="nf">user_params</span>
<span class="n">params</span><span class="p">.</span><span class="nf">require</span><span class="p">(</span><span class="ss">:user</span><span class="p">).</span><span class="nf">permit</span><span class="p">(</span><span class="ss">:first_name</span><span class="p">,</span> <span class="ss">:last_name</span><span class="p">,</span> <span class="ss">:email_address</span><span class="p">)</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>
<p>Whoa, that <code class="highlighter-rouge">#create</code> method is getting pretty long. More concerning though is how much logic it encapsulates - logic that can’t be reused outside the controller. In addition, I’ve only shown a single action in this example. A complete RESTful controller will have seven.</p>
<p>Let’s pull all that creation code into the service object instead:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">class</span> <span class="nc">CreateUser</span>
<span class="k">def</span> <span class="nc">self</span><span class="o">.</span><span class="nf">call</span><span class="p">(</span><span class="n">params</span><span class="p">)</span>
<span class="n">new</span><span class="p">(</span><span class="n">params</span><span class="p">).</span><span class="nf">create</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">initialize</span><span class="p">(</span><span class="n">params</span><span class="p">)</span>
<span class="vi">@params</span> <span class="o">=</span> <span class="n">params</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">create</span>
<span class="n">user</span><span class="p">.</span><span class="nf">tap</span> <span class="k">do</span> <span class="o">|</span><span class="n">u</span><span class="o">|</span>
<span class="k">if</span> <span class="n">u</span><span class="p">.</span><span class="nf">valid?</span>
<span class="n">send_email_address_to_braze</span>
<span class="n">send_signup_email</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="kp">private</span>
<span class="k">def</span> <span class="nf">send_email_address_to_braze</span>
<span class="no">BrazeClient</span><span class="p">.</span><span class="nf">new</span><span class="p">.</span><span class="nf">register_email</span><span class="p">(</span><span class="n">user</span><span class="p">.</span><span class="nf">email_address</span><span class="p">)</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">send_signup_email</span>
<span class="k">if</span> <span class="n">send_free_trial_email?</span>
<span class="no">UserMailer</span><span class="p">.</span><span class="nf">with</span><span class="p">(</span><span class="ss">user: </span><span class="n">user</span><span class="p">).</span><span class="nf">free_trial_email</span><span class="p">.</span><span class="nf">deliver_later</span>
<span class="k">else</span>
<span class="no">UserMailer</span><span class="p">.</span><span class="nf">with</span><span class="p">(</span><span class="ss">user: </span><span class="n">user</span><span class="p">).</span><span class="nf">welcome_email</span><span class="p">.</span><span class="nf">deliver_later</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">send_free_trial_email?</span>
<span class="no">Flipper</span><span class="p">.</span><span class="nf">enabled?</span><span class="p">(</span><span class="ss">:free_trial_email_at_signup</span><span class="p">,</span> <span class="n">user</span><span class="p">)</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">user</span>
<span class="vi">@user</span> <span class="o">||=</span> <span class="no">User</span><span class="p">.</span><span class="nf">create</span><span class="p">(</span><span class="vi">@params</span><span class="p">)</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>
<p>I really like this. Not only is the public API minimal, I can hang a bunch of helper methods onto the class that I might have been hesitant to add to the controller.</p>
<p>And by extracting the user creation logic into the service object, the controller now does a whole lot less:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">class</span> <span class="nc">UsersController</span> <span class="o"><</span> <span class="no">ApplicationController</span>
<span class="k">def</span> <span class="nf">create</span>
<span class="vi">@user</span> <span class="o">=</span> <span class="no">CreateUser</span><span class="o">.</span><span class="p">(</span><span class="n">user_params</span><span class="p">)</span>
<span class="k">if</span> <span class="vi">@user</span><span class="p">.</span><span class="nf">valid?</span>
<span class="n">redirect_to</span> <span class="n">dashboard_path</span><span class="p">,</span> <span class="ss">notice: </span><span class="s1">'Welcome aboard!'</span>
<span class="k">else</span>
<span class="n">render</span> <span class="ss">:new</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="kp">private</span>
<span class="k">def</span> <span class="nf">user_params</span>
<span class="n">params</span><span class="p">.</span><span class="nf">require</span><span class="p">(</span><span class="ss">:user</span><span class="p">).</span><span class="nf">permit</span><span class="p">(</span><span class="ss">:first_name</span><span class="p">,</span> <span class="ss">:last_name</span><span class="p">,</span> <span class="ss">:email_address</span><span class="p">)</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>
<p>But skinny controllers aren’t the only benefit.</p>
<h3 id="bulk-user-importer">Bulk User Importer</h3>
<p>In my mind, the most significant benefit of the service object approach is <em>code reuse</em>.</p>
<p>Let’s say our company starts offering our services b2b and we need to create a bunch of user accounts for all the people who work at another company. We decide to add a bulk user importer to our system that’s capable of reading a CSV file and creating a bunch of user accounts all at once. This exact scenario came up at one of my previous jobs.</p>
<p>Fortunately, our user creation logic is conveniently encapsulated into a service object, so reusing it is a piece of cake:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">require</span> <span class="s1">'csv'</span>
<span class="k">class</span> <span class="nc">UserCsvFile</span>
<span class="nb">attr_reader</span> <span class="ss">:path</span>
<span class="k">def</span> <span class="nf">initialize</span><span class="p">(</span><span class="n">path</span><span class="p">)</span>
<span class="vi">@path</span> <span class="o">=</span> <span class="n">path</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">import</span>
<span class="n">table</span><span class="p">.</span><span class="nf">each</span> <span class="k">do</span> <span class="o">|</span><span class="n">row</span><span class="o">|</span>
<span class="no">CreateUser</span><span class="o">.</span><span class="p">(</span><span class="n">row</span><span class="p">.</span><span class="nf">to_h</span><span class="p">)</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="kp">private</span>
<span class="k">def</span> <span class="nf">table</span>
<span class="vi">@table</span> <span class="o">||=</span> <span class="no">CSV</span><span class="p">.</span><span class="nf">parse</span><span class="p">(</span><span class="no">File</span><span class="p">.</span><span class="nf">read</span><span class="p">(</span><span class="n">path</span><span class="p">),</span> <span class="ss">headers: </span><span class="kp">true</span><span class="p">)</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="no">UserCsvFile</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="s1">'/path/to/users.csv'</span><span class="p">).</span><span class="nf">import</span>
</code></pre></div></div>
<p>You could copy and paste the code from the controller into the <code class="highlighter-rouge">UserCsvFile</code> class, <a href="https://speakerdeck.com/tenderlove/but-at-what-cost">but at what cost?</a> Every time the controller changes, so does <code class="highlighter-rouge">UserCsvFile</code>. At some point, someone’s gonna forget to update both codepaths.</p>
<h3 id="thats-all-there-is-to-it">That’s… all there is to it?</h3>
<p>I’m sure some of you reading this are now thoroughly fed up. Has it really taken this guy over a thousand words just to tell you about Ruby classes?</p>
<p>Well, that’s the thing about service objects. They really can be that simple. In fact, service objects aren’t even a design pattern. They’re just a code organization tool for extracting chunks of procedural code from controller actions, i.e. “do this, then do this, then do this last thing.” The “service object” moniker is just a name. We could easily call these chunks of code “actions” or maybe “commands” as Jason mentions.</p>
<h3 id="loss-of-object-orientation">Loss of Object-Orientation</h3>
<p>In his blog post, Jason makes the following assertion:</p>
<blockquote>
<p><strong>Service objects throw out the fundamental advantages of object-oriented programming.</strong></p>
<p>“Objects” like this aren’t abstractions of concepts in the domain model. They’re chunks of procedural code masquerading as object-oriented code.</p>
</blockquote>
<p>He’s absolutely right that service objects aren’t abstractions of concepts in the domain model. They exist to encapsulate procedural code. After all, controller actions tend to be procedural, so it follows that service objects are as well.</p>
<p>This encapsulation idea is one of the tenets of object-oriented programming; the data needed to perform the action is held by the object, and the object’s method’s (<code class="highlighter-rouge">call</code> in our case) uses that data to perform the action.</p>
<h3 id="advanced-techniques">Advanced Techniques</h3>
<p>Because service objects are just classes with basically no rules, you have the full power of the Ruby language at your disposal. Pretty much anything goes.</p>
<p>For example, consider the various ways our <code class="highlighter-rouge">CreateUser</code> operation can fail. Might be kinda nice to support some failure modes:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">class</span> <span class="nc">UsersController</span> <span class="o"><</span> <span class="no">ApplicationController</span>
<span class="k">def</span> <span class="nf">create</span>
<span class="no">CreateUser</span><span class="o">.</span><span class="p">(</span><span class="n">user_params</span><span class="p">)</span> <span class="k">do</span> <span class="o">|</span><span class="n">result</span><span class="o">|</span>
<span class="c1"># creation succeeded</span>
<span class="n">result</span><span class="p">.</span><span class="nf">success</span> <span class="k">do</span> <span class="o">|</span><span class="n">_user</span><span class="o">|</span>
<span class="n">redirect_to</span> <span class="n">dashboard_path</span><span class="p">,</span> <span class="ss">notice: </span><span class="s1">'Welcome aboard!'</span>
<span class="k">end</span>
<span class="c1"># user is invalid</span>
<span class="n">result</span><span class="p">.</span><span class="nf">failure</span> <span class="k">do</span> <span class="o">|</span><span class="n">user</span><span class="o">|</span>
<span class="vi">@user</span> <span class="o">=</span> <span class="n">user</span>
<span class="n">render</span> <span class="ss">:new</span>
<span class="k">end</span>
<span class="c1"># CreateUser.() raised an error</span>
<span class="n">result</span><span class="p">.</span><span class="nf">error</span> <span class="k">do</span> <span class="o">|</span><span class="n">e</span><span class="o">|</span>
<span class="no">Rollbar</span><span class="p">.</span><span class="nf">error</span><span class="p">(</span><span class="n">e</span><span class="p">)</span>
<span class="n">flash</span><span class="p">.</span><span class="nf">now</span><span class="p">[</span><span class="ss">:error</span><span class="p">]</span> <span class="o">=</span> <span class="s1">'Something went wrong, please try again'</span>
<span class="n">render</span> <span class="ss">:new</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="kp">private</span>
<span class="k">def</span> <span class="nf">user_params</span>
<span class="n">params</span><span class="p">.</span><span class="nf">require</span><span class="p">(</span><span class="ss">:user</span><span class="p">).</span><span class="nf">permit</span><span class="p">(</span><span class="ss">:first_name</span><span class="p">,</span> <span class="ss">:last_name</span><span class="p">,</span> <span class="ss">:email_address</span><span class="p">)</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>
<p>Dang that’s nice 🔥🔥🔥</p>
<p>Here’s what the service object might look like:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">class</span> <span class="nc">CreateUser</span>
<span class="k">def</span> <span class="nc">self</span><span class="o">.</span><span class="nf">call</span><span class="p">(</span><span class="n">params</span><span class="p">)</span>
<span class="n">new</span><span class="p">(</span><span class="n">params</span><span class="p">).</span><span class="nf">create</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">initialize</span><span class="p">(</span><span class="n">params</span><span class="p">)</span>
<span class="vi">@params</span> <span class="o">=</span> <span class="n">params</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">create</span>
<span class="k">if</span> <span class="n">user</span><span class="p">.</span><span class="nf">valid?</span>
<span class="n">send_email_address_to_braze</span>
<span class="n">send_signup_email</span>
<span class="no">Success</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="n">user</span><span class="p">)</span>
<span class="k">else</span>
<span class="no">Failure</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="n">user</span><span class="p">)</span>
<span class="k">end</span>
<span class="k">rescue</span> <span class="no">Exception</span> <span class="o">=></span> <span class="n">e</span>
<span class="no">Error</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="n">e</span><span class="p">)</span>
<span class="k">end</span>
<span class="kp">private</span>
<span class="k">def</span> <span class="nf">send_email_address_to_braze</span>
<span class="no">BrazeClient</span><span class="p">.</span><span class="nf">new</span><span class="p">.</span><span class="nf">register_email</span><span class="p">(</span><span class="n">user</span><span class="p">.</span><span class="nf">email_address</span><span class="p">)</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">send_signup_email</span>
<span class="k">if</span> <span class="n">send_free_trial_email?</span>
<span class="no">UserMailer</span><span class="p">.</span><span class="nf">with</span><span class="p">(</span><span class="ss">user: </span><span class="n">user</span><span class="p">).</span><span class="nf">free_trial_email</span><span class="p">.</span><span class="nf">deliver_later</span>
<span class="k">else</span>
<span class="no">UserMailer</span><span class="p">.</span><span class="nf">with</span><span class="p">(</span><span class="ss">user: </span><span class="n">user</span><span class="p">).</span><span class="nf">welcome_email</span><span class="p">.</span><span class="nf">deliver_later</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">send_free_trial_email?</span>
<span class="no">Flipper</span><span class="p">.</span><span class="nf">enabled?</span><span class="p">(</span><span class="ss">:free_trial_email_at_signup</span><span class="p">,</span> <span class="n">user</span><span class="p">)</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">user</span>
<span class="vi">@user</span> <span class="o">||=</span> <span class="no">User</span><span class="p">.</span><span class="nf">create</span><span class="p">(</span><span class="vi">@params</span><span class="p">)</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>
<p>And finally here are the result classes:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">class</span> <span class="nc">Result</span>
<span class="k">def</span> <span class="nf">initialize</span><span class="p">(</span><span class="n">args</span><span class="p">)</span>
<span class="vi">@args</span> <span class="o">=</span> <span class="n">args</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">success</span><span class="p">(</span><span class="o">&</span><span class="n">block</span><span class="p">);</span> <span class="k">end</span>
<span class="k">def</span> <span class="nf">failure</span><span class="p">(</span><span class="o">&</span><span class="n">block</span><span class="p">);</span> <span class="k">end</span>
<span class="k">def</span> <span class="nf">error</span><span class="p">(</span><span class="o">&</span><span class="n">block</span><span class="p">);</span> <span class="k">end</span>
<span class="k">end</span>
<span class="k">class</span> <span class="nc">Success</span> <span class="o"><</span> <span class="no">Result</span>
<span class="k">def</span> <span class="nf">success</span><span class="p">(</span><span class="o">&</span><span class="n">block</span><span class="p">)</span>
<span class="k">yield</span> <span class="o">*</span><span class="vi">@args</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">class</span> <span class="nc">Failure</span> <span class="o"><</span> <span class="no">Result</span>
<span class="k">def</span> <span class="nf">failure</span><span class="p">(</span><span class="o">&</span><span class="n">block</span><span class="p">)</span>
<span class="k">yield</span> <span class="o">*</span><span class="vi">@args</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">class</span> <span class="nc">Error</span> <span class="o"><</span> <span class="no">Result</span>
<span class="k">def</span> <span class="nf">error</span><span class="p">(</span><span class="o">&</span><span class="n">block</span><span class="p">)</span>
<span class="k">yield</span> <span class="o">*</span><span class="vi">@args</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>
<h3 id="conclusion">Conclusion</h3>
<p>I hope this article has explained why service objects deserve a place in your Rails app. Just remember to keep ‘em out of your model code.</p>
<p>Disagree? <a href="https://twitter.com/camertron">Hit me up</a> on Twitter.</p>Cameron DutroThis article is a response to Jason Swett’s “Beware of ‘service objects’ in Rails” blog post. In it, Jason warns of the dangers of letting service objects rob you of the benefits of object-oriented programming. I’ve read Jason’s post several times, and listened to a number of discussions he’s had about service objects on his podcast, Rails with Jason.