As web developers, we've all come to appreciate the layer of sanity that libraries like jQuery slather atop the inconsistencies and awkwardness of what the platform provides. What was once constructing an
XMLHTTPRequest object over a handful of lines becomes a single-line invocation of
$.ajax, and interaction with the DOM through jQuery rarely involves platform-specific hacks and workarounds.
Anyone who's used languages like Python1 or Java knows what it's like to have a batteries-included standard library available. In the browser, jQuery was (and to many, still is) that standard library, along with toolkit libraries like underscore. Like most standard libraries, everyone's code has a tendency to become deeply coupled to it. And with so much code tied to these APIs, like the platform itself, it becomes increasingly difficult to move forward without breaking the web. And forget about including multiple versions of these libraries for compatibility; they're often larger than 30KB and write to the global
window object by default.
I once accepted this problem as an unfortunate truth of developing software. That all changed, for me at least, with the Node.js ecosystem.
The rise of small modules and mass composability
On npm, small modules have become the norm, with users like substack and Sindre Sorhus publishing upwards of 685 and 760 modules respectively, each following the UNIX way of doing one thing and doing it well. Modules such as
array-union, which returns the union of two arrays, and
svg-create-element, which provides a great API for creating SVGs in the DOM, seem so absurdly obvious and small that they ought to ship with the language or platform.
Sindre even has a module named
negative-zero, which simply returns whether or not a value is
-0, and is effectively one line long. While it may appear extreme to create a package out of such simple functionality instead of inlining the function into one's own code, a problem can be solved once and then later expressed by what one means, rather than repeating implementation details. Sindre speaks to this in an incredible reply to an "Ask me anything" question regarding small modules. I highly recommend giving it a read.
Even the incredibly popular Express web framework is distributed as the kernel of a web application. In contrast to large frameworks like Ruby on Rails or Django which come bundled with templating, ORMs, csrf protection, and other features, Express only ships with middleware to host static files. Instead, the application developer is free to use whichever implementation of these features they like, and compose them together to create their application. Many middleware packages come and go as ideas are improved upon and others dismissed. It's the Node way.
The greatest quality of these modules is that they aren't bound tightly to their platform: they're free from being frozen in time by the curse of the standard library, and with the help of Semantic Versioning can freely iterate on their API without breaking all of their dependent users.
A directed acyclic graph representing a small module's dependencies.
A story of streams
Node includes streams, an abstraction over asynchronously flowing data, and is most often used for connecting and transforming sources of I/O. Node's initial implementation of streams, which shipped in Node 0.4, left a lot to be desired and led to potentially lost data if used incorrectly. In response, Node 0.10 shipped with a revision of streams (also known as Streams2). But Streams2 wasn't a simple iteration to the streams pattern, and actually went through a number of changes before shipping in Node 0.10. When it did ship, it was compatible with apps running in Node 0.8 without being backported into a release of Node 0.8.
How could this be possible? Well, Streams2 was born from the readable-stream module, which began life as an independent module by Isaac Schlueter in July of 2012 long before it shipped with Node 0.10 in March of 2013. There, it went through iteration as its API and functionality matured, and the Node community found itself with a far superior implementation of streams.
Even today, the latest implementation of
readable-stream is maintained as a userland module in npm for use in Node versions dating back to 0.8. Many users use the userland module rather than the bundled one altogether to ensure compatibility with the ecosystem.
A series of unfortunate APIs
Object.observe, an ES2016 proposal which allowed observation of an object's properties, was recently withdrawn after React's preference for unidirectional data-flow and immutability took the web by storm.
Confused by all of this yet? We shouldn't be expected to get things right the first time, but we need a platform that lets us get it wrong first, and then iterate towards perfection.
Evolving the platform
Thankfully, we've already seen the benefits of this approach. No longer do we have to be stuck with features like AppCache that are implemented without much real-world usage. Instead, we have a standardized implementation of the Promises A+ Specification, which was proven in npm modules like Q and shipped as part of ES2015 earlier this year. The WHATWG is also working on a specification for streams, which is heavily influenced by the evolution of streams that developed from Node.
Like the rest of the platform, these new standards are admittedly difficult to change, but luckily their ideas and APIs were iterated upon in userland. Thanks Node!
For a great example of the curse of the standard library in Python, check out the saga of urllib, urllib2, and the excellent requests library, which would rather not be included in the standard library ↩