It’s worth pointing out that Yarn, which even promises to give non-Facebook-wide developers a major performance boost, still uses the npm registry and is essentially an instant replacement for the npm client.
As Sebastian McKenzie, Facebook software engineer, and Tom Occhino, engineering manager, told me, the company had developed a lot of internal infrastructure around npm. “But over time, as we added new parts to it, we realized it didn’t work so well,” McKenzie said. So instead of bypassing the limitations of npm, Facebook decided to rewrite it from scratch.
Considering that npm works well for millions of developers, why hasn’t it worked for Facebook? The team told me that there were a few fundamental issues with npm for the company’s workflow. Performance was one of them, so Yarn does a better job of caching files locally, ensuring that it doesn’t have to hit the network as often as it used to. Yarn is also able to parallelize some of its operations, which also speeds up the process of installing new modules.
At Facebook, npm has slowed the company’s continuous integration workflow. Initially, engineers had to manually run the ubiquitous “npm install” command, but this did not work in the sandboxed and isolated continuous integration environments that the company uses for security and reliability reasons. Archiving all modules in a repository was also inefficient as even a minor change could easily trigger massive commits. React Native, for example, currently has 68 dependencies (which themselves have their own dependencies). Once you’ve unwound all of those files, you end up with 121,358 files. This is obviously not very effective.
Another problem Facebook faced was that npm is, by design, non-deterministic – yet Facebook engineers needed a consistent and reliable system for their DevOps workflow. Depending on the modules you have already installed, the node_modules The directory that is part of each project can be very different depending on the developer’s machine you are looking at. Yarn uses lock files and a deterministic installation algorithm to create consistent file structures on all machines.
By default, npm also allows developers who write these packages to run other code needed as part of the installation process. This does create security issues, however, so Yarn does not have this feature.
As McKenzie told me, the team tried to ‘fix’ npm for their purposes, but in the end, many features of the existing npm client that didn’t work for Facebook were not bugs, but bugs. features. Occhino added that many of the features the team wanted to create weren’t the kind of changes the npm community would likely have accepted.
Npm, the business entity that supports the npm project, is obviously aware of this new project, but given that its business model revolves more around the ledger than the customer, there is much less conflict here than there is. think at the outset.
The thread is now available on GitHub. Since a number of other companies contributed to the project, the team decided to host it outside of Facebook’s own repository. However, it’s unclear what Yarn’s governance model will look like. “Our hope is that everyone who has contributed to this so far can help us understand this,” Occhino said.