The function pkg_stability() provides a
Stability Score for a selected package. This function
looks at the entire history of package releases, and compares them to
identify breaking changes. It then uses the number of breaking changes
to calculate a Stability Score. This vignette describes
how the score is calculated.
Before describing how the Stability Score is calculated, we first have to understand what is meant by a “breaking change”. In the context of pkgdiff, a “breaking change” is a function that exists in one release, but is removed in the next. It can also be a function parameter that exists in one release, but is removed in the next. There are, therefore, two kinds of breaking changes tracked by pkgdiff: removed functions and removed parameters. Either one of these will count as a breaking change.
Note that there are other kinds of changes not tracked by pkgdiff. Functions that are changed, but not removed, are not tracked. Also, functions that are deprecated, but not removed, are not tracked. That means you can start getting a deprecation warning on a function, but as long as it is not actually removed, pkgdiff does not count that as a breaking change.
A package version with at least one breaking change from the previous version is called a “breaking release”. A package version that is compatible with the previous version is called a “non-breaking release”.
The Stability Score is calculated as the percentage of non-breaking releases. In other words, the Stability Score is the number of breaking releases divided by the total number of releases, subtracted from one. The basic formula for the stability score is thus: 1 - (BR / TR), where BR is Breaking Releases and TR is Total Releases.
For example, if a package has 10 releases, and one of them is a breaking release, the Stability Score will be .9 or 90%.
Breaking releases are weighted over time. This consideration reflects the perception that breaking releases many years ago matter less than recent breaking releases. Breaking releases therefore start with a value of one (1), and that value is reduced by .01 for each month away from the present date. That means a breaking release in the last month will count as 1, and a breaking release two years ago will count as 1 - (.01 * 24) = 0.76.
Observe that this weighting over time will eventually cause the breaking release to not be counted at all. The discounting of a breaking release occurs at 8.333 years. That means breaking releases older than 8.333 years will not be considered in the Stability Score.
Also note that the number of releases eventually age out of the calculation. Any release older than 10 years from the current date will be discarded from the Stability Score. These releases will still be shown in the Stability Data. Keeping them in the data helps to give you an accurate understanding of the package history.
In addition to the above Stability Score, the
pkg_stability() function also provides a Stability
Assessment. The Stability Assessment is a
categorization of the Stability Score. The Stability Assessment is a way
to give you an English interpretation of the Stability Score. The
Stability Assessment is determined as follows:
The above categorization sets a high bar. The purpose of setting a high bar is simply to bring the R Open Source language up to the stability level of proprietary languages. It was designed to promote stability and encourage package developers to reduce the number of breaking changes in their Open Source packages. A high stability score is something that R package developers should be proud of, and R users should look for when selecting packages to include in their programs.
The Stability Score and Stability Assessment only work with packages published on CRAN. The reason is because CRAN preserves all releases of a package, and those archived releases are easily available to pkgdiff. Releases on Github are not reliable, as they are subject to whether the package developer remembers to create a release version or not.