Skip to content

New: quality ratings for devices and their projects. Please share any feedback here.

altstrument

About / Quality

How quality ratings work

What ratings are for

Some devices have hundreds of entries, and a flat list does not say which ones are worth trying first. A quality rating is a short hint about a project's standing: whether others build on it, whether it is mature and widely used, or whether it is new and picking up. It is a hint, not a verdict.

Ratings are built not to punish old, stable foundations: a project that was finished years ago and still works keeps its standing (see the finished-project rule).

The ratings

Behind every rating is an internal score from 0 to 100, built from six pillars (below). The score is never shown. It orders lists when you sort by Quality, and it sets the bar: entries scoring under 48, or without GitHub data, get no rating, because there is not enough to go on yet. The one exception is the device maker's own open firmware and SDKs, which are Foundational regardless. Above the bar, an entry gets the first rating whose rule it meets:

Foundational: Other projects build on it, or it is the maker's own open firmware or SDK.
At least 3 catalog entries derive from it, or its influence pillar is at least 60%. The device maker's own open-source firmware, and its SDKs and libraries (tagged "sdk" or "library"), are always Foundational, even without enough data for a score.
Established: Mature and widely used.
Maturity of at least 50% and popularity of at least 40%. Quiet, finished projects qualify.
Rising: Under a year old, with real traction.
Less than a year old, and above the bar.
Notable: Stands out from most entries, without a clearer category.
Above the bar, but none of the above.

Ratings sit beside an entry's status (Active, Maintained, Dormant), which says how recently it changed. A rating says how it stands.

The six pillars

Each pillar is a number from 0 to 1, shown on entry pages as Strong, Moderate or Low. The internal score is the weighted average of five of them, times 100. Influence is added on top as a bonus of up to 15points, because most projects have nothing built on them yet and should not be marked down for it.

  • Maturity weight 25 of 100

    How long and how much the project has been built.

    • Age, capped at 10 years (weight 2)
    • Releases, capped at 20 (1.5)
    • Commits, capped at 600 (1.5)
    • Newest release looks stable, 1.0 or later (0.5)
  • Popularity weight 25 of 100

    How many people have noticed it.

    • Stars, capped at 500 (2)
    • Forks, with active forks (starred, or worked on after forking) counted twice, capped at 100 (1)
    • Watchers, capped at 30 (1)
  • Vitality weight 20 of 100

    Whether it is still being worked on, unless it is finished.

    • Commits in the last year, capped at 300 (1)
    • Recency of the newest commit or release, falling to 0 over 24 months (1)
  • Issue health weight 10 of 100

    Whether reports and contributions get handled.

    • Share of issues closed, smoothed (0.75)
    • Share of pull requests merged, smoothed (0.25)
  • Influence bonus, up to 15 points

    Whether other projects in the catalog build on it.

    • Catalog entries built on this one, by the sum of their own scores, capped at 3
  • Community weight 20 of 100

    How many people could keep it going.

    • Contributors, capped at 15 (2)
    • Bus factor, capped at 3 (1)
    • Contributors and bus factor count 80%; a bonus of up to 30% for GitHub Discussions (capped at 50 threads) or a community link on the entry

Log-cap normalisation

Raw counts are squashed so that the first stars, commits or contributors matter more than the thousandth. For a signal S with a cap T:

norm(S, T) = log(1 + S) / log(1 + max(S, T))

Zero scores 0, and anything at or above the cap scores 1. The caps sit near the 95th percentile of this catalog, so a project does not need to be a giant to score well. They are listed with each pillar above. This is the same idea as the OpenSSF Criticality Score.

The finished-project rule

Quiet is not the same as dead. Many firmware projects are done: the code works, people still use it and build on it, and nobody needs to commit. A project gets the finished-project rule when it is not archived, is at least3 years old, and either its maturity or its adoption (the larger of popularity and influence) is at least 50%. Its vitality is then raised to 70% of the larger of those two, if that is higher than its real activity. Entry pages mark these as "Finished: quiet but established". Archived projects always get 0 vitality.

Smoothing of issue and PR ratios

A repository with one closed issue should not outrank one with 200 closed and 20 open. Ratios are smoothed toward a neutral prior of 0.6:

ratio = (n·x + 10·0.6) / (n + 10)

Here x is the closed (or merged) share and n the number of issues (or PRs). With none at all, the neutral 0.6 is used.

Influence

Being built on is a strong signal. Influence counts only other projects in the catalog that build on this one, either as a GitHub fork or through the entry's based_on field. Each counts by its own score (without its own influence), so one level deep only, and the sum is capped. Plain GitHub forks are not influence; they count toward popularity, where forks with stars or real work after forking count twice.

First-party bonus

Projects from the people who make the device get 10 extra points on the internal score, capped at 100. An entry counts as first-party when it is tagged official or one of its authors is the device's manufacturer. Official source and SDKs are what most other projects build on, and they often live in quiet repos that understate them.

Device ratings

A device page sums up its entries in a collapsible Quality block: how many have each rating, and which ones. A device whose entries are all unrated shows no block. Entries whose rating is hidden are left out.

Device-level ratings and overrides

Some devices keep all their programs in one repository, with one community around it. GitHub numbers for that repo say nothing about one folder inside it, so such a device can name its shared_repo: every entry whose source is in it shares one rating, worked out for the repo itself plus the device's community links. The device page shows that rating once, in its Quality block, instead of on every card, and those entries keep their recently-updated order.

When the numbers mislead, a rating can be set by hand with quality.rating, or the internal score withquality.score and a reason, on an entry or for a whole device. The entry page shows the reason. Where a rating would say nothing useful, quality.hide on an entry or a device shows none; those entries still sort by their score.

Entries without GitHub data

Entries without a GitHub repository, or not yet in the snapshot, have no popularity, issue or contributor data. They get no rating. For sorting, each missing pillar takes the median of the entries that do have it, so their score sits near the middle on those pillars instead of at zero.

Where the numbers come from

Ratings come from a snapshot of GitHub data, not live data. It is refreshed once a week, and new entries are added to it when they are merged (the last full refresh was on 2026-11-16). Time-based signals are measured from the date each project was last fetched, so a rating only changes when the data does.

What it does not measure

  • Sound, features or how good the firmware is to use.
  • Safety. A rating says nothing about brick risk; read the entry's warnings.
  • Projects hosted outside GitHub (GitLab, forums, personal sites), which get no rating.

Prior art