# The State of CRA-Relevant Evidence in Public Software Repositories - 2026 ## Research boundary This research does not determine whether the CRA applies to any project and does not assess legal compliance. It measures only CRA-relevant evidence observable in public repository material. Question: How much of the evidence a manufacturer may need is already observable in a sample of popular, active software repositories? Publisher and collector: ConformOps, a commercial CRA evidence-management software provider. This is a publisher-run, automated descriptive benchmark, not an independent certification, representative readiness survey, or manually validated policy audit. No repository receives a compliance score. No AI classifies the observations. ## Population and selection, fixed before observation 300 distinct public GitHub repositories, 50 per stratum: .NET (GitHub primary language C#), JavaScript/TypeScript, Python, Java/JVM (Java or Kotlin), Go and Rust. Language is a sampling proxy: C# is not all .NET, Java/Kotlin is not all JVM, and repositories may contain multiple languages. Each repository has at least 500 stars, is not a fork, is not archived, has a push in the preceding calendar year, and has at least one published stable GitHub release. Prerelease-only projects are excluded, a stricter rule than simply having any release. Push activity can concern a branch other than the sampled default branch. For each language, fetch the first 100 results of GitHub repository search ordered by descending stars. Combine and deduplicate languages within each stratum. Sort candidates by the hexadecimal SHA-256 of `cra-evidence-2026:` and take the first 50 eligible entries. This deterministic ordering reduces discretionary picking within a popularity-biased frame; it is not random sampling from all software or all eligible GitHub repositories. Star-sort ties and the live search index can change the frame. The published candidate list is authoritative for this edition. Screen repository name and description for documentation/list/course signals using the exact exclusion expression in the collector. Before observation, a targeted purpose review removed six obvious teaching/resource collections missed by the English expression: rust-lang/rustlings, halfrost/LeetCode-Go, krahets/hello-algo, datawhalechina/hello-agents, Zie619/n8n-workflows and bannedbook/fanqiang. `sample.json` records this review; `PURPOSE_EXCLUSIONS` in the collector preserves it. Replacements followed the original candidate ordering. This is not a comprehensive manual project-purpose classification; software frameworks, development tools and templates remain eligible. Publish every candidate and its disposition, including candidates not evaluated once the quota was filled. No replacement based on evidence results is allowed. `sample.json` retains query strings, search totals, retrieval times, selected commit/tree hashes and stable-release metadata. `sample-freeze.json` binds the sample, repository CSV and candidate CSV by SHA-256. Freeze and commit these files before running `scan`. This provides auditable ordering within the PR, not an independently registered preregistration. Default-branch commits are observed, not release-tag source trees. The selected release reference establishes a release record, not an attestation that that release was built from the sampled commit. ## Collection scope Read GitHub's recursive tree at each frozen tree hash. Inspect regular blob paths; do not follow symlinks or submodules. Exclude path segments named test/tests, testdata, fixture/fixtures, sample/samples, example/examples, vendor/vendored, third_party/third-party/thirdparty, node_modules and .git. This reduces fixture and bundled-dependency matches but can omit genuine evidence. Root-level JSON examples can still look like SBOMs. Path metrics inspect the entire returned, filtered tree. Text metrics inspect at most 30 candidate documents: Markdown, reStructuredText, AsciiDoc or text files whose basename starts with readme, security, support, releas, maintain, contribut, governance, architecture, design, vulnerability, disclosure or lifecycle, plus GitHub Actions YAML. Rank SECURITY.md first, then shallower paths, then English locale path order. Inspect at most 20 named SBOM JSON candidates, separately, in path order. Each fetched document is limited to 256,000 bytes. These bounds are part of the measurement, not an assertion that other material is absent. Read files from raw.githubusercontent.com at the frozen commit. Never execute repository code. Source contents, contact addresses and matched excerpts are not saved; retain file URLs, Git blob identifiers, SHA-256 digests and match line numbers. Raw content exists only in collector memory. No product account, production binding, model API or analytics sink participates. Rate and availability errors remain unknown observations. Out of scope: release assets and release bodies, package registries, repository wikis, external documentation sites, inherited organization SECURITY.md files, authenticated advisories, CI run outcomes, private evidence, generated dependency graphs, non-English equivalents of English text rules, and source code semantics. A CI configuration is not evidence that a build passed. A security-tool mention is not proof the tool ran. SECURITY.md nested under a non-excluded directory counts; it need not be GitHub's effective security policy. ## Metric definitions All expressions are exported in `evidence-map-version.json` and implemented in `scripts/cra-benchmark.mjs`. | Metric | Positive means | Does not establish | | --- | --- | --- | | Dependency manifest | Recognized manifest filename in filtered tree | Valid parse, complete dependency inventory or shipped dependencies | | Lockfile | Recognized lockfile filename in filtered tree | Reproducible build, integrity or correct resolved versions; go.sum is intentionally not a lockfile | | Repository SBOM | Named JSON candidate declares CycloneDX with a version and nonempty components, or SPDX 2.x with an SPDXID and nonempty packages | Schema validity, completeness, freshness, manufacturer supply, release binding or a shipped SBOM | | CycloneDX / SPDX | Respective structured JSON discriminator above | XML, SPDX tag-value, SPDX 3 JSON-LD or other unnamed JSON SBOMs; the formats may overlap per repository | | SECURITY.md | Case-insensitive SECURITY.md basename | A substantive or operative security policy | | Reporting contact | Reporting/security/vulnerability context near an @, HTTPS URL or advisories reference | Verified delivery channel, a usable contact or private reporting | | Disclosure process | Coordinated/responsible disclosure or disclosure/vulnerability policy/process or triage/acknowledgement/embargo language | A complete, actionable disclosure process | | Security updates | Security update/patch/fix language near release/distribute/publish/upgrade/install/backport language | Actual delivery, support commitments or user notification | | Support / EOL | End of life, supported versions, support policy/period/window or long-term support phrase | Product-specific lifecycle commitments; it may concern a dependency | | Release process | Release/releasing process/workflow/procedure/checklist or how-to/steps to create/publish/cut a release | Execution of the procedure | | CI / build | Recognized CI configuration filename | Successful or reproducible builds | | Architecture / design | Architecture, design or architectural-overview document basename | Adequate or current architecture documentation | | Security testing | Named security tool or security scan/test or dependency-review phrase | Executed tests, test quality or results | | Release reference | Stable release metadata returned during selection | A metric independent of selection, signed provenance, or lineage to the sampled commit | Text metrics are deliberately reported as automated **signals**, not counts of repositories with validated policies. Positive lexical matches can concern examples, aspirations or dependencies. Absence of a match can reflect different wording, languages, filenames, external documents or private records. No precision/recall estimate is claimed because no manually labelled validation set was collected. Use source links to adjudicate individual observations before making stronger claims. ## Version history The sample and initial collector were committed before observation in commit 8d8c657. Independent pre-publication review identified cross-metric failure propagation, inconsistent coverage columns on tree failure and a completion timestamp based on repository start time. Collector version repository-evidence@2 fixes these and recollects all 300 pinned repositories. No version 1 results were published and the frozen sample is unchanged. ## States, aggregation and interpretation One row per repository per metric, with `observed`, `not_detected` or `unknown`. Observed takes precedence when positive evidence exists despite incomplete collection. A truncated/unavailable tree makes nonpositive metrics unknown. Text fetch failures, oversize documents or the 30-document cap make nonpositive text metrics unknown. Failed/oversized SBOM candidates or the 20-candidate cap similarly affect nonpositive SBOM metrics. `not_detected` means no rule matched in the defined corpus; it never means the project lacks the evidence. Percentages use all 300 sampled repositories, or all 50 in the stratum, as the denominator, including unknowns. Display observed counts, denominator and unknown counts together. Unknowns are not classified as negative findings. No ecosystem-weighted population estimate, ranking of readiness, statistical significance or confidence interval is justified by this non-probability sample. The 100% stable-release result follows from eligibility and is labelled as such. ## Reproduction and data dictionary Use Node.js 22.23.2 or later and GitHub CLI authenticated for public read access, or provide `GH_TOKEN` through the environment. Do not put a token in command arguments. The collector uses only built-in Node modules and read-only HTTP endpoints. To reproduce the evidence run, copy `sample.json`, `sample-freeze.json`, `repositories.csv` and `candidates.csv` into a fresh output directory, then run: ```sh node scripts/cra-benchmark.mjs scan /path/to/fresh-output node --test tests/cra-benchmark.test.mjs ``` The checked-in collector is linked from the research page. The map records its SHA-256. Use that exact collector revision and the published frozen inputs. Re-fetching at pinned commits reproduces detections while content remains public and available; timestamps and availability outcomes can differ. GitHub metadata at selection time cannot be recreated from a later live query, so the frozen records are the selection evidence. For a new sample, run `node scripts/cra-benchmark.mjs freeze /path/to/new-edition`, publish and commit that sample before `scan`. Never overwrite this edition with a new sample. The scan resumes from local `work/cra-benchmark` checkpoints keyed by output path, sample digest and collector digest. Use a fresh output directory for a fresh run. No checkpoint contains source text. - `repositories.csv`: selected repository, stratum, stars, primary language, push time, flags, branch, selection rank, commit/tree hashes, commit time, stable release ID/tag/time/URL and selection time. - `candidates.csv`: candidate frame and `selected`, `quota_filled`, `excluded_nonsoftware_description` or `excluded_no_stable_release` disposition. Quota-filled candidates were not fully evaluated. - `observations.csv`: repository, stratum, commit, metric, state, immutable file link (or selection-time release URL), Git blob hash, raw-document SHA-256 where fetched, match line, bounded reason, collection time and map version. Blank evidence fields mean no positive evidence was retained. CSV is RFC-style quoted UTF-8; import columns as text. - `coverage.csv`: tree completeness, candidate counts, successful/failed reads and selected document counts. `documents_read` includes both text and SBOM candidates. - `results.json`: aggregate counts, collection interval and sample/observation hashes. Percentages are derived at display time, not rounded in the dataset. - `evidence-map-version.json`: versioned rules, exclusions, limits and collector digest. This research map is separate from the product's deterministic legal/assessment engine. ConformOps releases its original observation dataset and methodology under CC BY 4.0: https://creativecommons.org/licenses/by/4.0/. Upstream repository content is not redistributed and retains its own licensing. Citation: ConformOps (2026), The State of CRA-Relevant Evidence in Public Software Repositories - 2026, repository-evidence@2, https://conformops.eu/research/cra-repository-evidence-benchmark-2026. Include the sample SHA-256 and access date when citing a specific dataset. Corrections: open an issue in https://github.com/Bauda/ConformOps with the repository, metric and pinned evidence URL. Corrections require a new map/results version and an explanation; do not silently replace observations or backfill the sample. ## Context and sources - Regulation (EU) 2024/2847, especially Article 2(12), Annex I Part II and Annex VII: https://eur-lex.europa.eu/eli/reg/2024/2847/oj/eng . These motivate evidence categories, not legal judgments about sampled projects. Non-commercial free and open-source software has specific treatment; the benchmark does not classify a project's economic activity or legal role. - OpenSSF / Linux Foundation, 2026 CRA Awareness and Readiness Report: https://openssf.org/wp-content/uploads/2026/06/2026_CRA_Awareness_Readiness_Report.pdf . A survey of awareness/readiness answers a different question from observable repository material. - ENISA, SBOM Adoption State of Play - 2026: https://www.enisa.europa.eu/publications/sbom-adoption-state-of-play-2026 . Organizational adoption is not repository-visible SBOM presence. - GitHub REST tree API and truncation behavior: https://docs.github.com/en/rest/git/trees . The collector keeps truncation visible rather than treating a partial tree as complete.