Course: Course 3 — LLM Fine-Tuning Masterclass Module: FT02 — The Open Spectrum: Weights, Data, and Trust Duration: 60 minutes Level: Senior Engineer and above Prerequisites: FT00 (The Steering Stack); FT01 (VRAM Math) recommended
After completing this module, you will be able to:
Layer 1 of the Steering Stack has a property that no other layer has: its openness is decided by someone else, before you ever touch it. Get this wrong in a regulated domain and you fail the audit before you finish the architecture.
When a lab releases a "model," what you actually receive varies enormously. Three tiers matter.
You get the weights — the trained tensors — plus an architecture spec and usually a license that permits use and some modification. You do not get the training data, and you usually do not get the full training code and configuration. The canonical example is Meta's Llama 3.x family: the weights ship under the Llama Community License (with its 700M-MAU and named-product restrictions), the pretraining corpus is described only in aggregate ("~15T tokens, multilingual, code, reasoning"), and the post-training recipe is summarized at a high level but not reproducibly specified.
You can run it. You can fine-tune it. You cannot audit what it saw, and you cannot reproduce it. For non-sensitive use this is often an acceptable trade — the capability is excellent, the license is permissive enough for most commercial work. For a HIPAA or air-gapped deployment where a regulator asks "prove to me what this model was trained on," it is a wall.
You get the weights and the training corpus — or, when the corpus is too large to ship verbatim, a documented, reproducible pipeline that names the exact source datasets, their proportions, the deduplication and filtering applied, and (in the best cases) the data itself. The OpenBMB MiniCPM family (Tsinghua + ModelBest), Allen Institute's OLMo 2 / Tülu 3, and HuggingFace's SmolLM3 occupy this tier. You can, in principle, rebuild the pretraining mix from their published components.
The defining property: you can prove what the model saw. That is the auditability predicate. It is also the reproducibility predicate — given the same data, seed, and code, you can expect to land in the same neighborhood (modulo framework nondeterminism, which is its own rabbit hole).
The strictest tier. Weights plus data plus the complete training code and configuration: optimizer state, hyperparameter schedules, the exact data-curation pipeline, eval harness, and (in the best cases) intermediate checkpoints. OLMo, Tülu 3, and SmolLM3 all ship this. The OLMo 2 paper (Olson et al., arXiv:2501.00656) explicitly describes "fully open language models" — data, code, weights, and intermediate checkpoints, by design. Tülu 3 (Lambert et al., arXiv:2411.15124) releases the full post-training recipe. SmolLM3's release blog documents "the full training recipe including pre-training, mid-training, post-training, and synthetic data."
The defining property: a skilled person can rebuild the model from scratch and a security team can prove no training-time exfiltration occurred. This is what IL5/IL6, HIPAA-covered, and air-gapped environments increasingly require. If the recipe is closed, you cannot rule out a hidden data pathway; if the recipe is open, the data lineage is auditable end to end.
The one-sentence summary. Open-weights lets you use the model. Open-data lets you audit what it saw. Open-recipe lets you reproduce and prove the whole thing. Each tier adds a trust property the one below lacks.
On 2024-10-28 the Open Source Initiative released v1.0 of the Open Source AI Definition at All Things Open. It is a deliberate compromise — and the compromise is exactly the gap you must understand.
The OSAID v1.0 requires, for a system to be called "open source AI," three things: the complete source code for training and running the system; "sufficiently detailed information" about the data used to train the system so that a skilled person can build a substantially equivalent system; and the usual freedoms to use, modify, and redistribute.
Read the data clause carefully. It requires sufficiently detailed information about the data. It does not require the data itself, and it does not require that data be openly licensed. This is deliberate. The OSI concluded that requiring full corpus release was impractical — much training data is private, licensed under terms that forbid redistribution, or contains personal information that cannot legally be republished. So the definition permits a model trained on undisclosed data to call itself "open source" as long as enough metadata (sources, composition, processing) is published for a "skilled person" to aim at a "substantially equivalent" result.
This is the OSAID gap, and it is the most important thing to understand about the definition.
The gap has two consequences. First, a Llama-style release — weights, code, and a high-level data description, but no actual corpus — can claim OSAID compliance without being reproducible in any rigorous sense. "Substantially equivalent" is not "identical," and nobody has defined the tolerance band. Second, OSAID compliance is not the same as auditability. If a regulator asks you to prove that a specific document was not in the training set, "we published a sufficiently detailed description of our data sources" is not a satisfying answer. The OSAID was engineered to bring the largest publishers into the tent; the cost of doing so was collapsing the data requirement to a description.
Critics — notably the Software Freedom Conservancy and senior figures at Hugging Face — argue this erodes the OSD's "no hidden inputs" principle (OSD #1 and #2). Defenders argue it is the best achievable definition and that a stricter bar would have excluded every frontier-scale release. Both are partly right. For our purposes, the practical takeaway is fixed:
OSAID compliance tells you the release is inspectable. It does not tell you the release is reproducible or auditable end to end. Treat "OSAID-compliant" and "open-data / open-recipe" as different claims, and never conflate them.
The single best government citation for the sensitive-data argument. Memorize the source; you will cite it in procurement documents.
In July 2024 the U.S. National Telecommunications and Information Administration published Dual-Use Foundation Models with Widely Available Model Weights — the congressionally-mandated report on open-weight AI. Among its findings, the NTIA states plainly that open-weight models "provide security benefits by allowing firms, researchers, and users to use potentially sensitive data" locally and on-premises, without routing that data through a third-party API.
That sentence is load-bearing. It is a federal agency, in an official report mandated under Executive Order 14110, explicitly endorsing the on-premises / air-gapped deployment pattern as a security benefit of openness — not merely an economic or convenience benefit. The reasoning is mechanical: if the weights are available, the model can run inside your trust boundary. If they are not, every inference request leaves your boundary, traverses a vendor's infrastructure, and depends on that vendor's data-handling, logging, and retention commitments.
This is why IL5 / IL6 (U.S. government Impact Levels), HIPAA-covered workloads, and air-gapped environments increasingly require an open-weight base. You cannot put a proprietary API-only model inside a SCIF. You can put an open-weights model there — and if that model is also open-data / open-recipe, you can additionally answer the procurement officer's question: "where did these weights come from, and what did they see?"
The NTIA report does not claim open weights are risk-free (it catalogs the dual-use risks at length). Its claim is narrower and stronger: the set of deployment environments that can use a model at all expands when the weights are available, and a meaningful subset of those environments — precisely the sensitive ones — are gated on openness. That is the argument this module exists to give you a citation for. (Module FT22 develops the air-gapped deployment fully.)
The Stanford CRFM's Foundation Model Transparency Index turns "openness" from a slogan into a number. The number is damning for closed releases.
The FMTI (Bommasani et al., arXiv:2407.12929; Hugging Face / Stanford CRFM, May 2024 edition) scores foundation-model developers on 100 indicators across domains including data, labor, hardware, usage, and downstream monitoring. The May 2024 results: the mean score was 58 / 100, the top score was 85 / 100, and the gap between the most transparent (fully-open) and least transparent (closed) developers was wide and structural — fully-open developers like the OLMo / Ai2 stack and IBM Granite cluster at the top, while closed developers like OpenAI (then ~49) and Anthropic (~51) sit below the mean.
Two readings matter for this course.
First, transparency is measurable and tiered, not a binary. The FMTI gives you a defensible, citable metric to compare candidate bases — which matters when a procurement RFP asks you to justify model choice on transparency grounds.
Second, the index tracks the openness tier cleanly. Open-recipe developers score highest because they publish data composition, data processing code, eval results, and usage policies; weights-only developers score lower because they publish none of the data-level indicators; closed developers score lowest of all because they publish almost nothing the FMTI measures. The tier you choose is, roughly, the score you will be able to defend.
The FMTI is the bridge from "openness is nice" to "openness is a procurement criterion." When a stakeholder asks "how do you know open-data is more transparent," you point at the index. (The 2025 edition added nuance — some transparency trends began to reverse — but the tier-correlation held.)
Eight releases, three tiers. This is the table to memorize.
| Release | Org | Tier | License | Data released? | Recipe released? | Auditable? |
|---|---|---|---|---|---|---|
| MiniCPM5-1B / 3-4B / V 4.6 / o 4.5 | OpenBMB (Tsinghua + ModelBest) | Open-data / recipe | Apache-2.0 | Yes (UltraChat, UltraFeedback, Ultra-FineWeb) | Yes | Yes |
| OLMo 2 (7B/13B/32B) | Allen Institute for AI (Ai2) | Open-recipe | Apache-2.0 | Yes | Yes (code, eval, checkpoints) | Yes |
| Tülu 3 (post-training) | Allen Institute for AI (Ai2) | Open-recipe | Apache-2.0 / ODC-BY (data) | Yes | Yes (full post-training recipe) | Yes |
| SmolLM3 (3B) | Hugging Face | Open-recipe | Apache-2.0 | Yes | Yes (pre/mid/post + synthetic data) | Yes |
| Nemotron (NVIDIA) | NVIDIA | Open-weights / partial | NVIDIA Open Model License | Partial (post-training data documented) | Partial | Partial |
| DCLM | DCLM consortium (DataComp-LM) | Open-data | Open (varies by component) | Yes (the point of the project) | Yes (data pipeline) | Yes |
| Llama 3.1 (incl. 405B) | Meta | Open-weights-only | Llama 3.1 Community License | No (described in aggregate) | No (summarized) | No |
| GPT-4o | OpenAI | Closed | Proprietary (API + weights withheld) | No | No | No |
Read the right-hand columns. "Auditable?" is the question this course cares about, because it is the question a HIPAA security officer, a FedRAMP assessor, or an IL6 authorizing official will eventually ask. The answer divides cleanly: open-recipe and open-data families are auditable; weights-only and closed families are not. Capability does not appear in this table on purpose — capability is FT03's concern. This module is about whether you can trust the provenance.
Why this course uses MiniCPM as the load-bearing example across modules.
OpenBMB ("Open Lab for Big Model Base") is a collaboration between Tsinghua University and the company ModelBest. Its MiniCPM family is the on-ramp this course returns to, because it sits in a sweet spot: genuinely open-data / open-recipe under Apache-2.0, small enough to run and fine-tune on consumer hardware, and accompanied by a stack of open datasets that are themselves reusable building blocks.
The MiniCPM family includes:
OpenBMB's associated open datasets — UltraChat (large-scale dialogue), UltraFeedback (preference / feedback), and Ultra-FineWeb (curated web pretraining mix) — are referenced across the data modules (Pillar 1) precisely because they are open, documented, and reusable. When this course says "load an open-data base," MiniCPM5-1B is the default. When it says "fine-tune on an open preference dataset," UltraFeedback is the default. The MiniCPM stack is the course's worked example of an auditable base — you can point at every byte the model saw.
Three properties that only open-data / open-recipe releases give you, each of which becomes a compliance requirement in regulated deployment.
1. Auditability — you can prove what the model saw. A HIPAA security officer conducting a risk analysis must be able to identify the sources of the model's knowledge. With an open-data base, you produce the corpus manifest. With a weights-only base, you produce the publisher's marketing description and a shrug. The NTIA report's "security benefits" sentence is the authority; the FMTI is the metric; open-data is the precondition.
2. Reproducibility — you can rebuild years later and prove no drift. A model deployed in a clinical system may run for years. If the vendor silently updates a closed model, behavior changes underneath your validated pipeline — a phenomenon called silent drift — and your validation no longer holds. With an open-recipe base, you pin the exact commit of data + code + weights and rebuild on demand. You can demonstrate to an auditor that the model you validated is the model you are running. You cannot do this with a closed API.
3. Supply-chain trust — you can rule out hidden training-time exfiltration. A weights-only or closed base is a black box at the moment that matters most: training time. You cannot prove the publisher did not fold sensitive or malicious data into the corpus. With an open-recipe release, the data lineage is auditable end to end — a security team can inspect the data pipeline and (in the air-gapped case) rebuild the model from audited inputs inside the trust boundary. This is the property IL5/IL6 and air-gapped environments actually require, and it is why this module sits directly upstream of FT22 (Air-gapped / on-prem deployment).
The thread, stated as one claim: Open-data / open-recipe releases are a compliance asset in sensitive domains, not merely an ideological preference. Auditability, reproducibility, and supply-chain trust each map to a requirement a regulator can name — and each is satisfiable only when you can see the data and the recipe.
The Hugging Face Hub is full of "merged" models — A merged with B, quantized, re-uploaded. Many have no documented data lineage, no evals, and a license they probably do not have the rights to grant. A model card that says "merge of X and Y, works great" is not an audit. In any sensitive deployment, trace the provenance back to the original bases and the original data. If you cannot, treat it as unauditable — because it is.
OSAID compliance is a real and useful signal — it means the release is inspectable and the publisher made a good-faith effort. It does not mean the release is reproducible, because the data requirement is "sufficiently detailed information," not "the data." A vendor that points at OSAID compliance as proof of auditability is conflating two different things. Ask for the data and the recipe; if they cannot provide it, the auditability claim does not survive.
Open data is auditable, not safe. An open corpus can still contain PII, copyrighted text, poisoned examples, or material that creates its own liability. Openness gives you the ability to vet; it does not do the vetting for you. In a sensitive deployment, the open-data base still needs a data audit (FT04–FT07), a PII sweep, a licensing review, and the same red-teaming you would apply to any other base. "It's open, so it's fine" is the open-source version of the cardinal error.
| Term | Definition |
|---|---|
| Open-weights-only | Tier 1; weights shipped, no training data (Llama 3.x). Usable, not auditable. |
| Open-data | Tier 2; weights + training corpus or a reproducible data pipeline (MiniCPM, OLMo, Tülu, SmolLM3). Auditable. |
| Open-recipe | Tier 3; weights + data + full training code/config (OLMo, Tülu 3, SmolLM3). Reproducible. |
| OSAID v1.0 | OSI's Open Source AI Definition (2024-10-28). Requires "sufficiently detailed information" about data, not the data itself. |
| The OSAID gap | The deliberate distance between OSAID compliance and actual reproducibility/auditability. |
| NTIA Open-Model Weights Report | July 2024 federal report; states open weights provide security benefits for sensitive-data use on-prem. |
| FMTI | Stanford Foundation Model Transparency Index; 100 indicators; May-2024 mean 58/100, top 85/100. |
| OpenBMB | Tsinghua + ModelBest; the MiniCPM family + Ultra* datasets; Apache-2.0; the course's on-ramp hero. |
| MiniCPM5-1B | OpenBMB's ~1B dense open-data base; the Layer-1 example used throughout the course. |
| Auditability | The property that you can prove what a model saw — satisfiable only with open-data / open-recipe. |
See 07-lab-spec.md. "The Openness Audit" — take five releases (MiniCPM5-1B, OLMo-2, Llama-3.1-405B, SmolLM3, GPT-4o), classify each on the open spectrum (tier, license, data released, recipe released, auditable), and write the one-paragraph HIPAA-deployment verdict using the structured audit template. No GPU required — this is a research/classification lab (~30–40 min).
# Module FT02 — The Open Spectrum: Weights, Data, and Trust
**Course**: Course 3 — LLM Fine-Tuning Masterclass
**Module**: FT02 — The Open Spectrum: Weights, Data, and Trust
**Duration**: 60 minutes
**Level**: Senior Engineer and above
**Prerequisites**: FT00 (The Steering Stack); FT01 (VRAM Math) recommended
---
## Learning Objectives
After completing this module, you will be able to:
1. Distinguish the three openness tiers — **open-weights-only**, **open-data**, and **open-recipe** — and classify any model release into the correct tier.
2. Explain what the **OSI OSAID v1.0** (released 2024-10-28) requires and what it deliberately does *not* require, and why this is a compromise that lets non-reproducible releases claim "open source."
3. Cite the **NTIA 2024 Open-Model Weights Report** as the government authority for the claim that open-weight models "provide security benefits by allowing firms, researchers, and users to use potentially sensitive data" locally and on-premises.
4. Use the **Stanford FMTI** (Foundation Model Transparency Index) to quantify the transparency gap between closed/weights-only releases and fully-open ones.
5. Defend, for a sensitive-domain deployment (HIPAA, IL5/IL6, air-gapped), *why* open-data is a precondition for auditability, reproducibility, and supply-chain trust — and write the one-paragraph "can I audit this?" verdict for any model.
---
# 2.1 — The Three Tiers of Openness
*Layer 1 of the Steering Stack has a property that no other layer has: its openness is decided by someone else, before you ever touch it. Get this wrong in a regulated domain and you fail the audit before you finish the architecture.*
When a lab releases a "model," what you actually receive varies enormously. Three tiers matter.
## Tier 1 — Open-weights-only
You get the **weights** — the trained tensors — plus an architecture spec and usually a license that permits use and some modification. You do **not** get the training data, and you usually do not get the full training code and configuration. The canonical example is **Meta's Llama 3.x family**: the weights ship under the Llama Community License (with its 700M-MAU and named-product restrictions), the pretraining corpus is described only in aggregate ("~15T tokens, multilingual, code, reasoning"), and the post-training recipe is summarized at a high level but not reproducibly specified.
You can run it. You can fine-tune it. You **cannot** audit what it saw, and you **cannot** reproduce it. For non-sensitive use this is often an acceptable trade — the capability is excellent, the license is permissive enough for most commercial work. For a HIPAA or air-gapped deployment where a regulator asks "prove to me what this model was trained on," it is a wall.
## Tier 2 — Open-data
You get the weights **and** the training corpus — or, when the corpus is too large to ship verbatim, a documented, reproducible pipeline that names the exact source datasets, their proportions, the deduplication and filtering applied, and (in the best cases) the data itself. The **OpenBMB MiniCPM** family (Tsinghua + ModelBest), **Allen Institute's OLMo 2 / Tülu 3**, and **HuggingFace's SmolLM3** occupy this tier. You can, in principle, rebuild the pretraining mix from their published components.
The defining property: **you can prove what the model saw.** That is the auditability predicate. It is also the reproducibility predicate — given the same data, seed, and code, you can expect to land in the same neighborhood (modulo framework nondeterminism, which is its own rabbit hole).
## Tier 3 — Open-recipe
The strictest tier. Weights **plus** data **plus** the complete training code and configuration: optimizer state, hyperparameter schedules, the exact data-curation pipeline, eval harness, and (in the best cases) intermediate checkpoints. **OLMo**, **Tülu 3**, and **SmolLM3** all ship this. The OLMo 2 paper (Olson et al., arXiv:2501.00656) explicitly describes "fully open language models" — data, code, weights, and intermediate checkpoints, by design. Tülu 3 (Lambert et al., arXiv:2411.15124) releases the full post-training recipe. SmolLM3's release blog documents "the full training recipe including pre-training, mid-training, post-training, and synthetic data."
The defining property: **a skilled person can rebuild the model from scratch and a security team can prove no training-time exfiltration occurred.** This is what IL5/IL6, HIPAA-covered, and air-gapped environments increasingly require. If the recipe is closed, you cannot rule out a hidden data pathway; if the recipe is open, the data lineage is auditable end to end.
> **The one-sentence summary.** Open-weights lets you *use* the model. Open-data lets you *audit* what it saw. Open-recipe lets you *reproduce and prove* the whole thing. Each tier adds a trust property the one below lacks.
---
# 2.2 — The OSAID Gap: What "Open Source AI" Now Officially Means
*On 2024-10-28 the Open Source Initiative released v1.0 of the Open Source AI Definition at All Things Open. It is a deliberate compromise — and the compromise is exactly the gap you must understand.*
The OSAID v1.0 requires, for a system to be called "open source AI," three things: the **complete source code** for training and running the system; **"sufficiently detailed information" about the data used to train the system so that a skilled person can build a substantially equivalent system**; and the usual freedoms to use, modify, and redistribute.
Read the data clause carefully. It requires *sufficiently detailed information about* the data. It does **not** require the data itself, and it does **not** require that data be openly licensed. This is deliberate. The OSI concluded that requiring full corpus release was impractical — much training data is private, licensed under terms that forbid redistribution, or contains personal information that cannot legally be republished. So the definition permits a model trained on undisclosed data to call itself "open source" as long as enough metadata (sources, composition, processing) is published for a "skilled person" to aim at a "substantially equivalent" result.
This is the **OSAID gap**, and it is the most important thing to understand about the definition.
The gap has two consequences. First, a **Llama-style release** — weights, code, and a high-level data description, but no actual corpus — can claim OSAID compliance without being *reproducible* in any rigorous sense. "Substantially equivalent" is not "identical," and nobody has defined the tolerance band. Second, **OSAID compliance is not the same as auditability.** If a regulator asks you to prove that a specific document was *not* in the training set, "we published a sufficiently detailed description of our data sources" is not a satisfying answer. The OSAID was engineered to bring the largest publishers into the tent; the cost of doing so was collapsing the data requirement to a description.
Critics — notably the Software Freedom Conservancy and senior figures at Hugging Face — argue this erodes the OSD's "no hidden inputs" principle (OSD #1 and #2). Defenders argue it is the best achievable definition and that a stricter bar would have excluded every frontier-scale release. Both are partly right. For our purposes, the practical takeaway is fixed:
> **OSAID compliance tells you the release is inspectable. It does not tell you the release is reproducible or auditable end to end. Treat "OSAID-compliant" and "open-data / open-recipe" as different claims, and never conflate them.**
---
# 2.3 — The NTIA Argument: Why Open Weights Win for Sensitive Data
*The single best government citation for the sensitive-data argument. Memorize the source; you will cite it in procurement documents.*
In July 2024 the U.S. National Telecommunications and Information Administration published *Dual-Use Foundation Models with Widely Available Model Weights* — the congressionally-mandated report on open-weight AI. Among its findings, the NTIA states plainly that open-weight models **"provide security benefits by allowing firms, researchers, and users to use potentially sensitive data"** locally and on-premises, without routing that data through a third-party API.
That sentence is load-bearing. It is a federal agency, in an official report mandated under Executive Order 14110, explicitly endorsing the on-premises / air-gapped deployment pattern as a *security benefit* of openness — not merely an economic or convenience benefit. The reasoning is mechanical: if the weights are available, the model can run inside your trust boundary. If they are not, every inference request leaves your boundary, traverses a vendor's infrastructure, and depends on that vendor's data-handling, logging, and retention commitments.
This is why **IL5 / IL6 (U.S. government Impact Levels), HIPAA-covered workloads, and air-gapped environments increasingly *require* an open-weight base.** You cannot put a proprietary API-only model inside a SCIF. You *can* put an open-weights model there — and if that model is also open-data / open-recipe, you can additionally answer the procurement officer's question: "where did these weights come from, and what did they see?"
The NTIA report does not claim open weights are risk-free (it catalogs the dual-use risks at length). Its claim is narrower and stronger: *the set of deployment environments that can use a model at all expands when the weights are available, and a meaningful subset of those environments — precisely the sensitive ones — are gated on openness.* That is the argument this module exists to give you a citation for. (Module FT22 develops the air-gapped deployment fully.)
---
# 2.4 — The FMTI: Quantifying the Transparency Gap
*The Stanford CRFM's Foundation Model Transparency Index turns "openness" from a slogan into a number. The number is damning for closed releases.*
The FMTI (Bommasani et al., arXiv:2407.12929; Hugging Face / Stanford CRFM, May 2024 edition) scores foundation-model developers on **100 indicators** across domains including data, labor, hardware, usage, and downstream monitoring. The May 2024 results: the **mean score was 58 / 100**, the **top score was 85 / 100**, and the gap between the most transparent (fully-open) and least transparent (closed) developers was wide and structural — fully-open developers like the OLMo / Ai2 stack and IBM Granite cluster at the top, while closed developers like OpenAI (then ~49) and Anthropic (~51) sit below the mean.
Two readings matter for this course.
First, **transparency is measurable and tiered**, not a binary. The FMTI gives you a defensible, citable metric to compare candidate bases — which matters when a procurement RFP asks you to justify model choice on transparency grounds.
Second, the index **tracks the openness tier cleanly.** Open-recipe developers score highest because they publish data composition, data processing code, eval results, and usage policies; weights-only developers score lower because they publish none of the data-level indicators; closed developers score lowest of all because they publish almost nothing the FMTI measures. The tier you choose is, roughly, the score you will be able to defend.
> **The FMTI is the bridge from "openness is nice" to "openness is a procurement criterion."** When a stakeholder asks "how do you know open-data is more transparent," you point at the index. (The 2025 edition added nuance — some transparency trends began to reverse — but the tier-correlation held.)
---
# 2.5 — The Model Family Comparison
*Eight releases, three tiers. This is the table to memorize.*
| Release | Org | Tier | License | Data released? | Recipe released? | Auditable? |
| --- | --- | --- | --- | --- | --- | --- |
| **MiniCPM5-1B / 3-4B / V 4.6 / o 4.5** | OpenBMB (Tsinghua + ModelBest) | Open-data / recipe | Apache-2.0 | Yes (UltraChat, UltraFeedback, Ultra-FineWeb) | Yes | Yes |
| **OLMo 2 (7B/13B/32B)** | Allen Institute for AI (Ai2) | Open-recipe | Apache-2.0 | Yes | Yes (code, eval, checkpoints) | Yes |
| **Tülu 3 (post-training)** | Allen Institute for AI (Ai2) | Open-recipe | Apache-2.0 / ODC-BY (data) | Yes | Yes (full post-training recipe) | Yes |
| **SmolLM3 (3B)** | Hugging Face | Open-recipe | Apache-2.0 | Yes | Yes (pre/mid/post + synthetic data) | Yes |
| **Nemotron (NVIDIA)** | NVIDIA | Open-weights / partial | NVIDIA Open Model License | Partial (post-training data documented) | Partial | Partial |
| **DCLM** | DCLM consortium (DataComp-LM) | Open-data | Open (varies by component) | Yes (the point of the project) | Yes (data pipeline) | Yes |
| **Llama 3.1 (incl. 405B)** | Meta | Open-weights-only | Llama 3.1 Community License | No (described in aggregate) | No (summarized) | No |
| **GPT-4o** | OpenAI | Closed | Proprietary (API + weights withheld) | No | No | No |
Read the right-hand columns. "Auditable?" is the question this course cares about, because it is the question a HIPAA security officer, a FedRAMP assessor, or an IL6 authorizing official will eventually ask. The answer divides cleanly: open-recipe and open-data families are auditable; weights-only and closed families are not. Capability does not appear in this table on purpose — capability is FT03's concern. This module is about *whether you can trust the provenance.*
---
# 2.6 — OpenBMB and MiniCPM: The Course's On-Ramp Hero
*Why this course uses MiniCPM as the load-bearing example across modules.*
**OpenBMB** ("Open Lab for Big Model Base") is a collaboration between **Tsinghua University** and the company **ModelBest**. Its MiniCPM family is the on-ramp this course returns to, because it sits in a sweet spot: genuinely open-data / open-recipe under **Apache-2.0**, small enough to run and fine-tune on consumer hardware, and accompanied by a stack of open datasets that are themselves reusable building blocks.
The MiniCPM family includes:
- **MiniCPM5-1B** — the ~1B dense base used as the Layer-1 hero in FT00's lab and throughout the early modules. Runs anywhere; fine-tunes on a single consumer GPU.
- **MiniCPM3-4B** — a denser mid-size base for modules that need a bit more headroom.
- **MiniCPM-V 4.6** — a multimodal (vision-language) variant.
- **MiniCPM-o 4.5** — an omni-modal (audio + vision + text) variant.
OpenBMB's associated open datasets — **UltraChat** (large-scale dialogue), **UltraFeedback** (preference / feedback), and **Ultra-FineWeb** (curated web pretraining mix) — are referenced across the data modules (Pillar 1) precisely because they are open, documented, and reusable. When this course says "load an open-data base," MiniCPM5-1B is the default. When it says "fine-tune on an open preference dataset," UltraFeedback is the default. The MiniCPM stack is the course's worked example of an *auditable* base — you can point at every byte the model saw.
---
# 2.7 — Why This Matters for Sensitive Domains (the setup for FT21 / FT22)
*Three properties that only open-data / open-recipe releases give you, each of which becomes a compliance requirement in regulated deployment.*
**1. Auditability — you can prove what the model saw.** A HIPAA security officer conducting a risk analysis must be able to identify the sources of the model's knowledge. With an open-data base, you produce the corpus manifest. With a weights-only base, you produce the publisher's marketing description and a shrug. The NTIA report's "security benefits" sentence is the authority; the FMTI is the metric; open-data is the precondition.
**2. Reproducibility — you can rebuild years later and prove no drift.** A model deployed in a clinical system may run for years. If the vendor silently updates a closed model, behavior changes underneath your validated pipeline — a phenomenon called *silent drift* — and your validation no longer holds. With an open-recipe base, you pin the exact commit of data + code + weights and rebuild on demand. You can demonstrate to an auditor that the model you validated is the model you are running. You cannot do this with a closed API.
**3. Supply-chain trust — you can rule out hidden training-time exfiltration.** A weights-only or closed base is a black box at the moment that matters most: training time. You cannot prove the publisher did not fold sensitive or malicious data into the corpus. With an open-recipe release, the data lineage is auditable end to end — a security team can inspect the data pipeline and (in the air-gapped case) rebuild the model from audited inputs inside the trust boundary. This is the property IL5/IL6 and air-gapped environments actually require, and it is why this module sits directly upstream of **FT22 (Air-gapped / on-prem deployment)**.
> **The thread, stated as one claim:** Open-data / open-recipe releases are a *compliance asset* in sensitive domains, not merely an ideological preference. Auditability, reproducibility, and supply-chain trust each map to a requirement a regulator can name — and each is satisfiable only when you can see the data and the recipe.
---
## Anti-Patterns
### Trusting a community merge with no provenance
The Hugging Face Hub is full of "merged" models — A merged with B, quantized, re-uploaded. Many have **no documented data lineage, no evals, and a license they probably do not have the rights to grant.** A model card that says "merge of X and Y, works great" is not an audit. In any sensitive deployment, trace the provenance back to the original bases and the original data. If you cannot, treat it as unauditable — because it is.
### Treating OSAID compliance as equivalent to reproducibility
OSAID compliance is a real and useful signal — it means the release is inspectable and the publisher made a good-faith effort. It does **not** mean the release is reproducible, because the data requirement is "sufficiently detailed information," not "the data." A vendor that points at OSAID compliance as proof of auditability is conflating two different things. Ask for the data and the recipe; if they cannot provide it, the auditability claim does not survive.
### Assuming "open" means "safe"
Open data is *auditable*, not *safe*. An open corpus can still contain PII, copyrighted text, poisoned examples, or material that creates its own liability. Openness gives you the *ability* to vet; it does not do the vetting for you. In a sensitive deployment, the open-data base still needs a data audit (FT04–FT07), a PII sweep, a licensing review, and the same red-teaming you would apply to any other base. "It's open, so it's fine" is the open-source version of the cardinal error.
---
## Key Terms
| Term | Definition |
| --- | --- |
| **Open-weights-only** | Tier 1; weights shipped, no training data (Llama 3.x). Usable, not auditable. |
| **Open-data** | Tier 2; weights + training corpus or a reproducible data pipeline (MiniCPM, OLMo, Tülu, SmolLM3). Auditable. |
| **Open-recipe** | Tier 3; weights + data + full training code/config (OLMo, Tülu 3, SmolLM3). Reproducible. |
| **OSAID v1.0** | OSI's Open Source AI Definition (2024-10-28). Requires "sufficiently detailed information" about data, not the data itself. |
| **The OSAID gap** | The deliberate distance between OSAID compliance and actual reproducibility/auditability. |
| **NTIA Open-Model Weights Report** | July 2024 federal report; states open weights provide security benefits for sensitive-data use on-prem. |
| **FMTI** | Stanford Foundation Model Transparency Index; 100 indicators; May-2024 mean 58/100, top 85/100. |
| **OpenBMB** | Tsinghua + ModelBest; the MiniCPM family + Ultra* datasets; Apache-2.0; the course's on-ramp hero. |
| **MiniCPM5-1B** | OpenBMB's ~1B dense open-data base; the Layer-1 example used throughout the course. |
| **Auditability** | The property that you can prove what a model saw — satisfiable only with open-data / open-recipe. |
---
## Lab Exercise
See `07-lab-spec.md`. **"The Openness Audit"** — take five releases (MiniCPM5-1B, OLMo-2, Llama-3.1-405B, SmolLM3, GPT-4o), classify each on the open spectrum (tier, license, data released, recipe released, auditable), and write the one-paragraph HIPAA-deployment verdict using the structured audit template. No GPU required — this is a research/classification lab (~30–40 min).
---
## References
1. **Open Source Initiative (2024)** — *The Open Source AI Definition v1.0*. Released 2024-10-28 at All Things Open. https://opensource.org/ai/open-source-ai-definition — defines "open source AI"; the data clause requires "sufficiently detailed information," not the data itself.
2. **NTIA (2024)** — *Dual-Use Foundation Models with Widely Available Model Weights*. July 2024, mandated under EO 14110. https://www.ntia.gov/programs-and-initiatives/artificial-intelligence/open-model-weights-report — the government authority that open weights "provide security benefits by allowing firms, researchers, and users to use potentially sensitive data" locally.
3. **Bommasani et al. (2024)** — *The 2024 Foundation Model Transparency Index*. arXiv:2407.12929; Stanford CRFM. https://crfm.stanford.edu/fmti/May-2024/index.html — 100-indicator scoring; mean 58/100, top 85/100.
4. **Olson et al. (2025)** — *2 OLMo 2 Furious*. arXiv:2501.00656. https://arxiv.org/abs/2501.00656 — fully-open OLMo 2 family (data, code, weights, checkpoints).
5. **Lambert et al. (2024)** — *Tülu 3: Pushing Frontiers in Open Language Model Post-Training*. arXiv:2411.15124. https://arxiv.org/abs/2411.15124 — open post-training recipe.
6. **Hugging Face (2025)** — *SmolLM3: smol, multilingual, long-context reasoner*. https://huggingface.co/blog/smollm3 — full pre/mid/post + synthetic-data recipe.
7. **OpenBMB / ModelBest** — MiniCPM family (5-1B, 3-4B, V 4.6, o 4.5) + UltraChat / UltraFeedback / Ultra-FineWeb. Apache-2.0.
8. **Software Freedom Conservancy (2024)** — *OSAID Erodes FOSS*. https://sfconservancy.org/blog/2024/oct/31/open-source-ai-definition-osaid-erodes-foss/ — the leading critique of the OSAID data compromise.
9. **Module FT22** — air-gapped / on-prem deployment (the downstream consumer of this module's auditability argument).