Method

How an assessment runs

One product, a fixed catalogue, and a report that separates what was observed from what the lab is willing to recommend. The work is independent product assurance. It is not official certification and it is not a Cyber Resilience Act conformity assessment.

Sequence

Five steps, the last one optional

Scoping is part of the work, not a formality. The recommendation is only as wide as the decision you named and the unit the lab had.

  1. Scope

    You describe the product, the ecosystem around it, and the decision the report must serve: ship, source, or deploy. The lab confirms what is in the box and what will not be tested.

  2. Intake

    One unit, the companion app, a hub if the product needs one, a test account you control, and any supplier documents you already have.

  3. Lab work

    The 22 cases. A case is marked N/A only when the architecture makes it inapplicable. Observations are kept with the evidence that supports them.

  4. Report

    A 2-page decision summary, a 10–15 page technical report, and machine-readable results. The recommendation is one of the four statements below.

  5. Retest, if you want one

    After you remediate, a shorter pass over the cases that failed or were inconclusive, and an updated summary.

From you

What the lab needs

  • The product sample, including a hub or gateway if normal use needs one.
  • The companion app, and a test account the client controls.
  • Supplier statements already in hand: support period, privacy notice, vulnerability contact, update policy.
  • The decision the report must serve, in a sentence: ship, source, or deploy.

Not this engagement

What stays out of scope

  • A conformity assessment under the Cyber Resilience Act, CE marking, or the Radio Equipment Directive.
  • Testing of cloud services you do not control.
  • Source-code review, or a full firmware extraction and audit.
  • Electrical safety, radio performance, or environmental testing.
  • A production sampling plan. The work covers the unit or units you supply.
  • A single score that collapses the catalogue into one number.

Specimen

The shape of the report

This is the structure of the documents, not a result. There is no client, no product, and no verdict in the example. Bracketed text is a slot the real report fills.

iottest.nu · decision summary Specimen · 2 pages

Page 1

Decision summary

Product [product name supplied by the client]

Ecosystem [sensor, hub, app, as named in the scope]

Decision [ship, source, or deploy]

Recommendation [Proceed / Proceed with conditions / Do not proceed pending remediation / Insufficient evidence]

Conditions [each one tied to a case ID, or none]

Limitations [what bounds the recommendation]

Not tested [cases in scope that were not run]

iottest.nu · technical report Specimen · 10–15 pages

One case, as it appears in the report

Case SEC-01 · Network exposure

Verdict [Pass / Fail / N/A / Not Tested / Inconclusive]

Observation [what was reachable, from where, in the agreed scenarios]

Evidence [capture or measurement reference]

Criterion [the pass/fail line agreed at scoping]

Mapping [related theme, for orientation only — not a conformance claim]

The rest of the technical report is scope and setup, method notes, the same block for every case, reliability figures with their method, the document review for GOV-01 to GOV-04, and the limitations again.

Machine-readable results

One record per case. JSON is the default file. The record does not include a combined score, because the lab does not produce one.

{
  "product": "[name supplied by the client]",
  "decision": "ship | source | deploy",
  "recommendation": "Proceed | Proceed with conditions | Do not proceed pending remediation | Insufficient evidence",
  "cases": [
    {
      "id": "SEC-01",
      "title": "Network exposure",
      "verdict": "Pass | Fail | N/A | Not Tested | Inconclusive",
      "observation": "[what was seen]",
      "evidence": ["[reference]"],
      "limitations": ["[what bounds this result]"]
    }
  ]
}

Vocabulary

Words the report is allowed to use

Decision summary, Technical report, Machine-readable results — and the lists below. A Proceed recommendation covers this catalogue, the decision in the scope, and the unit tested. It is not a statement that the product is secure in general.

Verdicts

  • Pass The observation met the criterion agreed for that case.
  • Fail The observation did not meet the criterion.
  • N/A The case does not apply to this product’s architecture. Hub power recovery is N/A when there is no hub.
  • Not Tested The case was in scope but was not run.
  • Inconclusive The case was run, and the evidence does not support Pass or Fail.

Recommendation

  • Proceed No failed case blocks the decision named in the scope. This is not a statement that the product is secure in general. It covers this catalogue, this decision, and the unit tested.
  • Proceed with conditions The decision can go ahead only if named conditions are met. Each condition cites a case ID.
  • Do not proceed pending remediation A failed case blocks the decision until it is fixed and, where agreed, retested.
  • Insufficient evidence The lab cannot support a proceed or do-not-proceed recommendation from what was supplied or what could be observed.