Test catalogue

Procedure Place the product on the lab network and record which services answer during the scenarios in the scope.

Acceptance Pass only if every service that answers is needed for the functions in the scope.

What is examined Which services the product exposes on the network it joins, and whether anything is reachable that the product does not need.

Procedure Follow the supplied setup and note how a person becomes the owner, including any credential that is present before that.

Acceptance Pass when ownership is bound to the intended account and a pre-set credential is not left in control of the device.

What is examined How the product decides who may control it, including default credentials and how ownership is bound to a person or account.

Procedure During ordinary setup, check whether an account that is not the owner's can add the device through the supplied app or hub.

Acceptance Pass when only the intended owner can complete pairing.

What is examined Whether someone who is not the owner can add the device to their account, app, or hub during setup.

Procedure Observe traffic between the device, hub, app, and the services it uses. The lab does not break encryption.

Acceptance Pass when that traffic is protected in transit, as far as observation can show.

What is examined Whether traffic between the device, hub, app, and the services it uses is protected in transit, as far as the lab can observe.

Procedure Record the hosts contacted during the agreed ordinary-use scenarios.

Acceptance Pass when the destinations match what the product needs, and unexpected hosts are absent or explained in the scope.

What is examined Which hosts the product contacts during the agreed scenarios, and whether unexpected destinations appear in ordinary use.

Procedure Look for an update path, note who can start it, and observe what the product does if an update is interrupted or cannot be reached.

Acceptance Pass when an update path exists and the product stays in a known state if an update does not finish.

What is examined Whether an update path exists, who can start it, and what the product does if an update is interrupted or cannot be reached.

Procedure Reset the unit by the method supplied to an owner, then check whether the previous owner, credentials, and local data are gone.

Acceptance Pass when a later owner cannot use the previous owner's access, and that owner's local data is no longer available.

What is examined Whether a reset clears the previous owner, credentials, and local data well enough that the unit can be handed on.

Procedure Without the owner's credentials, use the ordinary controls for settings that affect safety, privacy, or connectivity.

Acceptance Pass when those settings stay unchanged.

What is examined Whether settings that affect safety, privacy, or connectivity can be changed by someone who is not authorised.

Procedure Set up a new unit from the supplied instructions, with the app and hub named in the scope.

Acceptance Pass when the unit reaches the working state those instructions describe.

What is examined Whether a new unit reaches a working state from the supplied instructions, with the app and hub named in the scope.

Procedure Produce the events the product claims to detect, under the conditions in the scope, and record what is reported.

Acceptance Pass when the reported events match the claim for those conditions.

What is examined Whether the product reports the events it claims to detect, under the conditions agreed in the scope.

Procedure Measure the time from a stimulus or command to the result a person can observe. Write down the method and the conditions.

Acceptance Pass when the measured time is within the limit agreed at scoping. If no limit was agreed, the report gives the figure and the method.

What is examined The time from a stimulus or command to the result a person can observe. The report includes the method and the conditions.

Procedure Leave the product with no intentional use for 24 hours, then check that it is still available and consistent.

Acceptance Pass when availability and reported state hold across that period.

What is examined Whether the product stays available and consistent for 24 hours when nobody is using it.

Procedure Remove the upstream connection, observe the device, hub, and app, then restore the connection.

Acceptance Pass when behaviour during the outage and after restoration matches the criterion agreed at scoping.

What is examined What the device, hub, and app do when the upstream connection is removed, and what they do when it returns.

Procedure Cut power to the hub or gateway, restore it, and record how the hub and the devices that depend on it behave.

Acceptance Pass when the hub and dependent devices return to the agreed state. N/A when the product has no hub.

What is examined What the hub or gateway does after a power cut, and how devices that depend on it behave. N/A when the product has no hub.

Procedure Drop the wireless network and bring it back. Record whether control and reported state return.

Acceptance Pass when control and state recover as agreed.

What is examined What happens when the wireless network drops and comes back, including whether control and reported state recover.

Procedure Compare the state shown by the device, the hub, and the app after ordinary actions and after the interruptions above.

Acceptance Pass when those views agree.

What is examined Whether the device, hub, and app agree on the state a person can see after ordinary actions and after the interruptions above.

Procedure Read what the supplier states, in public or in the documents provided, about how long updates and support last.

Acceptance Pass when a support period is stated clearly enough for the decision in the scope.

What is examined What the supplier states, in public or in the documents you provide, about how long the product will be updated and supported.

Procedure Look for a published way to report a vulnerability, and for what the supplier says happens after a report.

Acceptance Pass when a reporting path is published and the next step is stated.

What is examined Whether there is a published way to report a vulnerability, and what the supplier says happens after a report.

Procedure Compare the privacy notice with what the product and app do, and follow the deletion path an owner is given.

Acceptance Pass when the notice matches what is observed, and the owner's data can be deleted by the path described.

What is examined What personal or household data the product and its app handle, and whether that data can be deleted, from observation and the privacy notice.

Procedure Review supplier documents and the observable update behaviour for what they show about update integrity. This is not a full firmware audit.

Acceptance Pass when the documents and the observed update behaviour support the integrity claim in the scope. Missing evidence is not rewritten as a pass.

What is examined What can be shown about update integrity or known vulnerable components, from supplier documents and from observable update behaviour. This is not a full firmware audit.

Procedure Use the product with the hub, app, or ecosystem named in the scope, for the functions the decision depends on.

Acceptance Pass when those functions work in that ecosystem.

What is examined Whether the product works with the hub, app, or ecosystem named in the scope, for the functions the decision depends on.

Procedure Remove the unit and set it up again, including a change of owner or site when that is in the scope.

Acceptance Pass when the unit can be commissioned again and the previous association is gone.

What is examined Whether a unit can be removed and set up again, including after a change of owner or site.