In short
A vulnerability assessment searches broadly for known vulnerabilities and ranks them by severity; a penetration test tries to exploit them to show how far an attacker can get.
Scanners produce false positives and miss application logic flaws: results must be interpreted by someone who understands networks and systems.
NIST points to a model of regular scanning interspersed with periodic penetration tests, with extra testing when systems change.
ISO 27001 (control 8.8) requires you to manage technical vulnerabilities; the ACN specifications require essential NIS2 entities to run vulnerability assessments and/or penetration tests on relevant systems.
A test is worth the fixes it produces: priorities, owners, retest and documented acceptance of residual risk.
One tells you which doors might be open, the other actually tries to get in. Here is how to choose between a vulnerability assessment and a penetration test, and how often to run them.
The two terms are often used as synonyms, in supplier quotations and in client questionnaires alike. They are not. A vulnerability assessment searches broadly for the known weaknesses in your systems; a penetration test checks how far an attacker could get by exploiting them. Choosing the wrong one means paying for a result that does not answer your question: a list of hundreds of alerts when you needed to know whether client data can be reached, or an in-depth test of one application when the internal network has never even been inventoried.
In this guide you will find the definitions from the reference sources (NIST and OWASP), a table of the differences, three typical situations to work out which test you need, guidance on frequency and the link with ISO/IEC 27001, NIS2 and the Italian ACN specifications. If you want to dig into web application testing, its approaches and the OWASP Top 10, everything is in our WAPT guide.
What is a vulnerability assessment?
A vulnerability assessment is a systematic search for known vulnerabilities in systems, networks and applications. It answers the question: "where am I exposed, and how badly?".
The most widely cited reference is the guide on security testing published by NIST, the US National Institute of Standards and Technology: SP 800-115. In its section on vulnerability scanning it explains that scanners identify hosts, operating systems, applications and open ports, and compare them against a database of known vulnerabilities. That is how they find outdated versions and missing patches, as well as misconfigurations, and they often suggest how to fix them. Scanning can run from the network or use login credentials to the systems, which let it see much deeper.
What it finds and what it misses
A vulnerability assessment is broad but stays on the surface. It covers many systems in little time, but it stops at the possible existence of a flaw.
NIST itself is explicit about the limits. Scanners have a high false positive rate, meaning they report vulnerabilities that do not exist. They rely on signatures, so they cannot recognise what is not yet in their database. And they do not tell you the overall risk of the network: they assess one system at a time. The OWASP guides add a limit specific to web applications: automated tools are generic, while applications are almost always custom-built, so logic flaws slip through. A scanner does not know that a user of the client portal should not see another company's invoices.
Scanning and assessment are not the same thing
Running a scanner is not yet a vulnerability assessment. What makes the difference is the interpretation of the results by someone who understands networks and systems, as SP 800-115 itself recommends.
In practice, a good assessment removes false positives, recalculates severity in context (a critical vulnerability on an isolated server weighs less than a medium one on the e-commerce site) and delivers a list ranked by priority. A scanner report with three hundred red lines, forwarded as it is to the IT provider, almost always ends up in a drawer. The same list, reduced to twenty actions each with an owner and a date, becomes a work plan.
What is a penetration test?
A penetration test is a test in which specialists simulate a real attack, with authorisation and precise rules, to find out whether and how the defences can be bypassed.
SP 800-115 defines it as security testing in which assessors mimic real-world attacks to identify ways of circumventing the security features of an application, system or network. The part that makes the difference is exploitation: where the scanner merely says a vulnerability might exist, the penetration test confirms that it exists by trying to exploit it. It often combines several minor weaknesses to obtain access that none of them would allow on its own. The result is not a list but a story: where the tester got in, what they reached, with what impact.
The four phases according to NIST
NIST describes the penetration test in four phases: planning, discovery, attack and reporting. Planning sets the rules, the goals and management approval.
The discovery phase gathers information and identifies targets, often using port and vulnerability scanners: this is why a vulnerability assessment is usually part of a penetration test, not an alternative to it. The attack verifies the hypotheses and, when it gains new access, loops back to discovery to explore what was previously invisible. Reporting describes vulnerabilities, risks and mitigation actions. The phases of application testing, with the black-box, grey-box and white-box approaches, are described in the WAPT guide: here we are interested in the general logic.
Scope and rules of engagement
A penetration test carries real risks and must be authorised in writing. NIST points out that it uses real attacks on production systems and data and that the risk of damage can never be fully eliminated.
This is why SP 800-115 includes a rules of engagement template. In practice, before starting you need at least: the exact list of addresses, domains and applications in scope, the agreed time windows, what is excluded (for example tests that could take a service down), an internal contact reachable during the test, and an agreement on how to handle any personal data encountered. If the application is hosted by a cloud provider or managed by a supplier, also check that their terms allow the test and inform them in good time.
Penetration test and vulnerability assessment: the differences in a table
The essential difference is between breadth and depth. The vulnerability assessment covers a lot and digs a little; the penetration test covers less and goes all the way.
Aspect | Vulnerability assessment | Penetration test |
|---|---|---|
Question it answers | Where am I exposed? | How far can an attacker get? |
Method | Mainly automated, with analysis of the results | Mainly manual, supported by tools |
Exploitation of flaws | No, it stops at possible existence | Yes, in a controlled way |
Coverage | Broad: many systems and addresses | Targeted: a defined scope |
False positives | Frequent, must be filtered | Rare, because every flaw is verified |
Application logic flaws | Rarely detected | Part of the test's objective |
Impact on systems | Low, but not zero | Higher, requires rules of engagement |
Output | List of vulnerabilities with severity and fixes | Attack scenarios, demonstrated impact, priorities |
Typical frequency | Regular and recurring | Periodic and tied to changes |
Neither column is "better". NIST points to a programme that alternates regular scans and periodic penetration tests: the former keeps day-to-day exposure under control, the latter checks that the defences really hold.
Which test do you need? Three typical situations
The choice depends on the question you need to answer, not on the name of the service. Here are three frequent situations in SMEs and the most sensible answer for each.
You do not know what you have exposed: start with a vulnerability assessment
Take a distribution company with 60 employees, an on-premises ERP, VPN access for sales agents, a brochure website and a dozen physical and virtual servers. Nobody has ever carried out a full inventory. Here a penetration test would be premature: first you need to know what is there.
An external vulnerability assessment on the public addresses and an internal one with credentials bring to light the unpatched VPN, the network printer with its factory password, the old server nobody remembered. The result feeds the asset inventory and the patching plan, which are also among the minimum cybersecurity measures every SME should start from. The penetration test comes later, once the most obvious flaws are closed and it is worth finding out what remains.
You have an application with client data: you need a penetration test
Now think of a software house with a portal where clients upload documents and consult contracts. The main risk here is not a missing patch but a flaw in access controls: a user who, by changing a number in the address, opens another client's file.
This kind of flaw is exactly what a scanner cannot see and what an application penetration test looks for, especially in grey-box mode, with credentials for different users. It is the typical case for a WAPT. A vulnerability assessment of the infrastructure hosting the portal is still useful, but it does not answer the question that matters to clients.
A client or a tender asks you for "a security test"
The third case is the most common: a client, in its supplier security questionnaire, asks whether you run periodic tests and to attach evidence. Before buying anything, clarify what you are being asked for.
Read the question literally. "Do you perform periodic vulnerability scans?" is met by a documented vulnerability assessment programme. "Has the application undergone a penetration test by independent third parties?" requires a real penetration test, with a report. In both cases you do not need to hand over the full report: a summary letter with scope, date, method and remediation status is almost always enough. We explain how to handle these requests in the guide to client security questionnaires.
How often should you run vulnerability assessments and penetration tests?
There is no frequency that suits everyone: risk decides, together with the obligations you are subject to and the changes to your systems. There are, however, solid reference points.
SP 800-115 observes that, given the cost and potential impact, a network penetration test on an annual basis may be sufficient, provided it is complemented by less demanding activities carried out regularly, such as scanning. It adds that a test is a snapshot of security at a given moment: if your systems change often, it makes sense to test more often. The ACN baseline specifications for essential NIS2 entities, which we look at below, require periodic activities and in any case before going live for relevant systems.
In practice, a reasonable calendar for an SME has two levels. The first is recurring: vulnerability assessments on a fixed cycle, more frequent on internet-facing systems, plus a targeted scan whenever CSIRT Italia issues an alert on a product you use. The second is event-driven: a penetration test before releasing a new application or a major change, after an incident and on a defined schedule for critical applications. Whatever frequency you choose, write it into the procedure and stick to it: for an auditor, what matters is that the cycle is justified and maintained.
What do ISO 27001, NIS2 and other rules require?
None of these frameworks obliges you to choose a specific test by name, with one important exception for essential NIS2 entities. All of them, however, require you to manage vulnerabilities and to check that your measures work.
ISO/IEC 27001: control 8.8
In Annex A of ISO/IEC 27001:2022, control 8.8 covers the management of technical vulnerabilities: obtaining information about vulnerabilities in the systems in use, evaluating the organisation's exposure and taking appropriate measures. Other controls in the same technological theme deal with secure development and application security testing.
The standard imposes neither the vulnerability assessment nor the penetration test. It leaves the choice to you, as long as it is consistent with the risk assessment and the Statement of Applicability. In an audit, what is checked is the process: where vulnerability information comes from, who evaluates it, how quickly fixes are made, how you handle what cannot be fixed straight away. A vulnerability assessment report with its remediation plan, or a penetration test with its retest, are the most convincing evidence. The full picture of the standard is in our ISO 27001 guide.
NIS2 and the ACN baseline specifications
The NIS2 Directive, transposed in Italy by Legislative Decree 138/2024, includes among the minimum measures vulnerability handling and disclosure and procedures to assess the effectiveness of security measures. The baseline specifications adopted by ACN, the Italian National Cybersecurity Agency, go into detail and distinguish between important and essential entities.
For both, they require a vulnerability management plan approved by the management bodies, monitoring of at least the CSIRT Italia channels, and prompt remediation of vulnerabilities or documented acceptance of the risk. For essential entities they go further: on relevant systems they require vulnerability identification activities that include at least vulnerability assessment and/or penetration testing, carried out periodically and before going live, documented in reports describing the activities, outcomes, vulnerabilities and impact. It is one of the few texts that explicitly names both tests. The specifications may be updated: always check the version in force on the ACN website. To find out whether and how NIS2 applies to you, everything is on our NIS2 page.
GDPR and, for the financial sector, DORA
GDPR also comes into play. Article 32 requires anyone processing personal data to have a process for regularly testing the effectiveness of security measures. A documented testing programme is one of the most direct ways of demonstrating it.
For banks, insurers and other financial entities, the DORA Regulation provides for a testing programme that includes, among others, vulnerability assessments and scans and penetration testing, with at least yearly tests on systems supporting critical or important functions. Entities identified by the competent authorities must also carry out advanced threat-led penetration testing (TLPT) at least every three years. If you work as an ICT provider to these entities, expect their requests to reflect these requirements.
From report to remediation: how to make the results useful
A test is worth the fixes it produces. NIST says so clearly: results should be taken seriously, vulnerabilities mitigated and outcomes presented to management.
The first step is triage. Each vulnerability gets a priority that takes into account technical severity and context: how exposed the system is, what data it holds, whether a known exploit already exists. Then each action is assigned to a person with a date. What you cannot fix, for example machine software the manufacturer no longer updates, must be isolated and the residual risk accepted in a documented way, not forgotten.
The second step is verification. After the fixes you need a retest, at least on the most serious vulnerabilities, to confirm that the flaw is really closed and that the fix has not opened others. Without a retest, the report describes the state of a day in the past.
The third step is keeping the evidence. Report, remediation plan, retest outcome and risk acceptance decisions are documents needed for the ISO audit, for NIS2 and for clients. Keep them with date, version and scope, and keep the full technical report, to be shared only with those who must fix things, separate from the summary for external use.
The most common mistakes
Some mistakes recur in almost every company that is starting out. Recognising them saves you a round of testing.
Calling an automated scan a penetration test: the report does not survive the first question from an experienced client or an auditor.
Defining the scope vaguely: the test covers the brochure site and leaves out the reserved area, where the data is.
Testing once and never again: a test from three years ago says nothing about today's systems.
Skipping the retest: the fix remains an intention, not a verified fact.
Treating the report as the goal: the aim is to reduce risk, not to file a document.
How Complaion helps you with vulnerability assessments and penetration tests
Complaion helps you choose the right test and make it part of your management system, not an isolated document.
Your dedicated Lead Auditor starts from the risk assessment and the obligations that apply to you (ISO 27001, NIS2, client requests) and proposes the path that suits you best: a vulnerability assessment programme, a penetration test on critical applications or both, with a justified frequency to write into the procedure. When the report arrives, they help you link findings, remediation plan and residual risk acceptance to control 8.8 and the NIS2 measures, so that in the audit you present a process, not an attachment.
The platform collects reports, remediation plans and retest outcomes as evidence, manages procedures under version control and keeps the schedule of periodic tests, so you know when the next assessment is due and what is still open. When the audit approaches, your evidence is already in order. For web applications we also carry out the WAPT, the application penetration test, and if you are building your management system you can do it within the ISO 27001 certification path. This is how we work with more than 1,000 clients. To find out which test you really need, talk to a Lead Auditor.
Frequently asked questions about penetration tests and vulnerability assessments
What is the difference between a vulnerability assessment and a penetration test?
A vulnerability assessment searches broadly, and mainly automatically, for known vulnerabilities in systems and applications, and ranks them by severity. A penetration test simulates a real attack and tries to exploit vulnerabilities to find out how far an attacker would get. The first measures exposure, the second demonstrates the impact. A penetration test often includes a scanning phase within it.
Can a vulnerability assessment replace a penetration test?
No, because it does not exploit vulnerabilities and is poor at detecting application logic flaws, such as faulty access control. The reverse is also true: a penetration test on a limited scope does not replace regular scanning of all systems. Reference sources such as the NIST SP 800-115 guide suggest combining the two approaches.
How often should a penetration test be carried out?
It depends on risk, obligations and how much your systems change. The NIST SP 800-115 guide observes that an annual network test may be sufficient if complemented by regular scanning. In any case it makes sense to test before putting a new or significantly modified application into production, and after an incident. What matters is that the frequency is written down and respected.
Is a penetration test mandatory for ISO 27001?
No, ISO/IEC 27001 does not impose a specific test. Control 8.8 requires you to manage technical vulnerabilities: gather information, evaluate exposure and take action. You choose how, based on your risk assessment. A periodic vulnerability assessment or a penetration test with retest are, however, very effective evidence in an audit.
Does NIS2 require vulnerability assessments or penetration tests?
NIS2 requires vulnerability handling and checking the effectiveness of measures. In Italy, the ACN baseline specifications require all entities to have a vulnerability management plan. For essential entities they also require, on relevant systems, activities that include at least vulnerability assessment and/or penetration testing, carried out periodically and before going live. Always check the version of the specifications in force on the ACN website.
Can a penetration test bring systems down?
Yes, the risk exists and cannot be fully eliminated, as NIST points out. It is reduced by written rules of engagement: a precise scope, time windows, exclusion of the most invasive tests on critical systems, an internal contact who can be reached. For very sensitive applications you can test a staging environment identical to production. A vulnerability assessment also has a low but not zero impact on the network.
Can I give clients the penetration test report?
Better not, at least not the full one: it describes your vulnerabilities in detail and is a document useful to an attacker. Clients are usually satisfied with a summary letter covering scope, date, method and remediation status. If a client has a particular need, consider protected sharing under a non-disclosure agreement or a review by video call.
